That would be amazing! I think there have been at least a couple crawlers written for it. (One being SearchYC's.) I'll let you know when I find something more concrete.
Is there anything else that would help make the ThriftDB setup go more smoothly?
Kartik is happy to contribute his crawler, but wonders if it wouldn't be easiest to reuse the HNSearch crawler (since both forums have the same structure). http://arclanguage.org/item?id=14774
Thank you, Andres. I put up a message about it on the Arc Forum as well: http://arclanguage.org/item?id=14772. Please email me at evanrmurphy@gmail.com if you need assistance.
Edit: Fixed link to have correct URL. Sorry for the mistake.