Roll your own real-time twitter search with thrudb
I’ve been using twitter quite a bit lately and really like the simplicity of the service and api. One thing thats missing though is search, but there are some great sites like tweetscan and summize that let you search public tweets in close to real-time.
I decided indexing twitter is a great application for thrudb, specifically the thrudex service. Thrudex is essentially a Thrift service for CLucene with some special sauce added. If you’d like to read about the inner workings read this.
Anyway, I whipped up a demo (in perl) for a realtime twitter search and have indexed a few days of tweets (over 3 million!) . check it out here.
One of our regular contributers, Thai Duong, was kind enough to port it to python+django for you new school folks.
*Note* this is running on a single dev box, so be forgiving… It’s currently polling the public timeline feed so it’s not going to catch every tweet. but It captures ~85%
We’ve added the code as a tutorial for thrudb here. Take it and build your own service… Any takers on building a ruby version or a cross site social search aggregation ala friendfeed?
![[del.icio.us]](http://3.rdrail.net/blog/wp-content/plugins/bookmarkify/delicious.png)
![[Facebook]](http://3.rdrail.net/blog/wp-content/plugins/bookmarkify/facebook.png)
![[Reddit]](http://3.rdrail.net/blog/wp-content/plugins/bookmarkify/reddit.png)
![[Slashdot]](http://3.rdrail.net/blog/wp-content/plugins/bookmarkify/slashdot.png)
![[StumbleUpon]](http://3.rdrail.net/blog/wp-content/plugins/bookmarkify/stumbleupon.png)
![[Twitter]](http://3.rdrail.net/blog/wp-content/plugins/bookmarkify/twitter.png)
Add New Comment
Thanks. Your comment is awaiting approval by a moderator.
Do you already have an account? Log in and claim this comment.
Add New Comment