Wednesday, December 05, 2007

Utilizing Ruby and Linux

I've recently been working on a site that aggregates a large amount of data. The data is made available at arbitrary times and delivered via xml feeds over the web. I needed to check for new content at timed intervals and I needed a mechanism for delivering a large amount of data quickly.

The first problem was easily solved by writing a ruby script that checks all the sources for new content and setting up cron to run the script every 15 minutes.

Making large amounts of data quickly available was solved by using Rails page caching, but the first request was still taking about a minute to serve. That issue was also easily solved by sending a curl request after each time the cache is swept.

One of the reasons that I love working with Ruby/Rails* is that not only do I have all the tools Ruby provides, but I also have easy access to the tools available on linux.

Next time someone asks me why I like Ruby more than other languages, I'll have to remember to add this to the list of reasons.

*Most languages give access to the underlying OS, but I find Ruby's access to Linux to be more pleasurable than other experiences in my past.
Post a Comment