XML::RSS::TimingBot

TorgoX on 2004-04-07T05:48:49

Dear Log,

After my endless testing and bugswatting, XML::RSS::TimingBot is basically useable now. It's an LWP::UserAgent subclass that stops you from requesting RSS feeds more often than they say they'll update. I.e., when you request a feed, the module transparently looks for its <ttl> elements and the like, and the next time you request it, if the previously-read information says that the feed won't updated by now, the module won't actually perform the request, but will generate a virtual response that looks like the server said "304 Not Modified" with no content. It also keeps track of last-modified times on RSS responses and inserts "If-Modified-Since" headers on subsequent outgoing responses. It's all very behind-the-scenes and transparent.

I intend this to be pretty much a drop-in replacement for LWP::UserAgent in RSS-fetching programs, and so I did just that -- I replaced "use LWP::UserAgent; $browser = LWP::UserAgent->new" in my personal RSS aggregator with "use XML::RSS::TimingBot; $browser = XML::RSS::TimingBot->new". And it worked!

A few more days of stress-testing just to be sure, and I'll release it on CPAN, along with XML::RSS::TimingBotDBI. (XML::RSS::TimingBot stores its state in a flat file database, but XML::RSS::TimingBotDBI overrides some methods so that the data is stored in a user-specified SQL database. The MagNet #perl folks helped me stay sane thru all the SQL today, because they are superstars.)