perl modules, IIS, RSS, and optimization

LTjake on 2003-06-13T19:22:10

The RSS aggregator that I've been talking about is moving along well.

There's one catch. It's running on IIS and it seems to drive the CPU usage to 100% whenever it runs. Ack.

The following modules are loaded via use statements in my app:

  • CGI::Application
  • CGI::Simple
  • Cache::FileCache
  • XML::Simple
  • XML::RSS

Of course, this means that there's more behind the scenes (like HTML::Template for CGI::App). I think my biggest problem was that, for every feed in the config file I was creating a new XML::RSS object. I've changed that to limit only the feeds that are to be displayed. I've also added the feed title to the config file so I don't have to load any feeds on the customization screen.

I'm considering further optimization by pre-parsing the feeds - but, since the user can select an arbitrary "max items" number, I'll have to generate one for 1..scalar @items. But that would eliminate the need to load XML::RSS and create a new object per feed.

I still need to make some other changes like:

  • stripping html from descriptions (not a problem right now because they're all local feeds)
  • properly handling dates
  • dc:subject handling