search.cpan.org has an RSS feed for recently uploaded modules but there's only one minor problem: the feed doesn't have rich metadata.
Daisuke Murase (aka typester on CPAN and IRC) created a site called CPAN Recent Changes a while ago and it's been really useful for people tracking activities on CPAN.
The feature the site provides is very simple: "a better recent change log for CPAN". The site tracks the recently uploaded modules from search.cpan.org and grabs Changes file and takes diff against the previous version, so you can see what's changed in the release (unless the author is too lazy to update the Changes file). The site of course publishes the RSS feed for the recent uploads to CPAN, with the changes in the summary field, so you can keep an eye on it without clicking the link to see what's changed.
You can also follow changes in different views, like modules under the specific namespaces (e.g. Catalyst or DBIx-Class) or modules uploaded by specific authors (e.g. me or Ingy) and they all come with RSS feeds too.
