The idea needs a lot of work, but imagine using CPAN::Mini::Extract to extract the tests for distributions which you currently installed, thus leaving you with a set of tests for what you really use. This would make it easier to check for bugs. It would be an extremely valuable tool for many people.
There are plenty of problems here, including core distributions, distributions without tests, determining which distributions a module is from, determining version numbers, and so on. However, a partial solution is better than none.
I've not started it, so feel free to write it :)
