My friend Jeff woke me up today because I IM'd him late last night saying "I have this great idea and I need to talk with someone about it!" This isn't that idea, but another one Jeff suggested.
We got to talking about the problem of CPAN dependencies. He mentioned that Python modules don't really have many dependencies, but they're not very complex either. CPAN modules can do some really complex things because they can let some other module do the work, but that leads to a long, and fragile, dependency chain.
We talked about some of the attempts to solve that and why they suck. They mostly revolve around depending on a fixed version. Debian packages can depend on a major version of another package (say, 3.x) and won't use a newer major version. Module::Install tries to solve it by shipping a fixed version of itself and any dependencies. only.pm tries to solve it by coding in specific versions. The problem with these approaches is they try to freeze the upgrade process.
The idea that modules get better over succeeding revisions is basically correct in the long run. Software is not wine, old versions don't get better with age. We just adapt to their quirks. The more adapted we are to a particular version's quirks the harder it becomes to upgrade. The dynamic upgrade environment avoids this sort of particularly nasty bit rot.
Jeff explained Debian's system and said it's to avoid API changes. Debian policy doesn't allow incompatible API changes within a major version. But that's not really CPAN's problem, they're usually bugs. With that realization, Jeff pointed out the key thing: the problems are *temporary*.
A single version of a CPAN module has a bug which breaks your dependency chain, but the next version will probably fix it. Or the next one, or the next one. Eventually, the march of versions forward will fix the problem. So hard coding "only use version 1.34" into your module takes a temporary problem and applies a permanent halt to upgrades.
So the solution should also be temporary and only last as long as the problem exists. Wouldn't it be neat if there was some way to flag the upgrade of a dependency to be problematic? Wouldn't it be neat if there was a way for the CPAN shell to query a service and request the advisories about a particular module? It's like traffic advisories.
"...and here's Michael Schwern with the CPAN traffic report. Michael?"
"Bill, I'm flying over the latest RSS feed and DBI is jack knifed across two lanes of dependency traffic. It looks like a bug fix in Test::More 0.71 revealed a flaw and caused DBI's test suite to skid. A couple of XML modules got tangled up in that crash, the patching crews expect to have them fixed by tomorrow evening. Those of you thinking about installing DBI should avoid the Test::More 0.71 and take the 0.70 instead today. We'll have more updates at the top of the hour. Over to Pudge with the sports. Pudge?"
"Today the Red Sox..."
What happens is when there's a dependency problem, say a bug fix in Test::More reveals test failures in DBI (which it does), the author of the dependent module (Tim) can report a CPAN Version Advisory that DBI 1.59 and Test::More >= 0.71 are incompatible. When a user goes to install DBI the CPAN shell queries the service for any advisories. It uses that information to decide what to do. In this case it might say "DBI 1.59 is incompatible with your installed version of Test::More (0.71). Would you like to downgrade to 0.70?" It could even get clever and note that a given module is only a build requirement and temporarily downgrade just for the build and tests.
You could even go so far as to have the advisories contain a patch.
Later on, when DBI puts out a fixed version and the dependency conflict is resolved, Tim can remove the advisory and dependency resolution marches forward as normal.
The advantages this has over hard-coding this information into, say, DBI's own Makefile.PL is that it's orthogonal to module releases. The problem occurs between DBI releases. DBI would have to release a new version just to say "I don't work with Test::More 0.71" and that's a lot of bother. It also does not require the author to issue the advisory. The author of the dependency could issue an advisory, or a trusted 3rd party. This removes several bottlenecks from the process.
It would seem to relieve much of the day-to-day problems with failing CPAN dependencies these days while hanging onto the basic, forward looking assumption that latest version good.
