In the CPAN modules I maintain, three use both ExtUtils::MakeMaker and Module::Build and the most recent only uses Module::Build. I find Module::Build easier to work with and it has advantages that I use. I don't like having to maintain and update both of them, so my most recent module only has Module::Build.
To improve my testing I use several optional modules, and use the build process to probe for and only run tests that can be run - rather than getting the tests to skip.
People doing smoke tests on my modules report false positives because they try to run tests that can't be run. I'll put steps in place to prevent that from happening, but in the mean time I wonder do people think that both build processes will hang around in parallel for ever? will one replace the other? which one will become "standard"?
Re:current standard
chromatic on 2007-02-06T23:20:13
Why is handling testing dependencies best done in the Makefile.PL/Build.PL?I believe that's so that META.yml can express dependencies required for the build properly.
What are the advantages over skipping tests?CPANTS penalizes you for not shipping POD checking tests, but their value to anyone but developers is minimal, for example.
Re:current standard
Alias on 2007-02-07T02:59:24
> Why is handling testing dependencies best done in the Makefile.PL/Build.PL?
It's not so much "testing" as "checking".
In multi-platform source packages (like CPAN packages) the dependencies of a package require turing-completeness to determine. That is, a dependency on your platform might be different to the dependencies on some other platform.
Part of the job of the Configure phase (Makefile.PL and Build.PL) is to resolve the Complete Dependencies down to a set of Static Dependencies. That static list can then be picked up by the parent CPAN client and recursed to install the dependencies that the source package needs to be able to build on that platform.
> What are the advantages over skipping tests?
As we've gotten more and more into testing, we are realising that there are multiple classes of tests that we should be adding to modules.
Most of these test basic functionality and so we need to execute them in all testing scenarios.
But it's worthwhile adding other forms of testing to the modules. As chromatic points out, there are things like POD tests.
Documentation quality tests make NO difference to the functionality of the module itself. When installing a module for production use, these tests are meaningless. To run them only creates more chances for unexpected false failures and increases the number of dependencies a module has. And dependency bloat is one of the great evils of CPAN-like systems, because it increases the complexity of the repository and means there's more chances for edge case errors to occur.
The POD tests should, however, be run by the author on release (Release Tests) and should probably also be run on any automated mass-testing systems.
Another class of tests are what you might call "unweildy tests". They allow for more-intensive testing, but they would place a heavy burdon on the user.
For example, many database-related modules now include tests that only run on automated testing environments. These will add a dependency on DBD::SQLite, or other heavy modules, and do more thorough testing.
But in the author's opinion, he has decided that a module which might be used only for MySQL shouldn't need to install SQLite JUST to do testing.
A third class is optional features. Some modules will interact with CPAN modules IF they are installed, but don't need those modules to work normally.
Adding a dependency makes no sense here, so the tests ONLY run if the optional module is available.
ExtUtils::MakeMaker will live forever, at least until you can upgrade all those systems with 5.5_003 (Solaris 8) or 5.8.4 (Solaris 10) deployed, or stop caring about the users of "old" versions of Perl. Whatever the goal of the Module::Build authors is, I hope that people will not use Module::Build where they can just as well use EU:MM.
