I do not think prerequisites are not the problem

brian_d_foy on 2003-01-22T03:40:41

Dave Cross, volunteer manager of the NMS project, thinks the dearth of Perl applications may be that anything good may require too many external modules from CPAN. Although "too many" is relative, his audience does not want any. He gets to deal with the people who want "turn-key" CGI scripts for free, which means his customers make a lot of demands without offering appreciation, thanks, or anything in return. They want high quality things for free with no obligation---indeed, this is the reason the original developer stopped supporting the scripts that NMS replaces.

I do not think prerequisites themselves are the problem. A lot of other things, like fink, apt, /usr/ports, and other things deal with it quite easily. It is mostly painless to install applications with those tools, even if things have to go into many different directories.

If it is not the prerequisites, what else could it be? First, Perl has to portable. Tools that work for a particular platform ( fink with Mac OS X, /usr/ports for FreeBSD, etc. ) can offer packages which work around problems on that platform. Perl packages, usually those implemented with XS and C, have to work for all of the platforms. Second, the Perl community enforces no quality assurance on its repository. The CPAN Testers do test almost every release on a variety of platforms, but the data is mostly academic since CPAN will store and CPAN.pm will try to install a module no matter the result from CPAN Testers.

I think the complaint about prerequisites is more a complaint about playing russian roulette with module quality. Perl cannot easily support multiple versions of the same modules at the same time. The first module with the right name is the one that Perl uses, and it finds them based on a combination of directory and file names. Perl also assumes that any version greater than a specified version is good enough, even though in some cases, like CGI.pm and its constantly changing default behavior, I would like to use only a particular version (or even any previous version). Places like McDonald's work because people know what they are going to get, even if it is unhealthy and not the best quality. With CPAN, new users have a lot to fear because the quality of the modules vary greatly, and thus, more prerequisites are more chances for trouble. I think the prerequisite problem would mostly disappear if the quality of CPAN modules, including mine, were better and consistent.

An often complaint is that a person who does not have root access cannot install modules. Although ExtUtils::MakerMaker can handle this just fine, not many people take full advantage of it. Any user should be able install Perl modules as long as they can create files. If we expect that though, we should start putting README files back into distributions (and that includes me) that explain the installation process, how to set PREFIX, and other instructions hidden in the ExtUtils::MakeMaker documentation (for instance how many people use prompt() in Makefile.PL?). I will write a README that anyone can use---expect it next week. Module authors can also see installation instructions on CPAN.

Another complaint is that a person only has FTP access or no shell access. In that case, Perl is not different from most other things. I think this complaint is mostly a red herring---it comes mostly from people trying to use free or very low cost web hosting services. The web space is only part of Perl, and those trying to do it on the cheap have set themselves up for failure, or at least limited success. As developers we should be sensitive to this issue, but not shackled by it.

A lot of the Mac OS X desktop software that I use now simply says "drag this to the Applications folder to install", including simple shareware programs to the overly complex Office X. Perl applications (like Moveable Type) can do this if they do not use any uncompiled C code, and with a bit more work provide packages with pre-compiled binaries for various platforms, just like other applications need to do to support multiple platforms. Dan Sugalski says that Perl 6 may ease this a bit with distributable bytecode files, but I am dubious since Java people still endure the headaches.

We cannot change the users. Even if we educate a person every time a person has a problem, we will always encounter new people. Education does not scale. Since programmers are typically not customer service oriented, they forget that even though they have heard a question a hundred times, the person at the other end has only asked it once. We cannot expect people to know where to find all of the answers without be taught where to look. Even if we want to act like geeks, we have to remember that not everyone wants to be a geek, and not everyone wants to use a unix-ish environment.

We, as Perl developers, need to take care of the entire software development process. Most programmers, no matter the language, focus on the least time-consuming portion---the actual coding. We need to provide documentation, installers, and so on. Even the stuff I saw in NMS do not have install scripts, and the installation instructions come after the script documentation. the steps involved can be mysterious to end users. I am guilty of the same thing. Lately I have started to put my scripts into full-fledged Perl distributions---ExtUtils::MakeMaker is just not for modules. It can handle distributions that only contain scripts. It can install the script in the right place, manify the documentation, set the script permissions, set up the initial configuration, and more, but only if we take the time to learn to use it and then decide the problem is important enough to work on. After a couple of years of hacking ExtUtils::MakeMaker, I am able to take care of most of the fear, uncertainty, and doubt customers had about installing Perl applications and their associated modules. I have not done much to pass on my knowledge though. Expect that to change too, especially since I now have my own press.

The module installation process is mostly painless with CPAN.pm, but it does have some problems, and some of them that we cannot fix. First, CPAN.pm installs the latest version if it needs to install a module, rather than the particular version I might specify in PREREQ_PM. The CPAN does not even necessarily know about all versions of a module since users, including me, delete old versions to keep the size of CPAN low and to prevent people from using buggy versions. The BackPAN maintains everything uploaded, but CPAN.pm does not download from it. Indeed, my habit lately has been a MINICPAN

Second, when CPAN.pm fails, it fails horribly and uselessly. There is no good way to tell the novice user what to do next or to whom they should complain. If someone tries to install a module with nested dependencies, the uninitiated end user will have no idea the number of authors involved, who those authors are, or how to get in contact with them. A standard "Complain to the author" mechanism could help this, and perhaps a graphical Perl installation tool could show people their options. The CPANPLUS and Module::Build efforts have ideas on improving this situation.

Third, as a community, we do not mentor module authors. We let them upload modules in any shape and size. The CPAN Testers test it, but that usually does not go much beyond reporting failures. When something does go wrong with a module, people tend to think that the author should know a lot of stuff that they apparently do not know about (mostly from a simple lack of experience), and treat them harshly. We have a lot of documentation about module mechanics, object-oriented programming, and syntax, but not practical advice on doing the deed. Sam Tregar's recent book, Writing Perl Modules for CPAN, is the only book I know that has as its core writing Perl rather than Perl.

Fourth, people need to disassociate open source, gratis, or libre software from full-time, compensated endevours. If someone pays $500 for software, they have a right to complain and they know who to complain to---the name on the check. If someone downloads a Perl module and has trouble with it, the first thought that comes to mind is most likely not "this could be a really good project if the developers could work on it more and not have to think about rent and food". However, CPAN is not for shareware or code that requests a fee of any sort. This is not an excuse for module authors to be sloppy, but for people to realize they benefit from a volunteer effort that they, in general, do not support. Even donations to Perl Mongers or Yet Another Society do little to help because hardly any of the 2,500 PAUSE users get some of that money. Maybe a micro-grant program could fix this---fix this bug for $100---make it possible for the right people to spend time on something. (I thought Collab.net was supposed to facilitate this sort of thing, but apparently has a completely different business model now.)

My solution? Module authors have to take better care of their users by anticipating their problems and taking care of them---better installation support, better documentation, and more testing. I picked on Dave a little, but I suffer from the same things---NMS just happened to be a handy target because they are doing good work over there and I know Dave cares about quality. He has already raised the quality of those scripts quite a bit.