What is it about crypto modules that makes them almost universally badly packaged?
Fair enough, crypto can be complicated, but is there some inherent property of that area of programming that makes them so much of a mess?
Many of the classic packaging anti-patterns are displayed in the various depths of the dependency chains of modules like OpenPGP or Net::SSH::Perl, both of which are more or less impossible to install.
Platform Bias
For Windows people, this is huge. The crypto modules are littered with unix assumptions and close-minded authors. In at least one case, when the Strawberry Perl compatibility team tried to contact the author to help him add support for Win32, they were told that "You people go back to your stupid Winblows. There's no way I'm adding support for it".
With the gpg command line generally not available on Windows, and the Perl OpenGPG module unable to install because of people like the above, this completely torpedoed any chance of having CPAN clients do security checks by default, or check signed modules.
Knowledge Bias
Having "options" in your Makefile.PL is almost universally bad.
Now, sometimes Makefile.PL script cannot auto-configure and need to ask for things like library paths, or your ORACLEHOME path or what have you. This is just fine, since it's a last resort.
But have a look at some of the questions for Net::SSH::Perl.
------------------------------------------------------------
As of version 1.00, Net::SSH::Perl supports both the SSH1 and
SSH2 protocols natively. The two protocols have different
module prerequisitives, so you need to decide which protocol(s)
you plan to use. If you use one or the other, only those modules
for your chosen protocol will be installed; if you choose both,
all of the supporting modules will be installed. Please choose
the protocols you'd like to use from the following list ("Both"
is the default).
[1] SSH1
[2] SSH2
[3] Both SSH1 and SSH2
Which protocol(s) do you plan to use? [3]
Some of the Net::SSH::Perl ciphers depend on a Crypt:: module from
CPAN. You may already have the necessary modules installed, in which
case you don't need to bother with this step. Otherwise you'll need
to install at least one cipher to use Net::SSH::Perl. Please choose
at least one from the following list (Crypt::IDEA is the default).
[1] IDEA
[2] DES
[3] DES3
[4] Blowfish
[5] RC4
Enter your choices, separated by spaces: [1]
Checking for optional modules
Crypt::RSA is required if you wish to use the ssh-rsa public
key algorithm (ssh-dss is used by default).
Would you like to install it? (y/n) [y]
--------------------------------------------------------
These sorts of questions assume that the person installing the module understands the same things the author does. That is, they understand crypto well enough to answer questions.
I personally argue that people should be able to install CPAN modules without knowing Perl at all.
One common response to that is that CPAN should be for Perl developers, not Perl users that don't know Perl. But what is a Perl developer? What separates a one week old newbie programmer from someone that doesn't know Perl at all? Very little.
And in particular, in module installation the recursion problem dictates that the Perl you are asking questions is probably just trying to install Plagger or Jifty or something else that is several layers abstracted from your module, and has no idea whatsoever about that specific domain.
For example, there's another question missing from that set of questions regarding something called "bubble babble" which I only know of as a video game.
Ratcheted Decisions
Worst of all, many of these options commit you to one working one particular way in the future forever. The only way to change that decision is to reinstall the module.
In particular, if you are using binary packages from some distribution, you can't do that anyway. If your binary package of Template Toolkit comes without the optional "Splash!" stuff, then too bad
One nice way to sloganeer this problem is as follows.
"For every option that you put in your Makefile.PL, RedHat will answer it wrong"
(this refers to the long history RedHat has of doing weird things with Perl and related packages).
What can you do about this?
There's a couple of fairly straight forward things that can be done to achieve what you want in any case.
Firstly, if the dependencies are themselves relatively sane (they work on all platforms and you don't mind inheriting any baggage they may have) they just make them a full dependency.
This adds some module bloat, but if those modules WORK, then it's not a problem.
Secondly, don't make it an option at all, just don't depend on the module.
If something one layer above needs it, then THEY can add the dependency on the optional module.
Thirdly, can that optional functionality be separated into a different distribution? Then do that instead.
The highly structured way of splitting this out is to make use of driver or plugin APIs.
And above all, remember this.
There are probably hundreds of thousands or millions of Perl programmers in the world. There are only 5000 on the CPAN. So merely by virtue of having uploaded someone to CPAN you are quite probably already in the top percentile or better.
So try to ALWAYS apply a low estimate for the experience of your users, and try to "Just make it fucking easy to install".
Re:Windows hatred...
Aristotle on 2007-04-19T02:40:22
As long as he only rants in the changelog but doesn’t refuse to consider the request, I can fully sympathise. Proc::Fork (which I’m herding) causes Perl on Windows to segfault, and I have absolutely no idea how to fix it. I tried a bunch of things and got nowhere, and no one else seems to have any useful suggestions.
So I can well understand it if people rant in frustration about Windows. (Or for that matter, about any other part of the environment; Unix is generally saner, but it too is software and software is hateful.)
Re:Windows hatred...
chromatic on 2007-04-20T18:59:45
So I can well understand it if people rant in frustration about Windows.Also, it's not as if there's a legion of Windows Perlers providing patches for these issues. I've asked for several and received approximately none.
I can't support a platform I don't use, and there's little point in trying to support a platform if its users don't care enough to help.
Re:Windows hatred…
Aristotle on 2007-04-20T19:44:06
It isn’t even at that point for me. I do have a Windows box to test on now, at least. (That wasn’t the case half a year ago.) Unfortunately that didn’t help – I might have the tools necessary to support the platform, but I still don’t know how to do it. I tried everything I could think of, but none of it worked. At this point there is nothing left I can do but say “Windows is unsupported; patches welcome.”
Oh, and rant in frustration in the Changelog.
Re:Agreed
Alias on 2007-04-17T16:44:03
There's two seperate issues here.
One is the dependency explosion. That is actually not so big a deal as long as all the modules install cleanly on all platforms, and don't have options or ask questions.
Yes it does become a problem when the dependency trees get large though, as a percentage of them will ask funny questions or potentially hit problems.
Good managers of these large projects will try to choose dependencies that are relatively light, and will also put effort into caring for their upstream, patching, fixing problems, and sometimes taking over modules as needed.
I could single out the Catalyst guys for doing a lot of good things in this regards.
The second problem, that you install different versions, actually isn't that big a deal at all. It's the reason that modules come with automated tests.
These tests are what allows all the modules to evolve independently, with minimal risk of unnoticed damage. Because if something changes, the tests will catch it, and then will be updated, or the module fixed.
Re:Dead
Alias on 2007-04-18T04:53:25
ya
It sort of seems like someone told them "Make it Work, then Make it Maintainable, then Make if Fast" and they stopped at the first one.
I don't know which author you're referring to, but that "winblows" attitude is awful. If you need that added layer of protection (and it seems reasonable), what are the odds of forking the module, ignoring the original author and switching to the fork? I hate suggesting that, but I can't think of an option other than "take the module away" and I would be horrified at the latter thought.
Re:Grrr ...
polettix on 2007-04-18T10:38:41
I think that there are many reasons why one could dislike Windows, and for those very reasons one would avoid spending effort/time in supporting that platform. This is something that happens all the time in many other fields, and not necessarily in the "Windows" direction - just think about the support for Linux of "mainstream software" like skype.
Ranting and denying any type of support is probably a shortminded reaction, but I think that we should respect the decision of someone to *not* support something for somewhat "religious" reasons.
Did I say "religious"? Well, history is full of scisms, and I fully agree with you that a fork is the way to go if a module is needed in an unsupported platform, and the author is not willing to give help. It is likely to broaden the audience, and to lead to a better implementation. Just call it Module::Name::Lefevre;-) Re:Grrr ...
Alias on 2007-04-18T13:23:18
Normally this might not be a problem.
But the crypto modules impact everything.
The result is "If you use security modules, you lose windows support".
CPAN cares more about platform-compatibility than it does about security. Better that it work at all than work securely.
Because of one single developer deciding he doesn't like windows, now nobody gets to have secure CPAN by default.
All limitations imposed by an individual impact EVERYONE in your dependency chain.Re:Grrr ...
polettix on 2007-04-18T22:01:33
I agree with you completely, and this is the very reason why a fork is not only justified, but necessary. Just to avoid being hostage of a single.
Probably, the situation could be though as if one had to implement those modules from the scratch, but having a working reference implementation already available:) Re:Grrr ...
Alias on 2007-04-18T14:02:53
It's not me that needs the module. This is always the problem.
The Module is depended on by Module::Signature, via OpenGPG, via 3 or 4 more layers of recursion.
It's a case of a few bad eggs ruining it for everyone.
Re:Net::SSH::Perl
Alias on 2007-04-19T03:01:13
Net::SSH::Perl is one of the better modules that eventually hits all the dependencies that cause problems.
