OMG WTF BBQ! Could Acme::Everything be leaving Acme?

Alias on 2006-02-20T07:49:30

The first module I ever wrote for CPAN was Class::Autouse.

I wanted something that would autoload classes, but I really didn't like the three existing auto-loading modules, because they all had the same problem.

They requires a whole bunch of extra import params to handle the fact they tried to fake that their call to import had really happened at autoload time. I didn't realise until recently that even trying to do this is ultimately futile for similar reasons to both the Halting Problem and the Perl document/code duality problem.

But at the time, all I really wanted was to be able to autoload the class without the extra mess. In OO you only really need the require half of loading a module 99% of the time anyway.

In discarding import capability, I found you could write a really clean syntax for class auto-loading, you didn't have to touch UNIVERSAL::AUTOLOAD (*phew*), and you could get perfect simulated loading (except for crashing on require, but you can't help that anyway).

Once I got everything running, it occured to me I could actually take it one step further by using UNIVERSAL::AUTOLOAD anyway. So I added the Class::Autouse "superloader".

Now with one simple use call, you could make every installed module load the first time you called a method. It was a good deal more evil, and it's not recommended in production publically released code (mostly due to an annoying tendency to explode violently when it encounters anyone else playing games with UNIVERSAL), but it does actually work.

And with the exception of the whole exploding thing, there are no known edge cases where it doesn't just Do What You Mean. Even the errors are simulated correctly. For any OO use, it genuinely looked loaded. Hell, it was loaded. You could call it duck-loading :) I was very happy with it.

At the time I was distracted by both the whole overwhelming coolness and nervousness factors of doing My First CPAN Module (and not screwing it up), so it wasn't until years later that I happened to spot Acme::RemoteINC.

This lets you set up a CPAN hook, so that if you try to load a module that is not installed, it wanders off to CPAN, downloads it, and installs it at load time.

Wow!

So there was certain level of symmetry when I wrote my first talk ever on Acme::Everything, which was created to combine the features of Acme::RemoteINC with the Class::Autouse superloader, to create a single use command that will (effectively) load all of CPAN.

It's amazingly evil, and quite messy (the install bit spams stuff to STDOUT/STDERR) but it does actually work.

I wrote the talk as a sort of "evil by example" talk, showing an overview of how Class::Autouse, Acme::RemoteINC and Acme::Everything worked, what the specific evil things they did was, and showing how in the non-superloader case Class::Autouse is able to contain the black magic it uses, and how much effort it actually takes (huge amounts of effort) to use black magic in production code safely.

With some time to spare at the end of Sydney.pm last week, and with people looking eager for an encore or something, I dug through my old talks and pulled it out.

It got the usual reaction, intent stares of initial confusion, followed by gasps, groans and giggles as things get progressively more evil and out of control.

But then at the end in questions I got something new from the back of the room. The sort of cringe-inducing comment you don't want when you've spent 5-10 minutes telling people why they don't want to do this stuff.

"Hey, we could really use that on our project!"

But it turns out, he might be right.

The commenters were two of the contributers to TWiki, a "corporatey" Wiki who has a userbase strongly populated by "managers", people that often wouldn't know a command line if you hit them with it.

They don't have the option of saying "Well if you can't install it, you don't deserve to use it" because there goes half of their entire userbase in one hit.

As someone who helps out with installer modules on occasion, they are both a blessing and a curse as users. A curse for obvious reasons, but a blessing because if we learn nothing else from the examples of Windows, PHP, Google and Skype it's that victory goes to those whoever makes technology easiest to use, and a big part of that is making it easy to install. And these users help you make installers better.

If your reasonably-intelligent but techno-clueless manager can't install a program, then you've taken someone eager and willing to try your project out, and blown your first and only impression. And people talk, so then you've lost 10 people that they know (at least) as potential users as well. Your project has ZERO good features, because they never got to see any of them, just the one big bad feature of "it didn't work".

If it does work (enough) that a manager-type can muddle through the install, then well you can bet it works for everyone else that is even remotely technically savvy.

So there's a huge penalty already if your project isn't completely DWIM to install and a huge advantage if it is, something I continue to find it difficult to explain to people who argue we should make things easy for the CPAN author instead of the end user, whenever given a mutually exclusive choice between the two. I see people consistently abandoning CPAN modules, be they big or small, that don't work or install everywhere properly, because ever module with a problem infects their work with the same problem, and they all add up until eventually you can't use your code anywhere.

In my case, I just recently abandoned IO::Scalar for IO::String, which is smaller, saner and more accurate, and I've abandoned just about every other way of launching external system programs to move over to IPC::Run3, which Does The Right Thing and works everywhere.

But back to the TWiki case...

It turns out that the TWiki users can write their own plugins like they can for other things. But because with TWiki there's no assumption these plugins have to be done as CPAN modules, people will actually learn some basic Perl just to write a custom little TWiki plugin, and then they'll put it into the plugin directory DIRECTLY and just expect it to work.

And it's here that Acme::Everything may just become useful in real life. Now the plugin author can put CGI->foo in their plugin, and it will Just Work.

Because the first time a page that uses it called, then TWiki will go off, install the module from CPAN, load it, and call the method. And it will Just Work. It might not work every time, but it should certainly work a lot of the time.

But wait! There's more!

We could go one step even further (and safer) by adding an "install custom plugin" form in TWiki itself, that lets admin users upload a .pm file directly from their hard disk.

It will then scan the .pm document with PPI, determine the dependencies, generate a suitable Module::Install Makefile.PL for the module, various other files, and maybe an auto-generated compile test into a temporary directory, to create a 100% automagically created plugin package, then run the Makefile.PL, installing the dependencies, making the plugin, and installing the plugin to the right place, with man pages and other POD docs.

And it _might_ be good enough that it will Just Work even to the point of compiling the package into a Debian .deb first, so it can even be uninstalled properly too.

I've created a skeleton of such a module, which I'll call Module::AutoPackage, in my subversion repository for the sake of giving such a beast a name. Who knows if it will ever get written, since we need the existing installation infrastructure to be a lot more reliable first (hence PITA).

But it's a very compelling idea for making CGI applications a lot simpler to extend for normal users.


What about external libraries?

bart on 2006-02-20T17:16:16

to create a single use command that will (effectively) load all of CPAN.
Only if you ignore external libraries.

XS modules by themselves are already problematic, but assuming people have a suitable C compiler, those libraries that are dependant on an external C library, cannot simply be installed using CPAN.pm. Not even if you already have the external library.

Well of course...

Alias on 2006-02-21T01:33:48

Or all the other reason you can't have it, like Win32:: on linux and so on.

BUT, they are there, and you can call methods on them.

It's just that the method might explode horribly :)