The part of the Perl community I really hate

Alias on 2005-06-14T07:24:20

Very very rarely do people piss me off. But the Perl community has one group that really gets me riled.

The BOFHs.

Traditionally, Perl has really shined in three areas, those being Admin, Web, and Data(munging|base).

As a result, quite a lot of Perl developers are professional sysadmins. And as a result of _that_, there's a percentage of BOFHs mixed in.

In fact, if you exclude the scientists and the speakers, most Perl people can be seen to fit on a scale that has "pure developer" at one end and "pure admin" at the other.

For some reason, admins seem to have a particular approach to life, that being to take an approach of doing things with the least amount of work possible. And BOFHs do this aggresively (with a good dash of misogeny thrown in).

Now, for normal day to day tasks, this is a good thing. We want people that can pull a bunch of tools, read some docs, and get stuff working quickly. And BOFHs are great in situations where you _really_ care about the infrastructure, and you don't want it to suffer because the users are bringing shitloads of jobs for the admins and distracting them.

In the development universe however, this approach completely sucks.

Developers, by their nature, are problem solvers of a different sort. Developers want to solve a problem, but they want to solve it once, properly, for the entire class of problem, in as many situations as possible. It's the "Lazy" part of that famous line. Putting in extra work now, so you can be lazy later.

Admins don't have this luxury. Despite the best efforts of an admin, hardware will still break, the business around them will change and invalidate their existing system, and things are generally more fluid.

Developers often have the blessing of a comparatively stable environment. Things don't just randomly break. If they do, it's due to a flaw, and not mere chance, and if you improve the solution, life is good again.

In the Perl world, the members of the "CPAN Double Figures Club" occupy the extreme end of this admin-developer scale.

These are the 162 people that have uploaded 10 or more packages that contain modules in the CPAN index.

The elite "CPAN Triple Figures Club" has only two members, Autrijus Tang(active) and Simon Cozens(retired).

For these 162 people, it is not so much that they wish to solve the problem once, but that they want to solve it once for everybody.

And it's done for the most part in the same spirit as Unix. Lots of small pieces that do one thing well, all interconnected.

So I get extremely annoyed when perfectly decent Perl people that normally act in a clueful manner seem to complete miss the point, and devolve into BOFH behaviour.

Today I got involved in an online conversation relating to testing. It's always perplexed me that the Perl community, filled to the brim with talented programmers and admins, using a language that shines at admin work, doesn't have a reliable way of testing packages.

Yes, we have CPAN Testers, but they only test CPAN packages, and only test what they want to test. In that regards, in the limited case we are willing to give code away to the community, we are behoven to the kindness of strangers to do our testing for us. We have to upload potentially broken packages, and then respond to the errors after they break.

Responding to errors and fixing it for each case is admin behaviour, evolved from an environment in which you can't predict errors until they happen.

As developers, we should be expected to solve it in a better way. And to do so that doesn't just work for CPAN, but is also useful for commerical packages.

Boosting the commercial Perl world boosts us all. It means more and better paying jobs for Perl programmers, and more and better Perl programmers.

So why not create a GOOD system for testing Perl packages of any sort.

The obvious solution (to me at least) is some form of defined testing system that anyone can use, made up of about 4 distinct components.

Lets call it Test::Platform.

A Test::Platform server must have an installation of perl configured to CPAN (or a minicpan).

It must have a service running on a defined port, probably in the form of a small web or SOAP server (wrapped up as a PAR dist to avoid altering the native perl installation), to which one can upload a CPAN-compatible tarball.

The Test::Platform::Server will take the tarball, unwrap it, configure, make, test and install the package, while capturing the output of the process. It will return the output to the client, that can do with it as they wish.

Test::Platform::Client would presumably take a file or URL, send it to the server, retrieve the result, analyze the result, and act on it accordingly.

It's a small step from there to imagine a script or module that would set up your system form you to act as Test::Platform-compatible server.

It's only another small step to imagine setting up a VMWare image like this, and having Test::Platform::Client::VMware boot up the image, connect to the standard testing image IP address, and fire the testing package at it.

Another step would see you being able to test a package against multiple images and collect the results in a database like SQLite.

And one more step sees someone hosting a repository of pre-built images somewhere.

You could install Test::Platform and VMWare, download a dozen images of your choice from the image repository, and test any package you wished.

All you would need would the hardware to run it on, and the patience to start it up and walk away while it ran the test run for each image.

Now, it seems perfectly straight forward to me, but _I_ am a mere developer, suck at admin stuff, and don't really have the time to learn on how to build VMWare images.

To have half a dozen BOFHs offer to help you out and say, "But it's so easy, you just $foo. Why do you need a module for that" is unbelievably frustrating, especially when what seems easy and simple to THEM is completely new and unexplored and difficult for you. Geeks and BOFHs are prone to this sort of thing.

Sometimes it's like hearing them say.

"It's easy, first you numb the area using any suitable neural analgesic, then drill a short-medium length into your head. Once there, use any applicable general relativity engine to write a Perl 6 implementation, then you're done. How hard could that possibly be!"

I don't want to solve my immediate problem.

Hell, I don't even HAVE an immediate problem of this nature. But there are 2800 CPAN developers, and hundreds of thousands of Perl programmers out there who would benefit if someone who knows this stuff that I don't would take the first small steps and write a basic Test::Platform implementation, or write a script to set up a VMWare image as a Test::Platform-compatible image, or even write a detailed or hell, a fucking cut and paste HOWTO so that someone else can write it.

ThreatNet has started to take off now. Why? Not because I wrote a bot that people could use, but because I added complete working config and filter code to the docs.

So all that anyone who was curious had to do was cut and paste and change some values.

When will the Perl community learn the key to making successful projects is to make it as trivially braindead as possible to install it and get started, and to make some fast early wins for the users.

At about the time they started to explode in usage, PHP and MySQL were (and to some extent still are) the ugliest and worst possible combination for making web applications in. And look where it got.

So in conclusion and on behalf of the 162 of us.

------------------------------

Dear Perl BOFH.

Show some fucking vision!

Take the first steps and encapsulate that great knowledge of yours, so that we can take the next steps and everyone can benefit.

We wrote most of the modules you use every day, how about giving something back occasionally.

Sincely

The CPAN Double Figures Club

---------------------------------------------



And if anyone is listening and wants to help start a project that could fix this whole testing problem once and forall, write a small module that will run a small web server on a specific port, that takes requests with three or four properties.

testid = string cpan = path to a cpan mirror package = file upload of a package tarball respose = URL to send the results to

Have it unwrap the tarball, run perl Makefile.PL, make, make test, make install, capture the results of STDOUT and STDERR, and then send the response via a HTTP PORT to the URL it was provided.


just post the password...

ask on 2005-06-14T12:09:25

Wouldn't it be easier if you just posted your root password?

Having a (standardized?) vmware image is missing the point as it's the different configurations that finds the errors that you as a developer can't test yourself.

  - ask

Re:just post the password...

Alias on 2005-06-14T14:39:27

It doesn't need to be heavily standardised. It only need to behave in a very limited number of specific ways. The bits that _are_ standard are likely to be outside the testing scope for all but a few cases.

Hmmm

sigzero on 2005-06-14T12:48:06

I got the impression while reading your post that you think "they" should do it. Try posting this on perl.qa group. I am sure "they" would encourage "you" to do it and in the meantime "they" will continue with building the test modules "they" are working on.

Re:Hmmm

Alias on 2005-06-14T14:44:29

That is not point. The point is that I should not have to be the one to write _everything_.

I'm absolutely sick of people that offer to help, and the proceed to describe not only why the thing they are offering to help on sucks and won't work, or "isn't needed cause you can solve your specific case like so".

When it will work, and it is needed.

More complex than you imagine

barbie on 2005-06-14T17:38:04

There are currently around 60 regular CPAN Testers, who test on a variety of platforms and perls. Last month there were 22 different platforms and 11 different perls giving 241 different setups. Being able to have perhaps a good selection of images, with at least the major perl releases and the most popular platforms is a good idea. And the idea you're proposing would hopefully catch most of the obvious bugs, but you aren't going to get away from using CPAN testers. I am pretty certain every single one of the 241 setups last month had a complete different set of CPAN modules installed too. That's why they are so useful in spotting edge cases and conflicts.

Part of your Test::Platform::Server is exactly what CPAN/CPANPLUS already do, so you would probably just need to write the actual server part. Your SOAP and client ideas are then down to someone to define the APIs.

However, the bigger part of the problem is where to host all the images. And all the extra gubbins you'll need to test all the distributions on CPAN. The biggest problem though concerns trojan distributions. These are the distributions that 'phone home' or attempt to test external libraries and applications. Who is going to host a server which could potentially become an open proxy? Having CPAN testers in the mix, usually ensures that these types of distributions are tested properly.

Your assumptions that the CPAN Testers only test what they want to, is a little undeserved. They mostly test what they can. In many cases they don't have all the external libraries and apps to do a thorough test of every distribution. In some cases they have to test manually, as some distributions insist on having parameters set before they will work.

I think your assumption that this is easy is wrong. It's taken several years for CPANPLUS to get to the stage it's at and there are still bugs. Doing all that around a testing server/client as well is going to take a lot of careful thought.

If you're willing to make a start then make a start. Just because those you've spoken to currently haven't take much of an interest doesn't mean that they won't in the future, or even someone new to the world of Perl won't get as enthusiastic as you about what you want to do. You obviously have some ideas around this already, so perhaps that's where you need to start. Also bear in mind some might not have as much time on their hands to do much work on this, we're not all young free and single with CFT (copious free time) on our hands.

It's an old saying (well from at least 1989!), but "build it and they will come".

Re:More complex than you imagine

Alias on 2005-06-14T21:56:26

CPAN Testers doesn't solve the problem of testing. Full stop.

They solve ONLY the limited subset "testing released CPAN packages".

CPAN Testers is no solution at all for company or private packages.

And of course they test only what they want. If they didn't want to be in CPAN testers, they wouldn't be there.

They have full control over what they do, and I have ZERO control over them, as you would expect from people contributing their time in this way.

I'm just starting to get a little frustrated that every time I move into a new area of CPAN, I have to rewrite to replace half of what I encounter because a lot of it is written for far too specific scenarios.

And I'm REALLY annoyed at people that ACT like they are volunteering, and then proceed to troll and generally act like people with "Bad Head"s (google it)

Re:More complex than you imagine

barbie on 2005-06-15T07:09:49

CPAN Testers doesn't solve the problem of testing. Full stop.

No it does solve complete coverage of testing, but it goes a long way to covering the bulk of scenarios. Trying to test every scenario that exists is a huge task and one I don't think will ever be achieved.

CPAN Testers is no solution at all for company or private packages.

Why? I assume you're talking from your own experience. In my experience the cpan-testers have proved useful to know what works on various platforms out of the box and what may need a little work to get it going.

And of course they test only what they want. If they didn't want to be in CPAN testers, they wouldn't be there.

To a degree that's true, but I think you're being very stereotypical, and there are plenty of testers who do their utmost to test as much as they can.

I have to rewrite to replace half of what I encounter because a lot of it is written for far too specific scenarios.

I'm curious as to why you appear to be berating the current approach, yet mosts of the task are built on top of each other. CPAN/CPANPLUS are installers, yet they work with the CPAN Testing tools. The CPAN Testing tools could probably be used in the kind of settings you require. Building the components makes more sense as they can then be used better in more than the intended scenario.

I think your last statement is misplaced. Although you may feel that people are trolling and not helping to get the job done, for myself I am interested in your idea, unfortunately I don't have the time to work on it right now. Others may be in a similar position. From what you've said I would say others are trying to offer encouragement and support where they can. It might not seem enough, but personally I would rather that, than they try and knock me down at every opportunity.

Re:More complex than you imagine

barbie on 2005-06-15T07:11:50

Sorry that first sentence should have been ...

"No it doesn't solve complete coverage of testing ..."

Re:More complex than you imagine

drhyde on 2005-06-15T09:25:01

Don't you think that a "company or private package" only has to work in the "company or private" environment and that consequently the "company or private" developer can, if he likes, use exactly the same tools as the CPAN testers to do his testing?

A few things that might be helpful to consider...

atcroft on 2005-06-15T04:26:49

First of all, an offering. Below is some bash-shell code that might be a start toward a portion of what you requested. The code was passed a parameter that was the name of a perl source tar-ball (.gz). It built the version of perl and installed it to a local usr/test/ directory tree relative to where it was installed, and at the time would execute the CPAN module's 'autobundle' command and write to a file. (It also removed the 'built-in' and 'command line' entries from the various Makefiles, because of issues I experienced.) #!/bin/bash +x CSD=`pwd` PATH=$CSD/usr/test/bin:$PATH file=$1 directory=`gzip -dc $file | tar tf - | head -1 | tr '/' ' ' | gawk '{print $1;}'` echo " $file - $directory " gzip -dc $file | tar xf - rm -fr $CSD/usr/test/* cd $directory rm -f config.sh Policy.sh sh Configure -des -Dprefix=$CSD/usr/test -Dusedevel -Uinstallusrbinperl for makefile in `find . -type f -name makefile` do cp -v $makefile $makefile.temp grep -v built-in $makefile.temp | grep -v 'command line' > $makefile done make make test make install ( cd /usr/include && $CSD/usr/test/bin/h2ph *.h sys/*.h ) date >> ~/.cpan/Bundle/$directory.txt perl -MCPAN -e 'autobundle;' cd $CSD echo rm -fr $directory rm -fr $directory

I could imagine after building the perl version desired using a Bundle file to have the system install the other modules desired. If the system had a complete local CPAN mirror, then testing under a number of different versions might even be possible.

Having said that, the only big issues I can see with the proposal are: a) the presence or lack thereof of external libraries or utilities that might be required to test a particular configuration, and b) the presence of tests for the module(s) involved (as there are some modules I have encountered in the past that lacked tests, which made it difficult to install them automatically from the CPAN module.

I would also suggest considering having the system email the results or notification of their availability, and possibly writing into some kind of report the submitter could come back to check later (such as a report stored in a database or something).

I don't know if any of what I said is of any help or anything, but if you choose to move ahead with a project like this, I wish you the best with it.

Re:A few things that might be helpful to consider.

atcroft on 2005-06-15T04:30:48

Argh! I didn't mean the formatting of the code to be so screwy. Anyway, an attempt to make the code look more readable (with appologies for having to resumbit):

#!/bin/bash +x

CSD=`pwd`
PATH=$CSD/usr/test/bin:$PATH

file=$1
directory=`gzip -dc $file | tar tf - | head -1 | tr '/' ' ' | gawk '{print $1;}'`

echo " $file - $directory "
gzip -dc $file | tar xf -

rm -fr $CSD/usr/test/*
cd $directory

rm -f config.sh Policy.sh
sh Configure -des -Dprefix=$CSD/usr/test -Dusedevel -Uinstallusrbinperl
for makefile in `find . -type f -name makefile`
do
        cp -v $makefile $makefile.temp
        grep -v built-in $makefile.temp | grep -v 'command line' > $makefile
done
make
make test
make install
( cd /usr/include && $CSD/usr/test/bin/h2ph *.h sys/*.h )

date >> ~/.cpan/Bundle/$directory.txt
perl -MCPAN -e 'autobundle;'

cd $CSD
echo rm -fr $directory
rm -fr $directory

Re:A few things that might be helpful to consider.

Aristotle on 2005-06-15T22:07:30

Use the <ecode> tags, Luke.

Sorry, I have to rant

drhyde on 2005-06-15T10:13:18

We can tell several things from your journal entry.
  1. You don't know any good sysadmins. When you say "for some reason, admins seem to have a particular approach to life, that being to take an approach of doing things with the least amount of work possible. And BOFHs do this aggresively (with a good dash of misogeny thrown in)" you miss the point that most of a sysadmin's job is tedious and easily automatable, and so it *should* be automated away. This isn't because he's lazy and wants to sit back and do nothing though, it's because it gives him time to get on with important things like designing and implementing robust scalable systems for you developers to run your code on. As for the aggression and misogeny, in my experience that can come from anywhere, not just admins, and is most likely to come from sales and marketing.
  2. You don't know anything about sysadminning. You seem to be saying that people who do quick n' dirty hacks are a good thing when you care about the infrastructure. This is as untrue in sysadminning as you know it to be in software development.
  3. You don't know much about software development either. Your claim that developers want to solve problems in the widest most generic way possible just ain't true. Developers work to deadlines and to customer requirements. And while it is good if they can also take account of the big picture, not only is that not always possible, it's not even always advisable! You frequently don't even know what the generic solution might look like until you've written a special-purpose one first. And a generic solution will often need so much wrapping and tweaking and poking and prodding before it can be applied to a particular problem that you might as well just write a special-purpose solution anyway.
  4. You don't know much about business requirements. You seem to think that developers live in a nice fluffy world where they are shielded from outside changes but that sysadmins are subject to the capricious whims of The Management. In reality, developers, admins *and* management are subject to random changes requested by people who don't know what they're talking about. In polite society we call them "customers". A competent developer or sysadmin will be able to explain to the person requesting changes the impact that this will have on such things as delivery dates, costs and so on, and will be able to plan the changed job properly.
  5. You seem to think that sysadmins don't plan ahead. A good sysadmin is *not* thrown into a panic when hardware fails. A good sysadmin *knows* that the hardware is out to get him, and will have automatic failover configured (and tested!) or will have plans to temporarily restore service elsewhere. Good sysadmins care about Mean Time To Recovery.
  6. You think you can speak for me. I'm one of your 162 superheroes. But contrary to what you claim, I am not your perfect developer. Some of the stuff I put on CPAN I put there because I had no convenient web server of my own to put it on at the time, and wanted to share it with one other person. Some of the stuff I put there I published because it looks good to potential employers. Some I put there because I want other people to use it. And some I put there only because I was using it, the previous author had abandoned it, and I needed to fix bugs. Not only are my motives in putting stuff in the CPAN suspect, but my background wouldn't be to your liking either - I'm at least as much of a sysadmin and perpetrator of quick hacks as I am a developer.
As for your Test::Platform proposal - it's an interesting idea, but not sufficiently interesting for me to care. My needs are taken care of already. If you want your better mousetrap you're going to have to build it yourself. Not necessarily all of it, but at least enough of it to demonstrate to us sceptics that it really is worth doing.

Oh and I should point out that your proposal for a Test::Platform::Client::VMware module breaks one of your own rules for being a good developer. It is insufficiently generalised. Surely you should have Test::Platform::Client::VirtualMachine, from which specific modules for VMware, Mac On Linux, Xen, User Mode Linux, LPARs, FreeBSD jails etc can inherit.

Re:Sorry, I have to rant

Alias on 2005-06-18T14:54:12

Your claim that developers want to solve problems in the widest most generic way possible just ain't true. Developers work to deadlines and to customer requirements.

Indeed, and they are compromises from the ideal.

You seem to think that sysadmins don't plan ahead. A good sysadmin is *not* thrown into a panic when hardware fails. A good sysadmin *knows* that the hardware is out to get him, and will have automatic failover configured (and tested!) or will have plans to temporarily restore service elsewhere. Good sysadmins care about Mean Time To Recovery.

Of course they do. My point is that in regards to their technical work, admin's lives are primarily exception based. The reality of devices that ultimately rely on quantum mechanics and probability to keep working.

You think you can speak for me.

Yes. It's my rant and I don't do it very often, so I'll on those rare occasions I'll speak for whoever I bloody well like.

Oh and I should point out that your proposal for a Test::Platform::Client::VMware module breaks one of your own rules for being a good developer. It is insufficiently generalised. Surely you should have Test::Platform::Client::VirtualMachine

A Test::Platform::Client is a virtual machine.

Re:

Aristotle on 2005-06-15T22:26:08

My very own thoughts: typical perfect world developer mindset to assume that a big and complex problem is straightforwardly and fully solvable. I expect that if you actually try, you’ll find reality to be pricklishly difficult, full of nasty edge cases and little ratholes. Testing is a messy problemspace; that’s why CPANPLUS is still rough.

However, I am open to being shown otherwise; if you think you really can solve this, go ahead and try. You’ll either find out that everyone else was right – or that they were all wrong. That's how progress is made. But prepare to go it alone until you have at least a working prototype.

Show us the code.