introducing Dist::Release

Yanick on 2008-11-11T04:01:41

(cross-posted from Hacking Thy Fearful Symmetry)

I know, I know, there's already more module release managers out there than there are Elvis impersonators in Vegas. Still, module releasing seems to be a very personal kind of itch, and like so many before I couldn't resist and came up with my very own scratching stick.

Of course, I've tried Module::Release. But, although it is intended to be customized to suit each author's specific needs, one has to dig fairly deep in the module's guts to do so. What I really wanted was something even more plug'n'play, something that would be brain-dead easy to plop new components in. Hence Dist::Release.

In Dist::Release, the release process is seen as a sequence of steps. There are two different kind of steps: checks and actions. Checks are non-intrusive verifications (i.e., they're not supposed to touch anything), and actions are the steps that do the active part of the release. When one launches a release, checks are done first. If some fail, we abort the process. If they all pass, then we are good to go and the actions are done as well.

Implementing a check

To create a check, all that is needed is one module with a 'check' method. For example, here is the code to verify that the distribution's MANIFEST is up-to-date:

    package Dist::Release::Check::Manifest::Build;

    use Moose;

    use IPC::Cmd 'run';

    extends 'Dist::Release::Step';

    sub check {
        my $self = shift;

        $self->diag( q{running 'Build distcheck'} )

        my ( $success, $error_code, $full_buf, $stdout_buf, $stderr_buf ) =
        run( command => [qw# ./Build distcheck #] );

        return $self->error( join '', @$full_buf )
            if not $success or grep /not in sync/ => @$stderr_buf; 
    }

    1;

Dist::Release considers the check to have failed if there is any call made to error(). If there is no complain, then it assumes that everything is peachy.

Implementing an action

Actions are only marginally more complicated than checks. The module implementing the action can have an optional check() method, which is going to be run with all the other checks, and must have a release(), which make the release-related changes.

For example, here's the CPANUpload action:



    package Dist::Release::Action::CPANUpload;

    use Moose;

    use CPAN::Uploader;

    extends 'Dist::Release::Action';

    sub check {
        my ($self) = @_;

        # do we have a pause id?
        unless ($self->distrel->config->{pause}{id}
            and $self->distrel->config->{pause}{password} ) {
            $self->error('pause id or password missing from config file');
        }
    }

    sub release {
        my $self = shift;

        $self->diag('verifying that the tarball is present');

        my @archives = <*.tar.gz> or return $self->error('no tarball found');

        if ( @archives > 1 ) {
            return $self->error( 'more than one tarball file found: ' . join ',',
                @archives );
        }

        my $tarball = $archives[0];

        $self->diag("found tarball: $tarball");

        $self->diag("uploading tarball '$tarball' to CPAN");

        my ( $id, $password ) =
            map { $self->distrel->config->{pause}{$_} } qw/ id password /;

        $self->diag("using user '$id'");

        my $args = { user => $id, password => $password };

        unless ( $self->distrel->pretend ) {
            CPAN::Uploader->upload_file( $tarball, $args );
        }
    }

    1;

As for the check(), Dist::Release figures out that a release() failed if there's a call to error().

Configuring for a module

Configuration is done via a 'distrelease.yml' file dropped in the root directory of the project. The file looks like this:

    pause:
        id: yanick
        password: hush 
    checks:
        - VCS::WorkingDirClean
        - Manifest
    actions:
        - GenerateDistribution
        - CPANUpload
        - Github

It's pretty self-explanatory. The checks and actions are applied in the order they are given in the file.

Crying havoc...

And once the configuration file is present, all that remains to be done is to run distrelease, sit back and enjoy the show:

    $ distrelease
    Dist::Release will only pretend to perform the actions (use --doit for the real deal)
    running check cycle...
    regular checks
    VCS::WorkingDirClean              [failed]
    working directory is not clean
    # On branch master
    # Changed but not updated:
    #   (use "git add ..." to update what will be committed)
    #
    #       modified:   Build.PL
    #       modified:   Changes
    #       modified:   README
    #       modified:   distrelease.yml
    #       modified:   lib/Dist/Release/Check/Manifest.pm
    #       modified:   script/distrelease
    #
    # Untracked files:
    #   (use "git add ..." to include in what will be committed)
    #
    #       STDOUT
    #       a
    #       blog
    #       xml
    #       xt/dependencies.t
    #       xxx
    no changes added to commit (use "git add" and/or "git commit -a")
    Manifest                          [failed]
    No such file: lib/Dist/Release/Action/DoSomething.pm
    Not in MANIFEST: a
    Not in MANIFEST: blog
    Not in MANIFEST: lib/Dist/Release/Action/Github.pm
    Not in MANIFEST: STDOUT
    Not in MANIFEST: xml
    Not in MANIFEST: xt/dependencies.t
    Not in MANIFEST: xxx
    MANIFEST appears to be out of sync with the distribution
    pre-action checks
    GenerateDistribution              [passed]
    no check implemented
    CPANUpload                        [passed]
    Github                            [passed]
    2 checks failed
    some checks failed, aborting the release

Getting the good

A first release of Dist::Release is already waiting for you on CPAN. It's beta, has no documentation, is probably buggy as hell, but it's there. And the code is also available on Github. Comments, suggestions, forks and patches are welcome, as always. :-)

Drop a file in the distribution?

Alias on 2008-11-11T06:20:47

Really? A file? In each distro?

So I need 170 identical YAML files added to my repository, and then to change something I need to change 170 different config files?

Re:Drop a file in the distribution?

Alias on 2008-11-11T07:56:09

... and all 170 yaml files need to contain my PAUSE admin login and password?
And in a repository that is visible to the public?

Re:Drop a file in the distribution?

Yanick on 2008-11-11T13:22:04

Actually, I skipped over some details in the post so as not to make everybody's eyes glaze over.
As it turn out to be, the configuration is first read from $ENV{HOME}/distrelease.yml (if it exists), which values can then be augmented/overrided from the local distrelease.yml. Although I have nowhere the gargantuan number of distros that you have, I'm not overly fond of needlessly replicating information either. :-)

Looks the same to me :)

brian_d_foy on 2008-11-11T11:44:25

You don't have to dig deep in Module::Release to do anything. You just make a method then call it. It's easy as pie. It's even easier in the 2.x stuff that's sitting on CPAN. If you haven't looked at it in awhile, take a peek even if to just steal ideas. When I get some time, I'll steal some of your ideas.

It's not a big deal if you write your own stuff. No one seems to use anything else that anyone writes anyway. :)

Your modules look a lot like the Module::Release stuff. I like that you configure it from a text file though. I've always wanted to write a DSL for Module::Release (perhaps using my Polyglot module). The slight differences are mostly what I would do if I started over.

Although Module::Release also has the problem of dropping a file in each directory, the next thing on my to do list is a global config that a per-dist config would override.

Re:Looks the same to me :)

Alias on 2008-11-11T13:22:16

Instead of a full DSL, maybe you could do something like Module::Install does.
Make a really tight MI style set of commands that get you most of the way to a simple step by step process, such that we can fall back on pure perl and mix it together if needed.

Re:Looks the same to me :)

brian_d_foy on 2008-11-11T22:33:20

Well, I think I have that. It's just that everything is a method call instead of a function call. Most of the release strict is either exposed scaffolding (which I really would like to fix when I have enough free time to fix things that work), or just a declaration of steps.
But, that's something to think about later. If someone really wanted it I might change it, but since the three people in this thread all have their own release software that no-one else uses, I'm not very motivated. :)

Re:Looks the same to me :)

Yanick on 2008-11-11T16:08:10

Your modules look a lot like the Module::Release stuff.
What can I say? I may be foolish enough to reinvent the wheel, but yet I know better than not let myself be inspired by what's already there. :-)
You don't have to dig deep in Module::Release to do anything. You just make a method then call it. It's easy as pie.
I'll give you that it's not that deep, and that it's not too hard, but I kinda balk at having to hack into the 'release' script if I want to add a new 'check_somethingelse()' action, or disactivate that 'Changes'-munging step that drives me bunker. It's not a lot, really, but then I don't need a lot of excuses to go a-hackin'. :-)
It's even easier in the 2.x stuff that's sitting on CPAN. If you haven't looked at it in awhile, take a peek even if to just steal ideas. When I get some time, I'll steal some of your ideas.
Ooooh, didn't know there is a 2.0 version kicking around. I'll look into that for sure. And yes, by all means, steal everything that you find interesting -- I'll be more than glad to return the favor.
Although Module::Release also has the problem of dropping a file in each directory, the next thing on my to do list is a global config that a per-dist config would override.
As I said in my reply to Adam, D::R already take care of that. The margin of the post was just too small to mention it explicitly. ;-)