I've been reading a few articles on integrating daily builds into the software development process:
We have a script called smoke that
All this is done by creating my own subclass of Test::Harness::Straps which can handle all those other non-*.t files.
Also, some of the test files are meta-test files. For example, there's a
#!/usr/bin/perl -w
use strict;
use File::Spec;
use File::Find::Rule;
use Pod::Checker;
use Test::More 'no_plan';
my $base = (shift || $ENV{TWROOT} || '.');
my $rule = File::Find::Rule->new;
$rule->or( $rule->new->directory->name('CVS')->prune->discard,
$rule->new->file->name( '*.pl','*.pm','*.t' ) );
my @files = $rule->in( $base );
our $podchecker = Pod::Checker->new( '-warnings' => 1 );
for my $file ( @files ) {
check( $file );
}
sub check {
my $filename = shift;
my $dispname = File::Spec->abs2rel( $filename, $base );
local $/ = undef;
open( my $fh, $filename ) or return fail( "Couldn't open $dispname: $!" );
my $text = <$fh>;
close $fh;
# Search for strict and warnings. You can get around this just by
# having the text in comments, but that's not the point.
ok( $text =~/use strict;/, "$dispname uses strict" );
ok( $text =~/use warnings;|perl -w/, "$dispname uses warnings" );
}
I'm doing all sorts of automated testing as I keep going on. I'm thinkin' of starting a series of articles in The Perl Review on it.
Re:Boy do we test
cbrooks on 2002-08-29T16:16:23
Cool stuff. What I'm trying to sort out for myself is if there are additional benefits to be found from doing a build on a nightly basis. That is, doing a cvs checkout into a fresh directory, firing up an Apache process which points to that directory, running the automated tests, then (assuming everything passes) tar'ing up the directory and storing it away.
The tests that you are doing seem to capture a lot of the benefits of a nightly build:
- Syntax checking of all of your scripts is the equivalent of checking to see that they compile correctly.
- Smoke testing by running unit and customized tests
- Forcing programmers to write tests, since modules without a
.t file will break the smoke test
I guess what I'm wondering is whether there are additional benefits that could be achieved by pulling the sources out of cvs and tar'ing them up. There seem like there might be two benefits:That last one may or may not make sense. Here's the problem that I'm concerned about: One of member of the team starts to update some code on existing production functionality (call it project A). The project isn't top priority, so they commit their (incomplete) project A code to cvs, and begin work on project B. Now, suppose that we had a nightly (or even weekly) automated release cycle which simply performs a cvs update from the staging server to the production server. We have now launched unfinished code to the production server. Even without an automated cycle, there's a problem -- you need an automated way to check whether committed files are ready for production. Doing a nightly build does not solve this problem -- but it ensures that programmers will be comfortable commiting unfinished code, since committed code is not automatically a candidate for release.
- If this is the core of your release system (i.e. you untar the resulting tarball on your production server(s) and kick the web server), you have tested that your release system works.
- You have separated the decision to release functionality from the decision to commit changes to source code.
