All the Perl that's Practical to Extract and Report

Test::Pod::Coverage is harmful

jjore on 2007-04-10T17:30:00

Its preventing File::Slurp, a prerequisite of Jifty, from installing cleanly just because its detecting some imported POSIX constants. WTF?!

This kind of test should be run only when an environment variable like AUTHOR is true, not for every user. I didn't want my Jifty install to fail just because some deep prerequisite was being too picky about making sure its own documentation covered everything it imported into itself.

Upgrade CPAN, use distroprefs and relax

LaPerla on 2007-04-10T20:22:25

(1) Upgrade to CPAN 1.90

(2) configure distroprefs: 'o conf prefs_dir ...'

(3) copy URI.File-Slurp.yml from the CPAN distribution to your prefs directory

(4) Relax and never ever care again about File::Slurp or Pod::Coverage.

Re:Upgrade CPAN, use distroprefs and relax

jjore on 2007-04-10T22:59:33

Er, no. That's a losing proposition. That just means the code remains broken on CPAN. Hopefully none of the people responsible for maintaining that code are using this to ignore failures.

When I'm testing for what a "regular" user sees, a raw configuration is better.

Re:Upgrade CPAN, use distroprefs and relax

brian_d_foy on 2007-04-11T13:54:16

It's not just a problem for a single person, but for every user.

I recently outgrown Test::Pod::Coverage because Pod::Coverage is just really dumb. It seems to handle only the simple solution: a single package per file, and that package has to named after the file path. That just doesn't work for me anymore.

Re:Upgrade CPAN, use distroprefs and relax

Aristotle on 2007-04-11T17:04:09

From the t/pod-coverage.t of my last module:

#!perl -T
use Test::More tests => 1;
0 or eval 'use Test::Pod::Coverage';
ok( 1, 'This file is just here as a sop for CPANTS' ) or all_pod_coverage_ok();
# vim:ft=perl:

Use best Test::Pod::Coverage, maybe

ishigaki on 2007-04-11T03:48:10

If your TPC is 1.04 or prior, upgrade it first to the latest version and it would be less harmful. The same thing can be said for every Test:: modules concerned.

Besides, skipping pod coverage test with environmental variables (TEST_POD or AUTHOR or anything) is not a good idea, as it tends to be ignored even by module authors (especially those who copy the test from someone else's modules, and thus, don't know it should be enabled somehow). Actually there're modules that have pod coverage test that fail if you enable TEST_POD. Pod coverage test should guarantee the modules' kwalitee. I don't want it to deceive us, even unintentionally.

Re:Use best Test::Pod::Coverage, maybe

jjore on 2007-04-11T17:16:10

I have no problem upgrading my own installation to the latest stuff. It's my own dev stuff, y'know? What I'm testing now is that a plain, fresh, ordinary perl can still install stuff. If modules are going to require testing infrastructure that doesn't work properly until a high enough version then modules are going to have to include those "high enough" versions in their PREREQ_PM lists.

Without proper PREREQ_PM, modules just fail to install and that sucks.

Ah good, people are getting it

Alias on 2007-04-12T10:58:41

Reading some of the comments, it's nice to see more people are realising what a stupid methodology Test::Pod::Coverage has. I tried it once, was curious how it could do what it said without using PPI, looked at how it worked, and ran screaming.

As for the AUTHOR flag, pending some newly created standard, the one you want to use is probably AUTOMATED_TESTING.

The tests will fail in CPAN Testers, but pass when a user installs it.

Here's the current version of 99_author.t from my repository (which is bundled with all of my modules).

-----------------------------------------------------------

#!/usr/bin/perl

use strict;
BEGIN {
        $| = 1;
        $^W = 1;
}
use Test::More;

# Skip if doing a regular install
unless ( $ENV{AUTOMATED_TESTING} ) {
        plan( skip_all => "Author tests not required for installation" );
}

# Can we run the POD tests?
eval "use Test::Pod 1.00";
if ( $@ ) {
        plan( skip_all => "Test::Pod 1.00 required for testing POD" );
}

# Test POD
all_pod_files_ok();

Re:Ah good, people are getting it

jjore on 2007-04-12T16:42:38

No, AUTOMATED_TESTING is checking for the wrong thing. There's a distinction between things that should be run by the developers and things that should be run by the "mere" users.

Smoke vs interactive is a different distinction. The QA thread that couldn't settle on a name centered around AUTHOR_TESTS, I think. I'm happy considering that variable standard-enough.

Re:Ah good, people are getting it

chromatic on 2007-04-12T18:52:53

Reading some of the comments, it's nice to see more people are realising what a stupid methodology Test::Pod::Coverage has.

I'm starting to think that it's stupid because the Pod::Coverage metric is stupid.

use the right metric

Eric Wilhelm on 2007-04-18T08:34:14

Pod coverage is not a bad metric iff you use the right one. P::C is a bit too arbitrary about what symbols need coverage. Requiring the code to execute to check the docs is plenty scary too. And slow.

I've found that looking for a /=head2 $1\b/ to match every /^sub (\w+)/ actually catches places where I renamed the sub and missed the pod. usefulness++, noise-- and all that. Yeah, there's no PPI involved there. Honestly, does it need to be? Not for *my* coding policy.

Maybe somebody with a stricter policy wants to put the money into doing it more thoroughly, in which case PPI would be the way to go.