CPAN POD testing

barbie on 2004-07-21T18:13:00

A post was made on the Birmingham.pm list today, regarding some POD errors in a recent distribution, and reawoke a thought I had sometime ago, but one I haven't gotten around to implementing.

I currently use Test::Pod and Test::Pod::Coverage in all my distributions by default. I also plan to use Test::Distribution once I get the time to slot it in. However, not everyone does. As a consequence some trivial mistakes can slip through, which although not essential to the coding, are obviously worth noting for future releases. The thought I had was, along side the CPAN testing, to run a set of simple test scripts which run the three above mentioned modules against a named distribution.

Back when I did my CPAN Testing talk earlier in the year, this was something that crossed my mind, but as I was too busy doing other things, didn't really formulate how to do it. I have pondered about it off and on for a few months, and since the post mentioned today, the thought has popped up again. At some point I'd like to do this, but wonder how people would receive it. Note that any mails would only be sent to the author, and not to the cpan-testers list, as it's more a request for enhancement. Would an RT entry be prefered, sending to the CPANID, another method or just a really bad idea? Picking and choosing would not be possible, and personally would prefer the CPANID mailing that currently happens for CPAN testings, as it's the easiest to do.

Thoughts?

I wonder whether this might be a lightning talk?

---

CPAN Testing

domm on 2004-07-21T21:09:56

I'm currently working on CPANTS (finally found/took some time..). CPANTS does pod testing. And a lot of other things.

I'll do a talk on it at YAPC::Europe in Belfast, and if you (and others) are interested, we could do a BOF.

For a short preview, CPANTS currently creates YAML-files for each dist containing various information. Additionally, it creates a SQLite DB for easy querying of stuff.

Here's the yaml-file for Acme::Bleach (which contains 3 pod errors):

--- #YAML:1.0
dist: Acme-Bleach-1.12
distribution:
    dist_without_version: Acme-Bleach
    extension: tar.gz
    extractable: 1
    extracts_nicely: 1
    package: Acme-Bleach-1.12.tar.gz
    version: 1.12
    version_major: 1
    version_minor: 12
files:
    build_pl: 0
    count_bad_permissions: 0
    count_dirs: 4
    count_files: 19
    count_symlinks: 0
    list_bad_permissions: ''
    list_dirs: demo,lib,lib/Acme,t
    list_files: ! >-
      Changes,MANIFEST,Makefile.PL,README,demo/demo_DWIM.pl,demo/demo_bleach.pl,demo/d emo_bleach_c.pl,demo/demo_bleach_eng.pl,demo/demo_bleach_lisp.pl,demo/demo_bleac h_sh.pl,demo/demo_empty.pl,demo/demo_morse.pl,demo/empty,lib/Acme/Bleach.pm,lib/ Acme/DWIM.pm,lib/Acme/Morse.pm,t/DWIM.t,t/bleach.t,t/morse.t
    list_symlinks: ''
    makefile_pl: 1
    manifest: 1
    meta_yml: 0
    ninja: 0
    readme: 1
    signature: 0
kwalitee:
    extracts_nicely: 1
    has_buildtool: 1
    has_manifest: 1
    has_meta_yml: 0
    has_readme: 1
    has_version: 1
    kwalitee: 7
    no_pod_errors: 0
    no_symlinks: 1
    permissions_ok: 1
pod:
    errors: 3
release:
    date: Tue May 22 02:17:11 2001
    epoch: 990490631
size:
    packed: 4299
    unpacked: 11479

Re:CPAN Testing

barbie on 2004-07-22T11:16:21

Looks like you gone a lot further than I was thinking, great stuff. I will be very interested to hear your talk, assuming I'm not schedule to talk at the same time, so will be along if I can.

However, are you also a producing a report of the errors you've found. In some cases rerunning the tests on a.n.other box may not spot the problems, so it's handy to have a detailed list of what you tested with (Test:: module versions too) as well as the errors.

Re:CPAN Testing

domm on 2004-07-22T14:11:35

I'm planning to include various pieces of metadata in the results like the version of CPANTS used to generate data.

But including versions of important modules used (and perl version, BTW) is a good idea. Thanks!

BTW. I ran CPANTS today, you can view the results here (this is a very temporary URI..):
http://test1.dev.zsi.at/
(very raw results, that is: yaml-files in directory metrics, and a sqlite DB. no docs currently, sorry)

Re:CPAN Testing

BooK on 2004-07-22T16:22:16

What's ninja (in files)?

Re:CPAN Testing

domm on 2004-07-24T16:22:01

Ask someone at fotango.

"Ninja" was a topic at the CPLAN meeting, but not discussed (I think).

So I got interested and did some research (using CPANTS). All people using it are employed by Fotango. It containts what looks like roleplaying stats (Strenght, Dexterity, etc). But I don't know what it's for...

eg: http://search.cpan.org/src/LBROCARD/Data-Page-1.03/NINJA

CPANTS YAML output

nkuitse on 2004-07-22T16:56:14

Why not use normal YAML arrays, like this?

bad_permissions: []
dirs:
    - demo
    - lib
    - lib/Acme
    - t
files:
    - Changes
    - MANIFEST
    - ...
symlinks: []

Re:CPANTS YAML output

domm on 2004-07-24T16:43:09

Because I'm lazy :-)

And because the YAML gets dumped into a text field in a database, anyway.

Hmm..., when I think about it, I could dump the YAML array into the textfield.

Thanks for the suggestion..