Test::Harness ouput change

Ovid on 2008-08-21T09:34:35

It's just a minor change that both Andy and I have wanted for a long time, but I do wonder if it will impact others. Instead of seeing this (use.perl inserts extra spaces, so this might not line up right):

t/proverun......................ok   
t/regression....................ok      
t/results.......................ok       
t/scheduler.....................ok       
t/source........................ok     
t/spool.........................ok   
t/state.........................ok     
t/streams.......................ok

You'll see this:

t/proverun .................... ok   
t/regression .................. ok      
t/results ..................... ok       
t/scheduler ................... ok       
t/source ...................... ok     
t/spool ....................... ok   
t/state ....................... ok     
t/streams ..................... ok   

Both of us found this much cleaner. However, this might have unexpected consequences. It also highlights the issue of Test::Harness's long-standing practice of stripping the .t extension from filenames. Why? If we want other extensions, stripping them is probably bad.


not a problem for CPAN::Reporter

dagolden on 2008-08-21T10:46:02

CPAN::Reporter ignores that and just looks for the grade at the end, so green light from me.

What do you do if the test file name is longer than your fixed column width?

t/something/something-else/with-a-long-long-long-long-test-name.t

-- dagolden

Re:not a problem for CPAN::Reporter

Ovid on 2008-08-21T13:28:10

The width of the reporting is based on the width of the test names, so it expands as needed.

Re:not a problem for CPAN::Reporter

jk2addict on 2008-08-21T13:46:34

And when that gets past 72/80?

Re:not a problem for CPAN::Reporter

Ovid on 2008-08-21T15:52:45

We don't worry about that. If you prefer a different format, you're welcome to subclass :)

Re:not a problem for CPAN::Reporter

jk2addict on 2008-08-21T17:11:28

Good. Now I can finally stop sticking to 72/80 in my perl source files too. Use a different editor/console! :-)

Re:not a problem for CPAN::Reporter

jk2addict on 2008-08-21T20:01:51

And Yes, apparently I'm full of piss and vinegar today. Maybe this is what happens when I drink coffee. :-)

Re: Test::Harness ouput change

jmcnamara on 2008-08-22T09:16:56

While we are on the topic of Test::Harness output, a change that I'd like to see is padding of the test number so that test comments are aligned.

For example consider the following trimmed down test program and output:

use Test::More tests => 11;

for (1 .. 11) {
    is(1, 1, 'This is a comment');
}

__END__

perl output1.t
1..11
ok 1 - This is a comment
ok 2 - This is a comment
ok 3 - This is a comment
ok 4 - This is a comment
ok 5 - This is a comment
ok 6 - This is a comment
ok 7 - This is a comment
ok 8 - This is a comment
ok 9 - This is a comment
ok 10 - This is a comment
ok 11 - This is a comment

I'd prefer to see that aligned something like the following so that the comments were aligned (using something like "%-4d"):

ok 1    - This is a comment
ok 2    - This is a comment
ok 3    - This is a comment
ok 4    - This is a comment
ok 5    - This is a comment
ok 6    - This is a comment
ok 7    - This is a comment
ok 8    - This is a comment
ok 9    - This is a comment
ok 10   - This is a comment
ok 11   - This is a comment

I usually put some work into having good test comments and having them left aligned is important to me.

To workaround this I usually format my comments with a leading "\s\t":

use Test::More tests => 11;

for (1 .. 11) {
    is(1, 1, " \tThis is a comment");
}

__END__

perl output2.t
1..11
ok 1 -          This is a comment
ok 2 -          This is a comment
ok 3 -          This is a comment
ok 4 -          This is a comment
ok 5 -          This is a comment
ok 6 -          This is a comment
ok 7 -          This is a comment
ok 8 -          This is a comment
ok 9 -          This is a comment
ok 10 -         This is a comment
ok 11 -         This is a comment

Do you think that there would be any appetite for this change?

John.
--

Re: Test::Harness ouput change

Ovid on 2008-08-22T09:36:43

I like this thought because things which improves comprehension are good, but there are a couple of issues here. First, many test descriptions (what you're referring to as a comment), are often close to the width of whatever terminal I'm working with, so pushing them out even further may be problematic. Second, since we don't always know up front how many tests will be run, we won't know what padding would be needed.

However, rather than a specific solution like this, how about a more general one?

prove -rv --format='%s %4s - %s' t/

The danger is that passing in a format means you can easily break TAP:

prove -rv --format='%2$s %1$4s - %3$s' t/

There's not much we could do to stop that, but it might just be a caveat emptor situation.

Re: Test::Harness ouput change

Aristotle on 2008-08-23T03:58:49

There’s not much we could do to stop that

You could generate a test message with this format and try to parse it and see that the parts all where you expected them. (Should the parts be randomly generated so the user can’t fool you?) If that fails you tell the user that their format is no good and bail.

Re: Test::Harness ouput change

Aristotle on 2008-08-23T04:00:30

Should the parts be randomly generated so the user can’t fool you?

Ah, no, there is a simpler way, which is to try twice, with messages that differ in all parts.

Looks nice.

schwern on 2008-09-08T03:26:19

I like it.