On the Perl-QA list, Gabor Szabo pointed out a problem with the testing toolchain. Test::Most is advertised as a drop-in replacement for Test::More, which is was.
The problem is that Test::More has also added my explain function. explain is very handy in tests because it lets you do this:
explain $some_data;
It's like diag(), but only outputs when the tests are run in verbose mode. Thus, if you do make test, you won't see a bunch of irrelevant information dumped to the terminal. Further, it automatically calls Data::Dumper::Dumper() on references so programmers no longer curse when they see this:
# ARRAY(0x80631d0)
It's a very handy tool in testing, but Schwern was concerned that it was doing too much. What if you want to automatically expand the references, even when you're not in verbose mode? There's a great argument for this:
isa_ok $some_object, "Some::Class" or diag explain $some_object;
If there's a failure, you might want to see it right away. So in Test::More, explain just formats the data and you have to call one of the following:
diag explain $some_data; # always appears note explain $some_data; # only appears in verbose mode
That covers everything. I felt that a simple explain $some_data was the common case and I wanted to optimize for that. Schwern argued that it was trying to do too much and wanted the separation of concerns. That's a very reasonable argument, but what Schwern and I both missed was what Gabor pointed out. It breaks the Test::Simple to Test::More to Test::Most upgrade path. The functions behave differently and Test::Most is no longer a drop in replacement.
So let's take this to its logical conclusion (with apologies to Schwern for this fun :)
diag() formats your data and outputs it to the diagnostic filehandle (typically STDERR). But wait! It's doing too much and we should separate the formatting from the displaying. Since we can't just print it (we need to make sure it goes to the correct handle), we do this:
output diag explain $some_data;
Oh, that fails. Because output() doesn't know if it's outputting TAP or diagnostics, we need this:
output diagnostics diag explain $some_data;
Shove a few colons in there and we can call this a DSL!
However, what we really want is respect. We want this from the Java programmers. So with some Acme::Dot love, we can do this:
Tests.output.diagnostics.diag.explain($some_data);
Now those Java programmers are going to take us seriously!
I want explain $some_data; to just work.
… I thought the title of this entry was “Everything explain()”. The reality was a bit of a let-down.
Re:For a moment there…
Ovid on 2009-02-16T20:31:42
My reality is a let-down for many people. I'm getting used to it
:) Re:For a moment there…
schwern on 2009-02-16T23:21:09
Everything explain() Yoda does.
Something that is lost in the caricature is that by Test::Most::explain() wrapping up output and dumping you can not use explain() to output as part of a normal failure. I like to do this:
is_deeply $have, $want or diag explain $have;
Test::Most thinks the opposite use case is important, it's the equivalent of "note explain". If you read the docs for explain() in Test::More and Test::Most it's pretty clear we have different ideas about how it's to be used. I could have just decided which one the user gets to do, and screwed the rest, but I err on the side of flexibility.
It's worth noting that Test::More::explain() is hard wired to "prove -v" so not only does it make you rerun the test to get your diagnostics (which makes heisenbugs difficult to track), but you have to be running it through prove. prove and I are not joined at the hip, but since that's how most users do it I guess that's ok. Oh, and if you want to archive your test results you don't get the extra output unless you remembered to run it in verbose mode... through prove. But most people don't archive test results so I guess that's ok, too.
And really, it's five characters. A place in Pittsburgh I worked had a saying: "Just because you drive to Ohio doesn't mean you're going to California". Put another way, it's the Ben & Jerry's fallacy. That being since eating too much ice cream is bad for you, eating one scoop of ice cream is bad for you and you should never eat ice cream.
Looking at it another way, Ovid has declared a Domino Theory of Design. That one step towards Java will lead the surrounding code becoming more Java-like and so on until you're supporting Central American dictators and involved in two land wars in Asia. I mean... until half a year after the decision was made you're still arguing over five characters.
Speaking of reductio ad Java, oddly enough yes, that is basically the way Test::More is going and for basically the right reasons. But it is doing it *internally*, at the Test::Builder level and below what the user normally sees. A good interface is not just flexible, but knows when and how much flexibility to show. Java falls flat on it's face by giving you all the flexibility all the time. Equivalent to one of those monster all-in-one remote controls.
As an aside, I remember a year or two ago when Java programmers discovered that inside of having to create and pass in all the delegate objects to a constructor, the constructor could make the for you! And the they gave it some fancy name and declared that nobody else but Java had it. In short, they discovered this:
sub new {
my($class, %args) = @_;
# Wow, we can initialize objects for you!
$args{buffer} ||= Some::Buffer->new;
$args{storage} ||= Some::Storage->new;
return bless \%args, $class;
}
But I digress. A lot.
Deciding what diagnostic information to display to the user and when is a test by test choice, so it's part of the user visible Test::More interface. Deciding what filehandle to output to and what format to output is a choice that effects the whole test suite. It's a choice to be made once and so is pushed down to the Test::Builder level, out of the user's way but available.
The increased flexibility will allow you can flip a few switches and have all your testing modules start outputting XML or POSIX or nothing (oft requested so you can use testing functions as normal comparison functions) instead of TAP.
In the end, I point all this out as much because I get a kick out of needling Ovid as to show that Test::More and Test::Most have different design philosophies and that's just fine. It's all working as designed. We don't have to agree on the One True Way because there is none. Different modules addressing different use cases but all working together and all benefiting from the flexibility built into the system.
The Inertia Theory of Design
Ovid on 2009-02-16T21:37:41
You're absolutely right that my solution is not as flexible. But
... Years ago I worked at a company where we made revenue projections for a particular industry. Industry executives could log on and when they saw what they were trying to project, they had a "weighting" factor that they could enter and the numbers would be adjusted with that. For the sake of argument, we'll say that number defaulted to 12, but based on the executive's knowledge of what they were offering, they would adjust that up or down.
After six months of hard labor, one programmer had revamped the system and the weighting factor defaulted to "15". Our customers were livid. They accused us of "cooking the books". Even though our new numbers were more accurate, somehow the executives thought we were cheating and demanded that we change the default back to 12. Our better revenue projections, ironically, became a PR disaster.
I and another programmer were called into a meeting with a Vice President and he asked us to change the number. The other programmer went to the whiteboard and started sketching. He explained how the resources worked, what weights were assigned to them, how revenue was multiplied by those weights and how six months of intensive regression analysis and revamping of our statistical model, blah, blah, blah and circles and arrows and a paragraph on the back of each one.
Fifteen minutes later, the programmer finished. The vice president looked at him and said "Yeah, now can you change the f***ing number to 12?"
Inertia is a terrible thing
:) Re:The Inertia Theory of Design
schwern on 2009-02-16T23:20:05
Yes, inertia driving design is bad. What does it have to do with this?
Re:The Inertia Theory of Design
Ovid on 2009-02-17T10:12:40
The vice president argued for a suboptimal solution because that's how things are now. Though I've got other reasons for wanting explain() to function the way it does, part of the reason to keep it the same is because that's how things are now.
Re:The Inertia Theory of Design
schwern on 2009-02-17T20:18:07
I think it's the "But..." that made me think it was an accusation that Test::More's explain() is the result of Design by Inertia... which doesn't make any sense since Test::More can't "do it the way it was always done" for something that didn't exist. Design by Inertia would have been to do what Test::Most does.
Are you considering changing explain() in Test::Most?
Re:The Inertia Theory of Design
Ovid on 2009-02-17T20:40:32
I wasn't planning on changing explain() in Test::Most. From what I can tell, it's now used widely enough that I'm not keen on breaking backwards-compatibility without a strong reason.
Re:The Inertia Theory of Design
schwern on 2009-02-18T20:53:03
So 12 it is then!
:P Re:The Inertia Theory of Design
schwern on 2009-02-16T23:22:46
PS That was at R*****k, right?
Re:The Inertia Theory of Design
brian_d_foy on 2009-02-16T23:41:56
I had a job like that. A huge commericial real estate company wanted to have all sorts of fancy graphs about prices and market conditions, but they didn't know how to do it themselves.
It turns out prices had nothing to do with the market. They just increased monotonically. They lost interest in showing customers market data after that.
In that case, I'd change the number back to 12, but insert a +3 somewhere else.
:)
