I have often thought it to be a nice to have, now Schwern has implemented it:
http://www.nntp.perl.org/group/perl.qa/4401
The BAIL_OUT feature I only noticed today due to a post from Peter Kay. So if you have a test at the beginning that is crucial to the workings of the distribution, e.g. checking a network or database connection, if it fails you can stop all testing there and then. Lovely. Thanks Schwern.
See Test Control for further details.