This Week on perl5-porters - 28 April-3 May 2008

grinder on 2008-05-09T12:47:00

I've been playing around with MAD this morning [...] The current state of the tests for MAD is a bit sad, though I hope to have something that will make interested porters glad in the near future. -- Jesse Vincent, not a bad lad to spend a tad on MAD. (gad, Dad! it's not a fad).

Topics of Interest

More on tainting

Nicholas Clark thought that a of points Paul Fenwick made regarding late-acting taint weren't borne out by an inspection of what the source code actually did.

Although, looking closely, he found a couple of suspect constructs. He thought that one possibility was to alter $^TAINT to be undef if no tainting, 1 (or true) if -T tainting, or 0 for late-acting tainting.

Then again, Paul's idea of deprecating late-acting tainting sounded like the path of least resistance.

  better to be safe than sorry
  http://xrl.us/bkb8d 

Support mallocs of struct T, which contains struct S[N]

Nicholas apologised to Jim Cromie for having not applied his patch, noting that at the moment Rafael seems to be the only person with enough motivation to apply patches.

He thought that Jim's avenue of research seemed promising and suggested he keep slogging on to see whether the rest of it works.

  http://xrl.us/bkb8f 

Bug in if(open(my $fh,...))

In a parallel thread (to bug #53504), Matt Sergeant reopened the discussion about filehandles persisting past their due date. The problem arises when one opens a file as a side effect of a conditional in an if statement.

When Matt learnt that it was sufficient to wrap the if block in a curly block to force the handle to be closed early, he deemed that it was no longer such a problem.

Hint: if bug reports don't contain the word "perl", they run the risk of being filed in a "possible spam" folder for human perusal.

  it's all right
  http://xrl.us/bkb8h
  http://xrl.us/bkb8j 

5.6.x in the wild

A recent change to the GNU coreutils package has broken the Configure script for 5.6.x (or rather, it's been fixed for more recent perls for ages, but hasn't been, and is unlikely to be backported to 5.6.x).

Jesse Vincent asked for a show of hands to see how many porters are still using 5.6, to gauge the importance of creating a fix for software that was released eight years ago.

Gabor Szabo won a surprisingly crowded race, by having the courage to admit that he has a client still running perl 4.x in a couple of places.

  an unscientific survey
  http://xrl.us/bkb8m 
  GNU coreutils 6.9 breaks F<Configure> (#53446)
  http://xrl.us/bkca2 

Prototypes, parsing and optrees

Nicholas Clark thought there was a smidgen of performance to be gained by figuring out what context a routine needed to used to call a function and that maybe a prototype, or its absence, could help. But Rafael explained that what he was looking at was actually from something completely different, so the question is moot.

  http://xrl.us/bkb8o 

Weird kernel failure in t/op/threads.t in blead@33758

David Landgren related the tale of a test suite falling into an infinite loop, spewing out the message

  Fatal error 'Exceeded maximum lock level' at line 199 in file
  /usr/src/lib/libpthread/thread/thr_rtld.c (errno = 136039556)

and wondered if it was the symptom of some recent change. Nicholas Clark thought that the fact that the test was run on a 6-cpu machine running FreeBSD 6.0 was probably a better explanation.

Nicholas also thought it would be slightly more helpful if the -V switch were able to report the number of CPUs on the machine, at least for a subset of platforms where such a test was easy enough to determine.

Abe Timmerman looked at the code in Test::Smoke::SysInfo and wrote no, but I think he meant yes. If so, it should just be a simple matter of programming...

  http://xrl.us/bkb8q

  http://xrl.us/bkb8s 

Smoker segfaults on OpenBSD 4.1

Stephen Schubiger reported that smoke tests of blead had been crashing with what turned out to be low-level XS problems. Abe Timmerman had heard of the problem also occurring on FreeBSD 7.0.

In the meantime, Stephen discovered that the latest release of the smoker's companion (Test::Smoke 1.32) fixed the problem.

  smokers of the world, upgrade
  http://xrl.us/bkb8u
  http://xrl.us/bkb8w 

5.8.9-to-be on Irix

David Cantrell reported the results of a smoke for the upcoming 5.8 release on IRIX. Everything was fine except for a couple of failures in Sys::Syslog. One issue involved taint, the other was possibly related to CPAN# 30710, but Sébastien Aperghis-Tramoni had no real idea about the heart of the problem.

  http://xrl.us/bkb8y
  http://xrl.us/bkb82 

Similarly, the same problem also occurred with blead, although Time::Piece and Archive::Extract were also giving grief.

  bleeding on Irix
  http://xrl.us/bkb84 

Parameterize ptr_table_new(table-size)

Jim Cromie delivered another fiendishly obscure patch to improve the performance of ptr_tables, which serve an important role in implementing hashes. Part of the aim was to expose the guts so that things like Storable could delve deeper into the core for an increase in performance.

Nicholas Clark was a bit uncomfortable with the idea, expressing the wish to see more performance tweaks that don't expand the public API. Apart from that, he felt that Jim's research might be help re-tune various magic numbers used for sizing things in the core.

The main sticking point was a useful set of benchmarks to help examine the result of changes to the core.


  http://xrl.us/bkb86 

Another perldoc shortcut

It suddenly dawned on Gisle Aas that instead of typing perldoc perlintro one could just as easily type perldoc intro. He wrote a patch so that if it failed to find something, it would prepend "perl" to the search string and try again. Thus perldoc re would continue to return the page on the re pragma, rather than the perlre page.

  all that's lacking is shell auto-completion
  http://xrl.us/bkb88 

Adriano Ferreira liked the idea so much he pushed Pod::Perldoc version 3.14_06 out to CPAN and asked for feedback.

  RFC
  http://xrl.us/bkb9a 

Upgrading to Digest::SHA 5.47

The CPAN version has apparently been moving files around in the distribution, and this is naughty, since it causes more work when integrating into blead. Still not sure if it's all tidied up yet.

  bumping into things in the night
  http://xrl.us/bkb9c 

Bug or not? constants warn only once

Nicholas Clark excoriated the list for wasting time on pointless questions (why is Ruby slow?) rather than answering more mundane questions which would help things move forward. The issue at hand was whether constants should warn once, or as many times as necessary. Both outcomes are doable, and feasible, so we just need to decide what to do. All he wanted was someone's opinion.

Aristotle Pagaltzis wanted to be sure that whatever was changed wouldn't make things slower than they are now, expressing the opinion (yes!) that things had become slower over time (although for the most part the price was happily paid). He also took the time to rewrite Nicholas's proposed tests into something he felt was easier to read.

Dave Mitchell was very surprised to learn of 2x slowdowns in the regexp engine, and thought that if this was true then it was something that needed fixing.

Aristotle referred him to an article by Ben Tilly.

  Ilya's insight
  http://perlmonks.org/?node_id=502408
  http://xrl.us/bkb9e 

MAD Dumper - missing encoding of XML-unsafe literals in "pre"

Jesse Vincent looked at the state of MAD mode in Perl and was dismayed by the paucity of tests as well as the way it generated XML. Part of the problem was the fact that the test suite relies on XML::Parser, which is a slightly odd state of affairs to have. At least until Dave Mitchell gets around to writing an XML parser using /bin/sh.

Unfortunately Jesse used 5.10.0 as a basis, and significant changes have been made in 5.11 and thus the patch could not be applied. Plus Gerard Goossen had already fixed it. Nevertheless, had it been applicable, it would have leaked a scalar, although fortunately Rick Delaney was around to show Jesse how that could have been fixed.

  teacher tell me what's my lesson
  http://xrl.us/bkb9g 


TODO of the week

The yada yada yada operators

Perl 6's Synopsis 3 says:

The ... operator is the "yada, yada, yada" list operator, which is used as the body in function prototypes. It complains bitterly (by calling fail) if it is ever executed. Variant ??? calls warn, and !!! calls die.

Those would be nice to add to Perl 5. That could be done without new ops.


Patches of Interest

weak.t

Alexandr Ciornii offered a patch to allow Scalar::Util's weaken function to be tested correctly both in the CPAN module and in the perl core.


  and strong coffee
  http://xrl.us/bkb9i 

Testing $/ with in memory files

Bram noticed that bug #44833 had been resolved, but no tests were ever added to help pin down the issue. There was an issue with the test being run by miniperl, which would possibly give it fits, so Bram tweaked it to be skipped if run by miniperl.

Rafael applied the patch as it stood, but Ben Morrow thought of a tweak for additional robustness. Bram adopted the idea, but it was apparently left unapplied.

  http://xrl.us/bkb9k 

Extra tests for t/op/tie.t

As a follow-up to the READONLY bug from last week, Bram added a couple of tests to make sure things stayed sane. After have mulled over the issue for a while, he returned to it again this week, asking the porters whether a deeper overhaul was needed concerning these tests.

  Warnocked
  http://xrl.us/bkb9n 

Win32 process ids can have more than 16 bits

Jan Dubois tweaked the Win32 code to return the full 32-bit PID value on Windows platforms. He also mentioned that system behaves slightly differently on Windows, in that it returns the PID and not the exit status.

Paul Johnson thought it was high time (since he's been waiting on it for ten years) to either document the issue properly, or come up with a better interface.

  32 bits ought to be enough for anybody
  http://xrl.us/bkb9p 

Two less double magic warnings

Vincent Pit wrote a patch to kill a redundant warning in the construct binmode $fh, undef

  http://xrl.us/bkb9r 

as well as tie $x, $m when $m is undefined.

  http://xrl.us/bkb9t 

Rafael liked these patches so much, he applied them.

Detab and move POD in Pod/Html.pm

In a prelude to major cleanups in Pod::Html, David Landgren submitted a patch to cleanse the whitespace of tabs, and a second patch to push the POD down to the end of the file (hint: this comes in handy when using Devel::Cover).

H.Merijn Brand was loathe to apply the patches, since he remembered Tom Christiansen vehemently opposing the change last time someone tried to reformat the code for "stylistic" purposes.

  to be continued
  http://xrl.us/bkb9v
  http://xrl.us/bkb9x 

David also noticed that the tests for Pod::Html relies on being able to find /bin/diff or /usr/bin/diff and wondered if is() from Test::More wouldn't do just as well.

  evil external OS-dependent dependencies
  http://xrl.us/bkb9z 

~~ with non-overloaded objects

With a very small dash of C, Vincent Pit tweaked smart matching to croak on a non-overloaded object. This made Ricardo Signes, who raised the issue in the first place, very happy.

  r e s p e c t (staying out of the living room)
  http://xrl.us/bkb93 

Add some links to external resources

Shlomi Fish suggested some links to web pages in the documentation, but qualified his patch with a complex web of licensing details. This prevented the patch from receiving much serious consideration.

  licensed to patch
  http://xrl.us/bkb95 


New and old bugs from RT

Pod::HTML generates incorrect HTML for =item * (#19655)

David Landgren went through the open bugs for Pod::Html and noticed that this one has been fixed as recently as in 5.10.0.

  resolved
  http://xrl.us/bkb97 

Bug in File::Find on Windows when target dir is bare drive letter and no_chdir = 1 (#41555)

Bram wrote a patch to correct the problem, but wondered how one would go about how one could write a test for the behaviour.

  http://xrl.us/bkb99 

Infinite recursion related to die, overloading, s/// and \x{...} (#41618)

Bram noted that this problem is still around on blead, and was able to refine the test cases to show what does and what doesn't cause a segfault.

  getting better all the time
  http://xrl.us/bkcab 

Unicode and case insensitive regex (#41664)

Bram discovered that for some reason the problem goes away if anchors are used in the pattern match.

  http://xrl.us/bkcad 

Carp::croak() replace user error message by own (#42329)

Bram wrote a patch to make croak() croak with a more useful error when there are no file descriptors available to pull in Carp::Heavy.

  now you know why
  http://xrl.us/bkcaf 

Data::Dumper ignores ^M in scalars? (#43617)

Bram tried to convince the bug reporter that this was more a problem of shell interpretation of newlines rather than being the fault of Data::Dumper.

  http://xrl.us/bkcah 

Missing tests for sprintf %f with Nan/+Inf/-Inf (#45383)

Bram wrote some patches to pin down the behaviour of sprintf. Applied by Rafael Garcia-Suarez.

  http://xrl.us/bkcaj 

Clarification about @ISA declaration in perlboot.pod (#45733)

Bram saw that the patch attached to this bug report had never been applied, and asked why. Rafael explained that he never saw it on the mailing list the first time around.

  will it be applied this time around?
  http://xrl.us/bkcam 

Split function broken on MacOS (#46073)

Bram found another patch that had been overlooked and so Rafael applied it.

  http://xrl.us/bkcao 

perl -x reports incorrect line number for errors or warnings (#46369)

Anon Sricharoenchai thanked Bram for his followup on how to resynchronise perl's concept of line numbers when dealing with files that contain Perl and non-Perl chunks.

  or try Inline
  http://xrl.us/bkcaq 

Clearing magic (or length($@)>0 for empty $@ if utf8 is in use) (#51370)

Animator had opened this can of worms last week on the clearing of magic on $@ with an exploration of sorts on how to fix it.

Rafael Garcia-Suarez commented saying that the results looked reasonable, but of course one could only be sure with a series of new tests to ensure that nothing had been broken.

So Bram wrote some code and tests and asked for comments. Nicholas Clark thought that the code Bram had written had the freeing and clearing of magic the wrong way around. The code also introduced a new class of macros to deal with the situation; Nicholas felt this added unnecessary complexity.

Another thing that caught his eye was the fact that one of Bram's tests induced a panic: sv_len_utf8 cache 17 real 0. This will need to be looked into. Bram thought that this was probably a side effect of the UTF-8 flag not being cleared when it should have been.

So Nicholas did that, and a after thinking about it for a bit thought that mg_free might be to blame. He and Bram kicked the idea around a bit more and at one point a proposed change might have had the side effect of allowing $@ to become tie-able. Nicholas thought that this was a bit sick, but then again, hmmm. There has been talk of promoting $@ to a first-class object...

  http://xrl.us/bkcas
  http://xrl.us/bkcau 

perl-5.10.0-33733 assertion with JSON::XS-2.2 (#53244)

This is the thread with Marc Lehmann discussing the use of SvCUR. The only followup this week was from Sam Vilain, wondering if there was a problem or not with sv_upgrade and whether Marc might not be too liberal in choosing when to access SvCUR.

  http://xrl.us/bkcaw 

Parse problem in Switch.pm (#53414)

Don't use it. Even the author says so.

  when given a real switch
  http://xrl.us/bkcay 

A bug with Readonly::XS that might be a guts bug (#53482)

A change in the core might have been the cause of the problem. Bram added some tests to make sure things didn't drift in the future. After a couple of idle suggestions, Andreas König came through with the patch that seemed to be the root cause.

  http://xrl.us/bkca4
  http://xrl.us/bkca6 

op/pwent.t should use the DirectoryService on OS X (#53500)

Jan Dubois outlined the approach that he thought should be taken to fix this bug, admitting that he didn't have the time right now to pursue the issue.

  feel free to jump in
  http://xrl.us/bkca8 

Some UTF-8 string with CP932 encoding will freeze DOS BOX (#53502)

Chihiro reported that writing a program in a Japanese environment worked, but if the program had a different name, it would freeze inside a Windows DOS box. No comments.

  might be a ^S
  http://xrl.us/bkcba 

Bug in if(open(my $fh,...)) { } scoping (#53504)

Matt Sergeant was surprised to discover that $fh was not destroyed until the end of the scope encompassing the if, rather than at the end of the if block.

In the subsequent exchange, a work-around was suggested, and the usefulness and hatefulness of lexicals "leaking" across if/elsif/elsif blocks was debated.

  been there since 5.003
  http://xrl.us/bkcbc 
  continued in May
  http://xrl.us/bkcbe 

Overload and fallback binary or (|) (#53550)

Bram opened a ticket related to bug #53482 which showed the overload fallback mechanism privileging string context rather than numeric context, which, in the context of the bug, would have been more useful.

Rafael explained that binary or (|) works on both string and numbers, so it would be hard to discern what was needed in a systematically correct way. David Nicol speculated on the possibility of allowing the author of the overloading to give numeric context the first chance and resolving the overload.

  "we travel on the quiet road"
  http://xrl.us/bkcbg 

Range operator and magic (#53554)

Bram opened another ticket regarding the range operator, wondering why the interpreter inspects the contents of $x six times in the fragment @y = $x .. $x, where $x was a tied variable (and thus if the magic was being used to piggy-back side effects, the double magic would throw things out of kilter).

The prompted Dave Nicol to ask if the range 1..$x was driving a for loop, what would happen if the $x value was modified in the loop. Bram thought that it shouldn't be allowed, and promised a TODO test for it.

  magic and loss
  http://xrl.us/bkcbi
  http://xrl.us/bkcbk 

Patch for linux LDAP groups (#53560)

David Dick encountered a test failure regarding groups (as in /etc/group) containing spaces in names. This occurred on a Linux box using LDAP authentication, and so his expedient fix was to add Linux to the list of platforms where group names could be expected to contain spaces.

Rafael applied the patch, but H.Merijn Brand rightfully pointed out this scenario would play out the same way on many other Unix-like platforms. In which case, a much more robust solution would be to determine if LDAP (or in fact any NIS/NSS mechanism, come to think of it) was in use, and use that as a basis for the test.

  wanted: finer discriminant
  http://xrl.us/bkcbn 

Zero-length regex bug (#53562)

Another victim failed to notice that // has special meaning.

That the problem comes up so frequently caused David Nicol to conclude that the documentation needs a special EMPTY PATTERN section.


  only a wafer-thin documentation patch needed
  http://xrl.us/bkcbp
  http://xrl.us/bkcbr 

perl5.10 and blead crash on win32 (#53586)

Dmitry Karasik uncovered a snippet that runs fine on FreeBSD but crashes on Win32. Then again, it does involve IO::Handle, SIGPIPE signals and setting far too many things to undef.

  bad magic
  http://xrl.us/bkcbt 

Perl5 Bug Summary

All hail Bram, the new Perl5 bug warrior.

  288 new + 1472 open = 1760
  Created this week: 8
  Closed this week: 64
  may his wrangling be long and fruitful
  http://xrl.us/bkcbv

  http://rt.perl.org/rt3/NoAuth/perl5/Overview.html 


In Brief

Bram's documentation patch for perlrun -x made it in

  http://xrl.us/bj2wx 

as did Vincent Pit's patch for segmentation fault with array ties (bug #51636).

  http://xrl.us/bkcbx 

On the Why is Ruby on Rails so darn slow thread, Alberto Simões forwarded a message from Leopold Tötsch that gave some Parrot benchmarks on crunching Mandelbrot sets.

  http://xrl.us/bkcbz 

John E. Malmberg took a second shot at getting the forked debugger working on LINUX/UNIX/CYGWIN/VMS.

  http://xrl.us/bkcb3 

Marc Lehmann read a fascinating paper on heapsort with n log(n) - 0.9n comparisons which is interesting because it reduces the number of comparisons that need to be performed, which is useful in a language where the compares themselves are expensive.

  http://xrl.us/bkcb5 

perl @ 33444 was still having problems in Cygwin due to db-btree.t, io_multihomed.t, HiRes.t and op/alarm.t.


  http://xrl.us/bkcb7 

Jesse Vincent found a teeny tiny typo in Encode 2.24's Makefile.PL . Applied.

  http://xrl.us/bkcb9 

Last fortnight's summary

  This Fortnight on perl5-porters - 13-27 April 2008
  http://xrl.us/bkccb 

About this summary

This summary was written by David Landgren.

Weekly summaries are published on http://use.perl.org/ and posted on a mailing list, (subscription: perl5-summary-subscribe@perl.org ). The archive is at http://dev.perl.org/perl5/list-summaries/ . Corrections and comments are welcome.

If you found this summary useful, please consider contributing to the Perl Foundation or attending a YAPC to help support the development of Perl.