This Fortnight on perl5-porters - 28 September-12 October 2008

grinder on 2008-10-19T22:49:00

This Fortnight on perl5-porters - 28 September-12 October 2008

"... because a git repository is a collection of assembled changes, not a linear progression, using a commit ID as a way to find out where your code came from doesn't work. This is not git's fault but is an issue with every distributed version control system (SVK tries to pretend otherwise). You could perhaps use a combination of the git repository you pulled from and the last commit ID, but [...] fundamentally the idea is gone. It's a small price to pay." -- Michael G. Schwern, coming to terms with the future.

Topics of Interest

$VERSION in Opcode

The tangent regarding the issue of module version numbers and the techniques to deal with them continued apace over the last fortnight. The core of Matt S. Trout's argument over version.pm is that it injects a method into UNIVERSAL, and this, in an of itself, produces action at a distance, even when version.pm is not at fault. The problem shows up when version.pm is used incorrectly by third-party code, and the resulting errors are very hard to track down.

All that being as it may, Matt appreciates the value of the version comparison code available, and would like to be able to get at it, while leaving UNIVERSAL untouched. Otherwise, it's akin to modifying $[ or $/ at the global level. Tom Christiansen concluded that only the highest level code should be putting things into UNIVERSAL, not by proxy via the use of other modules.

  universal language
  http://xrl.us/oudw3

  http://xrl.us/oudw5 

Later on, in a particularly magnanimous gesture, John Peacock announced that he had a patch that would do all that Matt wanted and more, and asked if Matt was satisfied by it.

  UNIVERSAL::VERSION and version.pm
  http://xrl.us/oudw7 

Y2038 branch

Steve Hay took Michael G. Schwern's Y2038+ code for a spin in the land of Windows, and made a few adjustments to have it play nicely with the compilers there.

Based on Steve's recommendations, Michael produced a series of changes that ended up delivering sane results. In the end, Michael had to lift some code from the FreeBSD project to address the problem of converting strings to long longs. This helped the Microsoft compilers that still had not caught up with the C standard as of the previous millennium.

  party like it's 1999
  http://xrl.us/oudw9
  http://xrl.us/oudxb 

Later on Michael released a newer patch that included all his latest research on the matter. Steve continued to find a number of rough edges on Windows. H.Merijn Brand thought that it might be worthwhile probing for 64-bit variants of time routines on the platforms that offer them, since that the whole point of the exercise in the first place. And so he did, and said he'd get around to probing for the reentrant versions as well.

  duty now for the future
  http://xrl.us/oudxd 

Michael then had a patch working on Windows and Cygwin, contingent on a couple of details that needed to be resolved. Some problems turned out to involve the building of miniperl itself, which uses some canned constants before Michael's work is able to work its magic.

Dave Rolsky dropped by to say that he was delighted that all of this work had a positive effect on the innards of Time::Local. On the other hand, it greatly complicated the task of keeping it dual-lifed on CPAN, so much so that he doubted that the new changes would work with older perl releases. Michael showed him how this could be done with some XS goodness.

  just in time
  http://xrl.us/oudxf 

Crash when dying on warning from backticks with vfork() (5.6.1 on Lamp)

Joshua Juran is the caretaker for Perl on the traditional Mac OS (that is, pre OS/X). Said platform doesn't know how to fork, but it can vfork. In the current state of affairs, asking a program to fork itself with something silly, like /dev/null, leads to some very interesting (read: not useful) results, in that one receives very little information as to what went wrong.

Joshua managed to get as far as showing how a minor change to Perl_do_exec(), Perl_my_popen(), and pp_system() would fix the problem, but he wasn't sure if his internals-fu was strong enough to consider all the ramifications.

Nicholas Clark reasoned that the problem Joshua faced was sufficiently weird (or rather, his platform of choice was weird) that he should write the tests to establish the desired behaviour out on the end of the continuum, and then the rest of us could just follow suit.

Apparently he has done this on a code repository hosted on Sourceforge, but unless I am mistaken they didn't make it to the list.

  pneumatic tubes
  http://xrl.us/oudxh

  http://xrl.us/oudxj 

Discussing the (im?)possibility of prototype-based polymorphism for 5.12

David Nicol responded to a previous thread (which I lack the tuits to track down) with the concept of polymorphism based on prototypes, which would, amongst other things, make the following work:

  sub foo($$);
  sub bar(@);
  bar foo 1, 2, 3, 4;
  # parsed as
  bar( foo(1, 2), 3, 4);

He promised to investigate the possibility with his Macrame module. In the meantime, Rafaël Garcia-Suarez reminded David that prototypes in Perl are more for changing the syntax rather that type inferencing.

  http://xrl.us/oudxm 

Small incompatibility between 5.8.8 and 5.10.0

Torsten Förtsch discovered a problem that had crept into 5.10, which appears to be something to do changes to the implementation of stashes. No-one commented.

  something new with ::
  http://xrl.us/oudxo 

protochk and C++

Tony Cook had been puzzled for a while over intermittent smoke failures when using C++ on Debian, and managed to discover that C++'s function overloading mechanism was the cause. He suggested a change to a Configure test, but H.Merijn admitted that it was far outside his scope of expertise and asked for other eyes to look at it and consider the ramifications.

  looking the other way
  http://xrl.us/oudxq 

TODO list for moving to git

Nicholas, having endured a painful bisection ordeal under Perforce to track down when something changed, wondered what was holding up the migration to git. His current problem was one of mapping a Perforce revision (a number in a monotonic sequence) to a git checksum.

Mark Mielke wanted to know if there was a page on the web somewhere that recorded the current status of the Perforce-git migration.

Michael G. Schwern offered an interesting parenthetical remark regarding the move from Perforce to git, and why it is the Right Thing. David Golden also noted that git solves the problem of branching at any point in time, hacking, and folding the changes back in a most elegant manner. The price you pay, though, is that it is very hard to point to a specific moment in development (since one only has an unsortable checksum, not a monotonic sequence).

Craig A. Berry, not fully versed in git-speak, was nonetheless able to describe how people would communicate points of interest in the development tree. At the same time, he thought that the barrier to entry to understanding git was much higher than that of Perforce. As it turns out, basic web projections of git repositories already come close to what the All Perl Changes browser provides.

  gitting there
  http://xrl.us/oudxs 

Michael went through the git threads on the list and neatly summarised them on the Perl5 wiki. He noted with satisfaction that there were no showstoppers, just a question of tuits.

  http://xrl.us/oudxu 

Perl_newSVpvf("%lld") is broken

One other item that fell out of the Y2038+ code was that, on 32-bit platforms, printf("%lld\n", 2**32) gives a result of -1. Michael and admitted to being terrified by the code and hoped someone else could dig down through it and figure out where the problem lay.

  quad wrangle
  http://xrl.us/oudxw 

Pathtools QNX regression

Nicholas Clark looked more closely at Matt Kraai's failures for 5.8.9-tobe on QNX. He determined that it was due to a refactoring of File::Spec::Unix::abs2rel between Pathtools 3.12 and 3.13.

Interestingly, this dredged up an interesting piece of Perl trivia, in that for a short time, the /t flag on a regexp match maintained the taint flag if the target string was tainted. The idea lived a short life.

Nicholas thought that creating a new File::Spec::QNX (and the like), since it would be easier to test and would probably simplify the implementation of catdir. In any event, he wrote a patch and Matt confirmed that it fixed up the problem on QNX 6 (a.k.a QNX Neutrino, or $^O eq 'nto').

Ken Williams folded Nicholas's patch into his Pathtools repository.

  http://xrl.us/oudxy 

git conversion discrepancy

Nicholas Clark uncovered a flaw in Same Vilain's Perforce-git import. Sam said he'd go over the import code to see if he'd missed anything. By and large most of the Perforce branches were bit-for-bit perfect.

  ancient history
  http://xrl.us/oudx2 

perl@34467

Nicholas produced another snapshot of 5.8.9-tobe. Of note were some problems on VMS (for which fixes existed) and some minor problems with Module::Build.

Slaven Rezic tested some 3700 CPAN distributions and listed a dozen or so regressions. Most of them turned out to be minor, but one was an honest to goodness C error, relying on a reallocated SV landing in exactly right place.

  getting closer
  http://xrl.us/oudx4 

Fedora (Assertion failure on glob assignment)

Nicholas Clark wailed and gnashed his teeth at vendors who insist on foisting a perl compiled with -DDEBUGGING upon their users, and pondered tweaking things to make it produce a nasty warning on STDERR during startup.


  make it so they have to reboot
  http://xrl.us/oudx6 

t/op/regexp.t

Abigail wondered how to deal with a test that causes a segfault during program exit, since even a fresh_perl_is from test.pl will return a correct result, not seeing that $? contains a non-zero value. Michael G. Schwern thought that the expedient measure of testing $? against zero should be sufficient.

  measuring success
  http://xrl.us/oudx8 


TODO of the week

A task that need a little sysadmin-type knowledge

Linker specification files

Some platforms mandate that you provide a list of a shared library's external symbols to the linker, so the core already has the infrastructure in place to do this for generating shared perl libraries. My understanding is that the GNU toolchain can accept an optional linker specification file, and restrict visibility just to symbols declared in that file. It would be good to extend makedef.pl to support this format, and to provide a means within Configure to enable it. This would allow Unix users to test that the export list is correct, and to build a perl that does not pollute the global namespace with private symbols.


Patches of Interest

Cross-compile for linux

Vadim Konovalov continued to deliver his patches for cross-compilation. Alas, few people with commit bits have had the time to review them and enter them into Perforce. There is hope that git may alleviate these sorts of problems.

  http://xrl.us/oudya
  http://xrl.us/oudyc 

5.8.x on Interix

Ben Morrow reported on the test failures on Interix for 5.8.9-tobe. Jan Dubois took issue with the skipping of tests, since that might lead the underlying bugs to be swept under the carpet. Ben thought that there was a distinction to be made as to whether the test suite was testing Perl or Perl and its interaction with its host environment (such as bugs in the C library).

Rafaël hoped that the changes, if they were good enough for 5.8, could also make it into blead. Ben saw no reason why not, he just wanted to help get 5.8.9 out the door at the moment.

  http://xrl.us/oudye 

Invalid read in regdupe in maint-5.8

Nicholas continued to ponder what Vincent Pit had uncovered, regarding an off-by-one allocation in the regexp compiler.

  http://xrl.us/oudyg 


New and old bugs from RT

Different behaviour using the -Idir option on the command line and shebang (#6665)

Renée Bäcker wondered why using -I on the shebang line of a Perl program appends the specified directory to @INC, rather than prepending.

Rafaël Garcia-Suarez explained how and why it was a feature, and Renée proposed a patch to make -I always prepend.

  http://xrl.us/oudyi
  http://xrl.us/oudyk 

return do { }: take 3 (or 4...) (#38809)

sub foo { do { return do { 1; 2 } }; 3 } returns undef. This is unfortunate. Vincent Pit has been chipping away at the problem, trying to solve it. This is his latest offering. He's still not happy with the fix, and suggests that it's either this, or ignore the bug.


  tests will tell
  http://xrl.us/oudyn 

Perl 5.8.8 Build Failures for SunOS 4.1.3 (#59456)

Bob Peterson called for help in getting a new Perl running on a very, very long in tooth SunOS platform. Andy Dougherty thought that the last time he had attempted such a task was in 2001, and thought that some of the errors Bob was encountering were due compiler bugs, and that building without -O2 optimisation may alleviate some of the problems.

Nicholas Clark identified some failures as being Mostly Harmless, but others were rather more disconcerting.

  teaching an old dog
  http://xrl.us/oudyp

  http://xrl.us/oudyr 

Unicode::UCD::casefold() does not work as documented, nor probably as intended (#58430)

Karl Williamson continued to work on Perl's simple-minded implementation of folding case for Unicode characters, the real-world problems that this induces, and a proposal on how to move to something that works whilst retaining backwards compatibility.

Rafaël asked for some background for the readers following along at home, and Karl responded with a very well written summary of the problem domain.

  all you need to know
  http://xrl.us/oudyt 

Segfault on weird but plain Perl in eval on 5.10.0 on Windows (#59498)

Eirik Hanssen and Alexandr Ciornii pinned down some code that failed in Strawberry Perl 5.10.0 and wondered if it was common to all Windows 5.10 builds. Cygwin and Unix builds appeared to be unaffected.

  broken windows
  http://xrl.us/oudyv 

Memory leak with regex in 5.10.0 (#59516)

Robin Hill offered an interesting bug on 5.10 that proceeded to eat all memory. The trigger was a regexp with a quotemeta'ed portion and some single character character classes.

Nicholas Clark noted that while lots of memory gets chewed up, if the program is able to run to completion, every last byte is correctly freed. Dave Mitchell produced a smaller test case to demonstrate the leak... which still cleaned up after itself if things ended in an orderly fashion. Believed (by Nicholas) to be solved.

  http://xrl.us/oudyx 

Surprising behaviour when $ is left off $last (#59522)

Inside a curly brace block, even just for lexical scoping, last will take you out of it without a murmur, even if you write $x < last instead of $x < $last. Unfortunately there's not a whole lot that can be done about it.

  http://xrl.us/oudyz 

readline returns EBADF at eof in 5.10.0 (#59550)

Mark Martinec reported that the readline() function (or a diamond I/O operator <>) returns a EBADF (Bad file descriptor) as its errno status on end-of-file when PERLIO is set to "stdio".

  http://xrl.us/oudy3 

Error installing a Perl 5.8.8 (#59610)

Leandro Gregorio needs some help with an installation on a 64-bit Linux platform.

  http://xrl.us/oudy5 

FOLDCHAR regop not produced for \x, \0, \N{U+....} (#59616)

Karl Williamson is now using the latest version of blead for his work and is pushing out the bugs he encounters in Unicode case folding. His latest item is that specifying characters in hex, octal or named characters defeats the regular expression engine.

  http://xrl.us/oudy7 

Save code generated by references in @INC (#59626)

Shawn Moore wanted to be able to profile code references in @INC via Devel::NYTProf, and found a number of hurdles in his way. He offered a patch that appeared to work, but Nicholas Clark pointed out that it did in fact only appear to work, and making it work would in fact be much more difficult.

  http://xrl.us/oudy9 

Unicode problem (#58182)

Karl Williamson continued on his Unicode trek this fortnight, with a summary of the issues concerning this bug (for which the main point is about how to land it gracefully in 5.10 in a backwards-compatible manner, switchable via a lexical pragma). Another item is what to do about regular expressions which are compiled in one (Unicode-encoding) context and then used in another.

Glenn Linderman and Rafaël made a few comments, and it appears that Karl has managed to reach a concensus of sorts.

  http://xrl.us/oudzb 

unicore/mktables expects wrong syntax from CaseFolding.txt for Turkish (#58656)

Karl pinned down the bug in Turkic character handling in Unicode. In his words: "it appears that the Unicode folks decided they were doing the wrong thing in 3.1, and changed it in 3.2. Perl is still looking for the 3.1 syntax." That is, they snuck it in without telling anybody. Karl hopes to be able to produce a patch.

  http://xrl.us/oudzd 

File::Copy does not handle file objects sanely (#59650)

Wow, another bug in File::Copy.

  http://xrl.us/oudzf 

Too late for -CS option (#59652)

Either that, or too early, since no-one has the tuits to implement something that would do the right thing. In the meantime, Aristotle Pagaltzis proffered the canonical workaround.

  http://xrl.us/oudzh 

SIGSEGV compiling regexp in 5.10.0 (#59654)

Salvador Fandiño found a way to make things go boom. But what could that pattern be used to match?

  6000 parens is not enough
  http://xrl.us/oudzj 

Cwd::realpath fails on unsearchable directory (AIX) (#59662)

John Wiersba showed that a short C program worked where Perl fell over and choked.

  just need to cut'n'paste
  http://xrl.us/oudzm 

Segfault when using (?|) in regexp. (#59734)

Abigail gives the regexp engine indigestion, with a short, sharp perl -wE '";" =~ /(?<a>(?|(?<b>;)))/;' to the ribs.

  new toys broken
  http://xrl.us/oudzo 

Documentation issues with 5.8.8/Math/BigInt.pm (#59746)

Paging Tels. White courtesy phone.

  unless fixed in blead?
  http://xrl.us/oudzq 

File::Find behaviour with files among @DIRS (#59750)

J. Flack noted with pleasure that File::Find does more or less the right thing when given a file as a starting point. And in fact, if it didn't, it would be incredibly awkward to make it so from the client side.

Unfortunately, it is completely undocumented, so his/her plea was to have it documented so that it could be relied on, unless perchance it doesn't work at all on some obscure platform.

  when you need it, you need it bad
  http://xrl.us/oudzs 

Changing effective userid from root to normal user fails on Linux (#59766)

Markus Huaraz filed a bug report on Linux regarding the setting of $> preventing perl from reading its own binary. The /proc symbol table entered the picture, but Rafaël Garcia-Suarez was unable to reproduce the problem.

  note to self
  http://xrl.us/oudzu 

Not OK: perl 5.11.0 +DEVEL +patchaperlup: on i686-linux-64int 2.6.22-1-k7 (UNINSTALLED) (#59776)

Andreas König reported a smoke failure and suggested the fix (in ExtUtils::MakeMaker).

  http://xrl.us/oudzw 

Cannot recurse inside (?|) (#59792)

Whilst working upon the next release of Regexp::Common, Abigail encountered some flaws in the implementation of (?|) and offered some tests to pin down the problem.

  broke again
  http://xrl.us/oudzy 

Hash collision? (#59796)

Nope, just symbolic references that use strict would have caught.

  priceless
  http://xrl.us/oudz2 

return 0 or die (#59802)

  return doit() or die "did it wrong";

It would be nice if that warned "unreachable statement".

  that or Perl::Critic
  http://xrl.us/oudz4 

Perl5 Bug Summary

  279 new + 1054 open = 1333 (+15 -5)
  http://xrl.us/oudz6
  http://rt.perl.org/rt3/NoAuth/perl5/Overview.html 


New Core Modules

Module::Build 0.30
  http://xrl.us/oudz8
  http://xrl.us/oud2a 
ExtUtils::MakeMaker 6.46
  http://xrl.us/oud2c 
IPC::Cmd 0.42
  http://xrl.us/oud2e 
File::Fetch 0.16
  http://xrl.us/oud2g 
Archive::Extract 0.28
  http://xrl.us/oud2i 

In Brief

Renée Bäcker pointed out that there are many patches in RT bug reports, just begging to be applied.

  http://xrl.us/oud2k
  http://xrl.us/oud2n 

Michael G. Schwern suggested an improved GM/LOCALTIME_MAX/MIN search algorithm. H. Merijn Brand suggested a tweak or two.

  http://xrl.us/oud2p 

Jim Cromie was stung by list context in assignment.

  bad luck
  http://xrl.us/oud2r 

Michael G. Schwern discovered that HP's Testdrive being replaced. As a long-time resource for testing VMS issues regarding Perl, it will be greatly missed. The HP engineer that H.Merijn contacted suggested that something could be worked out.

  thanks for all the fish
  http://xrl.us/oud2t 

Nicholas Clark wasn't sure if CPAN bug #39403 involved something that needed to be fixed in the core (the ability to overload or intercept an attribute, other than using a source filter).

  http://xrl.us/oud2v 

Nicholas also proposed a wild-assed guess to allow parallel testing on Win32 but Steve Hay reported no joy. Andy Armstrong understood the resulting symptoms sufficiently to volunteer to see if he could figure out why.

  http://xrl.us/oud2x 

Salvador Fandiño noted that Paul Fenwick was not the first person to whether the prototype for CORE::truncate is wrong.

  http://xrl.us/oud2z 

Nicholas removed the missing type VSTRING in the documentation to ref since it wasn't supposed to be there at all, being an overly optimistic backport from blead.

  bug #59264 solved
  http://xrl.us/oud23 

John O'Rourke ran into some odd behaviour of substr(lc($utf8)) in 5.8.8 that Nicholas had fixed, he thought, thanks to a TPF grant. Since that work had made it to 5.10 and had been backported to the upcoming 5.8.9, he was happy to leave things there.

  http://xrl.us/oud25 

Nicholas noticed some make distclean failures, in that there were some files left over from a run of the test suite that weren't cleanup up after the target was run. Andy Armstrong began to look at cleaning things up, since he had a nagging suspicion that a number of them might be his fault.

  http://xrl.us/oud27 

H.Merijn Brand released Data::Peek to CPAN.


  a collection of low-level debug facilities
  http://xrl.us/oud29 

Nicholas saw a Debugger saved lines regression and wondered out loud what was the new officially sanctioned way of noting new subroutines that are defined during an eval. When the right technique is found, a test he wrote should begin to pass.

  http://xrl.us/oud3b 

If Dave Mitchell is ignoring you, resend your message.

  the tale of davem's lost emails
  http://xrl.us/oud3d 

Last week's summary

  This Week on perl5-porters - 15-21 September 2008
  http://xrl.us/oud3f 

About this summary

This summary was written by David Landgren.

Weekly summaries are published on http://use.perl.org/ and posted on a mailing list, (subscription: perl5-summary-subscribe@perl.org ). The archive is at http://dev.perl.org/perl5/list-summaries/ . Corrections and comments are welcome.

If you found this summary useful, please consider contributing to the Perl Foundation or attending a YAPC to help support the development of Perl.