This Week on perl5-porters - 22-27 September 2008

grinder on 2008-10-05T21:10:00

"I'm really tempted to fix toke.c:S_incline() instead. I don't think there will be much breakage, right?" -- Rafaël Garcia-Suarez, hiking in mountain boots where angels fear to tread.

Topics of Interest

perl@34300

Matt Kraal reported what tests on 5.8.9-tobe were failing on QNX, and when prodded by Nicholas Clark, produced the delta regarding the tests that were already failing on 5.8.8.

  no showstoppers
  http://xrl.us/osxd6 

$VERSION in Opcode

Matt S. Trout revived the thread about Opcode and Safe by talking about version.pm, saying it was a really nice module, just a shame about it messing with UNIVERSAL. When John Peacock showed him the scars of six years working to make version comparisons work, and asked for a test case to show where it didn't work, Matt skirted the issue by saying that he really liked version.pm's version comparison code, and would John accept a patch that switched off the UNIVERSAL hackery.

  like a version
  http://xrl.us/osxd8 

oplines - make perl runtime 8% faster and smaller

Reini Urban mentioned that he was carrying out research to restructure the way opcodes carry around the original source line number (for eventual error reporting). First results seemed to indicate that reasonable code could expect to see an 8% speed increase.

Dave Mitchell saw that the main thrust of the work was to do away with nextstate ops, and warned that one of their other purposes, apart from carrying around line numbers, was to give the interpreter an opportunity to clear out the temps from the stack.

Ben Morrow wondered whether the place where the clearing out took place could be moved from pp_nextstate into Perl_runops_standard et al instead. Reini thought it wouldn't be necessary, since he was identifying nextstate ops that do trigger temps housekeeping, and he wasn't touching those.

  all lined up
  http://xrl.us/osxea 

Non-deterministic behaviour with simple array initialization

Thomas Karcher was having problems with perl installations on two different Xen VM machines, which, while nearly identical, were producing two very different results. Despite a couple of good suggestions from Nicholas Clark, no good answers turned up.

  the art of vm maintenance
  http://xrl.us/osxec 

DDump

H.Merijn scratched the same itch twice in the space of a week regarding the ability to dump a scalar in excruciating detail (using a secret handshake involving sv_dump) à la Devel::Peek within Data::Dumper, and so wondered whether he should keep it to himself, or push the changes outwards either as a CPAN module or somewhere within core.

Nicholas Clark thought that that the concept was sound although he took issue with pumping the results out to a file handle, and thought it would be better to send it to a scalar, as this gives the client more flexibility in deciding what to do with it. This would require changes to all the *dump routines. Graham Barr thought that it would be easier to define new API entry points that performed the existing dumps to a scalars.

In the following discussion, H.Merijn applied a number of changes that fixed various problems that, alas, my time budget does not allow me study in greater depth, but he concluded that the changes where worthy of backporting to the 5.10 branch, and that more tests were needed for the *dump family of functions.

  a better dump
  http://xrl.us/osxee 

This work wound up producing failure in XS::APItest, that Steve Hay eventually fixed.

  http://xrl.us/osxeg 

Release and patch sizes in perlhist

Abigail observed that perlhist had acquired over six years of bitrot regarding the statistics of perl test releases, and wondered whether it was better to bring it up to date, or delete the information altogether. Moritz Lenz voted for bringing them up to date, citing a love of statistics.

Tom Christiansen had also noted the oversight in the recent past, but wasn't sure whether this was a result of official policy or just something that had slid under the radar.

Yitzchak Scott-Thoennes also weighed in saying that it would be Good and hoped that Abigail would find the tuits to do so.

  fun with numbers
  http://xrl.us/osxei 

perlperf.pod submission

Richard Foley sent in a draft for a document on Perl Performance and Optimization Techniques (or PPOT). H.Merijn suggested merging in the points covered by Nicholas Clark's "When Perl isn't fast enough" talk.

Reini Urban took the time to fill out the SEE ALSO section with modules on CPAN that deal with performance issues. Tim Bunce suggested another place to dig up some nuggets.

The discussion naturally zeroed in on Devel::DProf, that Time Bunce declared was broken (and no-one contradicted him) and said that Devel::NYTProf was in all ways superior.

Nicholas Clark thought so too, but wondered if NYTProf would be ready in a year or so for freezing within the official distribution. He also thought that the NYT (New York Times) aspect was of great marketing value to show who used and contributed to Perl, in contrast to Richard Foley's idea of renaming it to NGProf for "New Generation".

Rafaël Garcia-Suarez greeted the inclusion favourably, but noted that it is (still) rather Windows-unfriendly and contains some rather unportable code. Interestingly, this was the first time I can recall that the core developers stated that a perfectly valid reason for removing a module from the core is that it's always available on CPAN for those who depend on it.

Richard then released a second version, based on the feedback received so far. It will be added to the core. Eventually.

  Do Not Engage in Useless Activity
  http://xrl.us/osxek 

Interpolated code blocks may not create lexicals

David Landgren (err, I) was stung by not knowing Perl sufficiently well and wanting to write

  my @want = @{[ my $rand = rand(); $rand < 0.5 ? 'pie' : 'beer' ]}

not realising that what goes between [ and ] is an expression and not a series of statements. The problem is that Perl's parser can do little more than a CP/M BASIC-era "Syntax error" message, which conveys insufficient information to determine the nature of one's sins.

Nicholas Clark devoted the time to figuring out that what needs to be done is to teach the tokeniser to react to encountering a ; token within an expression and acting accordingly.


  http://dilbert.com/strips/comic/2001-10-25
  http://xrl.us/osxen 

Hints file for MirOS BSD

Chris "BinGOs" Williams was playing around with Mir OS (an OpenBSD fork) after having read about it in Michael G. Schwern's use.perl blog (search engines take note: Michael G. Schwern has a Perl blog on use.perl.org). He discovered that the developers there had created a hints file. He pushed it up to the porters, and Nicholas folded into the codebase, berating all the while the package maintainers of the world who choose to live in autarky.

  a bell is a cup until it is struck
  http://xrl.us/osxep 


TODO of the week

Here's a task chosen entirely at random that requires a certain amount of C knowledge, and would be a delightful task for someone wishing to learn a bit more about how the parser works.

Error reporting of [$b; $c]

Using ; inside brackets is a syntax error, and we don't propose to change that by giving it any meaning. However, it's not reported very helpfully:

    $ perl -e '$a = [$b; $c];'
    syntax error at -e line 1, near "$b;"
    syntax error at -e line 1, near "$c]"
    Execution of -e aborted due to compilation errors.

It should be possible to hook into the tokeniser or the lexer, so that when a ; is parsed where it is not legal as a statement terminator (ie inside {} used as a hashref, [] or ()) it issues an error something like ';' isn't legal inside an expression - if you need multiple statements use a do {...} block. See the thread starting at http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008-09/msg00573.html

(or the thread summarised elsewhere in this issue. Consider it a fringe benefit that goes with the job).


Patches of Interest

2038 fix

Michael G. Schwern sent the heads-up on his ongoing work to make Perl deal correctly with dates beyond 2038 (which would otherwise overflow the underlying 32-bit storage used). He was delighted to receive some Windows feedback from Steve Hay, and asked for the changes to be folded back into blead in order for the code to receive wider cross-platform exposure.

  crossing the date line
  http://xrl.us/osxer 

Fix DosGlob to work with paths containing ()'s

Alex Davies taught DosGlob to deal with filenames that contain parentheses. This is important on Windows where one often encounters files names of the Copy (2) of foo nature. Shlomi Fish suggested that a test-case would be a nice addition to the patch, as it would have caught a problem in Alex's own patch.

  touché
  http://xrl.us/osxet 

Invalid read in regdupe in maint-5.8

Vincent caught another puzzling bug in 5.8 with threads that appeared to be simple off-by-one sizing error. Nicholas Clark thought that the code was identical as far as he tell to that of blead, and yet there was no problem there. There were other puzzling symptoms that made Nicholas suspect there was something deeper afoot, but couldn't see how to coax the problem out into the open.

  by the pricking of my thumbs
  http://xrl.us/osxev 

Lies, damn lies and end-of-block comments

Whilst working on his Y2038+ code, Michael G. Schwern discovered that reentr.h was riddled with #ifdef foo/#endif # bar that didn't line up, so he straightened them out.

  comment rot
  http://xrl.us/osxex 


New and old bugs from RT

Recursive multithreading causes massive amounts of context switches (#55560)

Thomas Karcher discovered the hard way that Perl doesn't have a fine-grained locking method for shared arrays with threads, but wasn't sure if this was a bug or a feature.


  arrayed against him
  http://xrl.us/osxez 

Volunteer for fixing the "Unicode" bug (#58182)

Glenn Linderman kindly provided the leads to track down previous p5p threads on the issue of Unicode implementation issues in Perl 5. He also identified the list of contentious operators and operations performed with Unicode data.

One important point Glenn raised was that Windows filenames and Unicode do not get along well, since Perl doesn't use the UTF-16 system APIs. Changing this will cause pain, but it currently prevents people from writing multi-lingual applications on Windows in Perl.

  http://xrl.us/osxe3 

5.8.x: regexp bug with (?!) (#59168)

Father Chrysostomos tracked down another regular expression bug in the upcoming 5.8.9 release. Nicholas Clark traced this back to change #30638, that Yves Orton had written for blead to fix bug #39583.

  http://xrl.us/osxe5 

Bad regex for #line directive in perlsyn. (#59170)

Elliot Shank found a typo in the recipe given for determining filenames and line numbers when decoding #line directives. He suggested changing the regular expression to make it correspond with what really happens under the hood. Rafaël Garcia-Suarez suggested that it might be better to fix the tokeniser in order to make it match the documentation, as what it purports to document is more useful than how things behave currently.

  fight the cause
  http://xrl.us/osxe7 

MIPS64 Configuration fails (#59186)

Phani Kancharia reported a distinct lack of success in configuring perl on a big endian MIPS64 platform. The problem was that there is a paucity of information in the hints/mips.sh file; so much so that the configuration probes are unable to determine a viable strategy for dealing with 64-bit integers. H.Merijn Brand hinted that it might take a significant amount of work to get things into a usable state.

  http://xrl.us/osxe9 

ext/B/t/concise fails test 155 without dynamic loading (#59206)

Andy Dougherty uncovered a problem revealed by a change made in September 2007. Nicholas regretted that a similar test for 5.10.0 would have picked up the problem much earlier, but corrected it anyway.

  wholly undynamic loadings batman
  http://xrl.us/osxfb 

DynaLoader.t test assumes dynamic loading (#59210)

Andy Dougherty discovered that the test suite Dynaloader itself fails if perl is configured without dynamic loading, and suggested that it is largely due to trivial differences in error messages. Nicholas Clark fixed things up with change #34404.

  http://xrl.us/osxfd 

Documentation to ref function, missing type VSTRING (#59264)

Hans Ginzel reported an inconsistency with ref, Renée Bäcker offered a documentation patch to clarify the matter.

  http://xrl.us/osxff 

PerlIO::via leaks memory at a high rate (#59268)

Marc Lehman produced one leak per write using the via PerlIO layer. His hypothesis is that newGVgen leaks.

  http://xrl.us/osxfh 

PUSH on tied array gives incorrect context to method and can result in needless calls to FETCHSIZE (#59280)

Luke Ross uncovered an inefficiency with push on tied arrays, and proposed a patch to attempt to address the problem. Eric Brine pointed out that subroutines (and hence, modules) that return values by accident when called in void context would break with Luke's change, not the least of which is Tie::Array.

Luke returned with a variant that seemed reasonable to Rafaël, who thought that, by extension, the same reasoning could be applied to unshift. So Luke returned with a third variant that took unshift into account as well. And things stopped there for the moment.

  when push comes to unshift
  http://xrl.us/osxfj 

In regular exprs, /\N{U+...}/ fails to compile properly if ... is > 256 (#59328)

In his quest to fix Unicode, Karl Williamson noted in passing that $foo =~ /\N{U+257}/ does bad things to S_reg_namedseq(). (Summariser's note: in trying to reproduce this, I was unable to coax any sort of error message. Methinks more context is required).

  http://xrl.us/osxfm 

Perl 5.10 make problems - math.h references (#59332)

Nathan Reed asked for help after failing to configure perl 5.10 on Solaris. H.Merijn Brand and Andy Dougherty diagnosed a problem of user defined configuration removing /usr/lib from the list of paths used to resolve C library references.

  http://xrl.us/osxfo 

chr(0400) =~ /\400/ fails for >= 400 (#59342)

Karl William noted another bug in passing as he proceeded in his work. Abigail and David Nicol weighed in explanations and suggestions, but Karl replied that it wasn't of great importance since he expected his ongoing work to resolve it anyway.


  http://xrl.us/osxfq 

regexp flag m ignored if applied on top of qr, bug or feature of 5.10? (#59368)

Marc Martinec stumbled over a change in 5.10 involving qr//'ed regular expressions embedded in other expressions when /m was added to the mix. Rafaël explained that this fixed a greater problem.

  it's a feature
  http://xrl.us/osxfs 

Get "Malformed UTF-8 character" message (#59378)

Karl also noted that "\xe0" =~ /[\x{100}]/i is parsed incorrectly.

  http://xrl.us/osxfu 

float modification with regex non-substitution (#59386)

Mark Pack discovered a very obscure corner case on a non-matching substitution nevertheless causing a change upon a scalar holding a floating point value.

Nicholas Clark correctly identified the problem as one of unintended stringification, and amended Mark's regular expression to not match unintentionally, thereby avoiding the stringification, and all was well. He went onto explain that Mark wasn't exactly wrong in his analysis, but to do as he suggested would incur a generalised cost to every single match, whether it applied to Mark's corner case or not.

  a matter of comprise
  http://xrl.us/osxfw 

Perl5 Bug Summary

  267 new + 1038 open = 1305 (+9 -4)
  http://xrl.us/osxfy
  http://rt.perl.org/rt3/NoAuth/perl5/Overview.html 


New Core Modules

Module::Build 0.30

Ken Williams announced the release of 0.30. There was a certain amount of discussion as to when it would support autobundling.

Steve Hay had a bit of trouble applying this to blead as there was a bit of fuzz in terms of things in blead that weren't in 0.30 as it stood.

This then segued into a discussion of Archive::Tar having some problems outstanding on VMS. To this, Jos Boumans replied that an all new 1.40 version of that module should be released soon.


  http://xrl.us/osxf2 
ExtUtils::MakeMaker 6.46

Michael G. Schwern announced that he had dragged ExtUtils::MakeMaker into the third millenium of additional metadata to allow modules to boldly qualify themselves where no modules had gone before (unless they used Module::Build). Specifically, it now adheres to the 1.4 version of the META.yml specification.

Here too were revealed problems of changes in blead not being in the module's own development repository.

  the module that would not die
  http://xrl.us/osxf4 
XSLoader 0.10

Sébastien "Maddingue" Aperghis-Tramoni delivered the latest version of XSLoader, that all the cool kids use a replacement for Dynaloader, to deal gracefully with the uncool kids who don't do dynamic loading.

  http://xrl.us/osxf6 

In Brief

Rafaël Garcia Suarez added Shlomi Fish's addition of open "|-" and open "-|" to perlopentut .

  http://xrl.us/osxf8 

Gábor Szabó's quest for a name of a debugger client reminded Matt S. Trout about Devel::ebug, which he thought already implemented a certain amount of what Gábor was trying to do.

  http://xrl.us/osxga 

Landry Breuil reported two regression tests failing on OpenBSD/sparc64. Rafaël Garcia-Suarez thought that 32/64 bit inconsistencies might be the reason.


  http://xrl.us/osxgc 

Karl Williamson had asked What do I do when I stumble upon an existing bug while I'm working on other changes? and the answer should be obvious upon reading this week's "New and old bugs" section.

  http://xrl.us/osxge 

Jerry D. Hedden reminded Nicholas Clark that he needed to propagate change 34377.

  http://xrl.us/osxgg 

He also simplified the loading modules in test.pl .

  http://xrl.us/osxgi 

Last week's summary

  15-21 September 2008
  http://xrl.us/osxgk 

About this summary

This summary was written by David Landgren.

Weekly summaries are published on http://use.perl.org/ and posted on a mailing list, (subscription: perl5-summary-subscribe@perl.org ). The archive is at http://dev.perl.org/perl5/list-summaries/ . Corrections and comments are welcome.

If you found this summary useful, please consider contributing to the Perl Foundation or attending a YAPC to help support the development of Perl.


What happened to the previous summaries?

grinder on 2008-10-06T07:38:15

I had difficulties over the past few weeks loading the summaries but things appear to be fixed now, which is why the last four are missing. Would people be interested in reading them if I post them, even if they are out of order?

Yes please.

grantm on 2008-10-06T20:42:17

I definitely would be interested. Maybe you could just add a short note about the ordering to avoid confusion.