Lots of work consting, compiling and testing perl this week.
Nicholas Clark commented on Jerry D. Hedden's cry for help concerning
the slow application of his threads patches, and wondered what
do other open source languages, such as Lua, Python, Ruby and Tcl
cope with the day-to-day drudgery of sifting through bug reports,
shepherding the changes to the code base, and generally making sure
that the project keeps moving forward.
Holding the fort http://xrl.us/k7mm
Nicholas Clark saw that some new taint tests were causing perl to hang on recent versions FreeBSD, and wondered what to do about it. Steve Peters reproduced the problem on OpenBSD as well.
On the other hand, Dominic Dunlop gave Mac OS/X Darwin a clean bill of health (despite the fact that it is derived from FreeBSD).
Anton Berezin (the man for Perl on FreeBSD) suggested linking with a different threads library. Everything worked when Nicholas tried that, leading him to conclude that hints/freebsd.sh could do with an update.
I'll give you a hint http://xrl.us/k7mn
Peter Scott wanted some "best practices" advice on how to build a threaded
and non-threaded perl, and have them coexist peacefully, falling back on
the unthreaded perl for all the ancillary tools such as perldoc and
perlbug. Specifically, and hence the title of his message, he wanted
to know if he could get away with using the same compilation of XS
modules for both perl instances.
Jan Dubois clarified the situation, explaining that threaded and unthreaded perls are binary incompatible, so modules with XS components need to be recompiled, and need to be installed in separate locations (so that the associated perl picks up the right libraries for itself).
Andy Dougherty welcomed Peter's suggestion to patch the documentation with his findings.
We build them here, we build them there, we build these modules everywhere http://xrl.us/k7mo
/ and X in unpack Peter Dintelmann had a question concerning the ability to walk back
the stream with unpack. He was trying to understand how to do
it but was struggling with the paucity of information. Sadahiro
Tomoyuki explained exactly how it worked, even going so far as to
examine the C source and realise that the error checking could do
with a little tightening up.
http://xrl.us/k7mp
Tomoyuki's explanation was just what Peter needed.
http://xrl.us/k7mq
test.pl David Landgren started to work on refitting t/op/pat.t to use
test.pl, which would make it simpler and easier to add new tests.
He had a couple of questions, mainly concerning how to modernise
some fairly archaic tests.
Andy Lester was thrilled. Nicholas Clark recalled that one useful feature that test.pl does not have but should, is the management of temporary files. Being able to create them on demand, and clean them up automatically at the end of the script. Apparently this sort of make-work code is scattered throughout the test suite.
Into the third millenium we go http://xrl.us/k7mr
David also had a look at a section in t/op/pat.t for patterns that warn when compiled and wondered whether a particularly intricate collection of closures was in fact a no-op. No takers.
The sound of chain-saw warming up http://xrl.us/k7ms
The new version was finally delivered. It generated a lot of discussion, but alas, remained unapplied.
http://xrl.us/k7mt
eval in t/op/append.t David then looked at t/op/append.t and saw that some of the test
results were eval'ed (rather than the thing being tested), and
also wondered why the tests were tested with regexps instead of
comparison operators. Sadahiro Tomoyuki explained that the tests
were there to see if concatenation stops on \0 (binary zero),
and regexps were used instead of comparison operators, in case the
comparisons also had an equivalent bug which stopped them scanning
on \0.
Which led David to wonder if there wasn't a just as likely possibility
for regexps to also stop on \0, in which case something like
rindex might be better, since it would never encounter any dubious
characters.
Tomoyuki thought that the eval was no longer necessary (although
it does mean that things happen at run-time rather than at compile-time,
which may have an influence).
http://xrl.us/k7mu
Elsewhere, David looked at the tests in t/op/grep.t and was
surprised to discover that some tests can pass, and print ok, or
they can fail, and print... nothing.
As long as everything works http://xrl.us/k7mv
David Bree asked for advice on compiling perl on Windows 98 using MinGW. He had in fact compiled it, but was wondering whether it was safe to install, since about five percent of the tests failed. Which doesn't seem to shabby. Unfortunately, no-one was able to offer any advice.
Maybe if we knew what had failed http://xrl.us/k7mw
wince and win32 directories Yves Orton started to tackle the long-talked-about much-awaited merge of
the wince and win32 directories, that contain the platform-specific
extensions for the Windows CE platform on one hand, and Windows in general
on the other.
Vadim Vladimirovich Konovalov (hereafter referred to as Vadim) promised
to add some improvements to the WinCE side, and was pleased to see that
someone was taking the necessary time to clean up the source tree. He
also mentioned the existence of a perlce.sf.net project, but that it
is pretty calm at the moment.
Yves's approach was to create a new win directory, starting with
the contents of win32, and then see what happened as he pushed
and poked bits of wince into it, until there is nothing left
over.
Where do you want Perl to go today? http://xrl.us/k7mx
z/OS Mohammad Yaseen reported the difficulties he was having with 5.8.7 on z/OS (the IBM mainframe OS). In this instance, t/op/stat.t was failing with messages about /dev/tty, depending on whether the test was run in single-user or multi-user mode.
Jim Cromie asked for some specific details, to help understand what single-user and multi-user means in the context of z/OS.
http://xrl.us/k7my
As Nicholas happily integrated a change from blead to maint
concerning bitmask flags on a field in the magic vtable structure,
he realised that there weren't a whole lot of spare bits that
remained, and even though code with the first definitions of those
flags has been out in the wild for some time, he wondered if some
of the newer bitmask flags could be reeled back in.
Dave Mitchell thought that three flags could be freed up, since they were either marked as being wildly experimental, or not documented at all.
Peak bits http://xrl.us/k7mz
taint and fork on Win32 David Golden had been taking a VanillaPerl on Win32 for a spin, and
noticed that Test::WWW::Mechanize kept dying a horrible death.
Carl Franks and he narrowed it down to a delightfully simple snippet:
perl -Te fork and then searched the bug database but found nothing
similar. So he wanted to know whether to file a bug report, or
whether the failure mode was common knowledge. No takers.
Can't have your fork and taint it http://xrl.us/k7m2
FileHandle IO::Seekable? Adam Kennedy related the problems he was having after having taken over
the maintenance of Archive::Zip. In a nutshell, he needs to have
seekable file handles. And yet it turns out that at one point in the
test suite, a file handle winds up being connected to a pipe, which
definitely is not seekable. So Adam wanted to know if something was
truly seekable, as opposed to merely having a seek method.
The best Randy W. Sims could come up with was a kluge with file test operators.
Seeking advice http://xrl.us/k7m3
Yves Orton repatched his trie work to simplify the sharing of the
code base between blead and maint.
Nice trie http://xrl.us/k7m4
5.8.0 could be supported as well, if there was some way to fake SAVEBOOL
via ppport.h.
http://xrl.us/k7m5
Reini Urban showed what needed to be done to get it to compile with the ActivePerl source.
http://xrl.us/k7m6
http://xrl.us/k7m7
consting goodness Andy Lester took a crack at adding some consting goodness to
doop.c , but Sadahiro Tomoyuki pointed out a mistake. So Andy
took had another go at it. Not applied, as far as I can see.
http://xrl.us/k7m8
He then killed a modal boolean that controlled the behaviour of
S_glob_2inpuv in sv.c, preferring to split the functionality
into two different routines. Applied.
http://xrl.us/k7m9
Andy then carried on, inlining a couple of static routines in perl.c. Also applied.
http://xrl.us/k7na
Andy found reasons to use NOOP macro, part one.
http://xrl.us/k7nb
Moving right along, he removed some unused (or rather, useless) casts in regcomp.c and regexec.c, which prompted Yves Orton to comment that when he was working on adding trie support to the regexp engine, he found those casts to be quite helpful in figuring out what was going on.
http://xrl.us/k7nc
He had more luck removing unused context in sv.c.
http://xrl.us/k7nd
And another context parameter in pp_ctl.c, with some consting
goodness thrown in for good measure.
http://xrl.us/k7ne
And some more places in which he could tweak the usage of SvREFCNT_inc
calls.
In mg.c http://xrl.us/k7nf
and pp_ctl.c http://xrl.us/k7ng
And some compiler warnings in perlio.c.
And that's what Andy did this week http://xrl.us/k7nh
v?snprintf/strlcpy/strlcat when useful Jarkko Hietaniemi patched a number of files to use the C run-time equivalent string routines (that have been vetted as working correctly at build time) rather than alternative, less safe routines. Jarkko asked for eyeballs to look carefully at the patch in case there were any glaring errors.
Steve Peters took the patch for a spin on the platforms he had lying around. In the meantime, Jarkko found a section in pp_ctl.c that he had missed.
H.Merijn Brand wondered what the status was concerning these routines, since
on some proprietary platforms, the C source to these routines had some fairly
fussy copyright noticed attached. Steve said he had dug up the source to the
Perl routines from an old copy of inn, and believed that they were in the
public domain.
Russ Allbery said that he authored inn, and that Steve was perfectly
entitled to having taken them for use in perl. This prompted H.Merijn to
suggest removing the #ifdef trickery surrounding the calls to these
routines, and instead use them everywhere, and bundle Russ's public domain
code to be used on platforms that don't provide them.
New uses for old http://xrl.us/k7ni
dVAR and -DPERL_GLOBAL_STRUCT_PRIVATE Jarkko continued on his dVAR quest with -DPERL_GLOBAL_STRUCT_PRIVATE
to move Perl C run-time data to the heap. As he marked a previously const
variable as non-constant, Andy Lester insisted on a full report. Jarkko
explained what had happened, and Andy thought of another approach. Jarkko
urged Andy to proceed cautiously, as it concerns only the Symbian port as
is also rather fragile. Any changes must be tested with
PERL_GLOBAL_STRUCT_PRIVATE and PERL_GLOBAL_STRUCT
Do not meddle in the affairs of Jarkko http://xrl.us/k7nj
And then sent in another dVAR patch, this time to make perl's
malloc happy, plus some corrections to fix up some signed/unsigned
mismatches that had crept in when no-one was looking.
http://xrl.us/k7nk
gcc -ansi -pedantic noise reduction The patches kept rolling in from way up north, this time cleaning
up the worst of the damage when one lets gcc -ansi -pendantic
loose on the source (it screams a lot). He recommended that porters
build perl from time to time using these two compiler options, and
taking the appropriate action based on the fallout. Steve Peters
applied the patch. Andy Lester was pleased.
I dare you http://xrl.us/k7nm
And wrapping up, Jarkko offered a handy tip for people who enjoyed
torture-testing the compilation of the codebase, to use the -O
gcc compiler switch in conjunction with -Wall (warn about
everything). It turns out that some code checks don't kick in until
the optimizer is brought to bear on the code. Andy Lester wanted
to know if different -O levels kicked in successive classes of
warnings.
Cheap trick http://xrl.us/k7nn
Andy posted the armada of compilation switches that he uses to compile the source.
-Wextra is good for a laugh, too http://xrl.us/k7no
David Landgren sent in a some patches to for files in t/op to use test.pl, instead of their home-grown each-one-different methods. One main benefit is that now the tests have names, so when test 43 fails, it becomes trivial to locate it in the source file.
In the process, he also found an nice bug of a string comparison using numeric equality in t/op/loopctl.t.
t/op/auto.pl http://xrl.us/k7np
t/op/defins.pl http://xrl.us/k7nq
t/op/delete.t http://xrl.us/k7nr
t/op/glob.t http://xrl.us/k7ns
t/op/hashwarn.t http://xrl.us/k7nt
t/op/loopctl.t http://xrl.us/k7nu
t/op/sort.pl (or what I did over my lunch hours this week) http://xrl.us/k7nv
(Summariser's note: At first I wasn't sure how I was going to summarise this, but Jim Cromie sort of forced my hand).
It's all Jim's fault http://xrl.us/k7nw
Fatal.pm documentation Tom Regner had a discussion with Ilya Zakharevich on comp.lang.perl.modules
about Fatal, and the result of the discussion was a POD patch. Unapplied.
http://xrl.us/k7nx
FAIL(M) MSWin32 Win2000 SP4 (x86/1 cpu) Abe Timmerman identified this failure as the result of the Windows part of the code base not finding change #27992 to its taste.
http://xrl.us/k7ny
FAIL(Xm) irix 6.2 (IP22/1 cpu) In this report, it appears that something had gone wrong with the gcc
C run-time library during the configuration phase.
http://xrl.us/k7nz
Animator discovered that the bizarre code leads to bizarre ARRAY assignment (bug #9374) had been fixed in 5.8.8 and 5.9.3. Andreas Koenig identified change #25828 as being the one responsible for the fix.
http://xrl.us/k7n2
In a similar bug report (#3112), Animator simplified the snippet that
causes a bizarre copy of ARRAY in last . No-one attempted a fix
for the time being,
for (1) { push @a, last; }
http://xrl.us/k7n3
and bug #9540 still produced Bizarre copy of HASH errors,
@{my %self; %self}{1} = 1;
http://xrl.us/k7n4
while #22238 continued to give you a Bizarre copy of HASH in leave
@{ my %self; %self }{1} = 1;
http://xrl.us/k7n5
and for the last in a series of bizarre bug reports, Animator noted that bug #36229 (Bizarre copy of IO) is a duplicate of an earlier bug (#3314).
$a = ${*STDOUT{IO}}
http://xrl.us/k7n6
So don't do that.
unpack fails on utf-8 strings (#33734) Nicholas Clark returned to the problem raised by Marc A. Lehmann back in
early 2005 and explained what he had done with Ton's code, which fixed up
the problem in a perhaps non-backwards-compatible manner. Glenn Linderman
wondered whether the new pack/unpack semantics could not be bundled
into a loadable module and made available via a feature pragma.
Marc Lehmann thought that unpack should simply be fixed (the problem
is how unpack behaves in the face of UTF-8 data) once and for all. If
the fix is too major, then it will appear in 5.10. If it can be arranged
to not break anything already relying on the current behaviour, then it
can be fixed in 5.8.
http://xrl.us/k7n7
File::Basename) (#38891) Dominic Dunlop nudged this bug report, requesting more information. No response received as we went to the press.
http://xrl.us/k7n8
Ulrich Windl coaxed a *** glibc detected *** double free or corruption out
of the debugger. Neither Dave Mitchell nor Nicholas Clark were unable to
reproduce the problem on the perls and platforms they had on hand.
Michael Shroeder remembered that Ulrich had already reported a
similar problem with Term-ReadLine-Gnu, and wondered if the two
problems were related, and Ulrich concurred that it appeared to be
a problem concerning the C libreadline library in his SuSE
distribution.
SEP http://xrl.us/k7n9
Albert Cahalan was debugging some unfamiliar code, and wanted to
set a breakpoint in some as-yet-unseen code, and wondered why the
debugger was refusing to do so, given that gdb will prompt for
more information when in similar circumstances when debugging C
code.
Rafael wondered if b postpone subname would do the trick.
Albert agreed that that was just the ticket, but thought that the
debugger could be tweaked to ask whether one wished to postpone the
breakpoint if the routine was currently unknown to the debugger.
People missed the point, and started making fun of Albert, suggesting
that if he didn't know what he wanted to debug, he should connect
the debugger to Symbol::Approx::Sub in order to breakpoint on a
routine that most closely matched.
Albert replied calmly with an example involving gdb that made
perfect sense. The main obstacle to adding this functionality is
that everyone suddenly appears to be very busy Doing Other Stuff
when it comes to patching the debugger.
Wouldn't it be nice http://xrl.us/k7oa
Mark Suter reported a failure in t/op/getppid.t, that is due to a changed assumption concerning the owning pid of an orphaned process (hint: it is not 1). It turns out that Sun had been kind enough to loan the porters a Solaris machine with zones set up, and the bug had been found and fixed four weeks ago.
Coming to a 5.8.9 near you http://xrl.us/k7ob
Salvador Fandiño filed a bug report with a nice short snippet showing
how to provoke an Attempt to free unreferenced scalar error. After
a bit of thought, Dave Mitchell showed how it could be simplified
even further. And then after a bit more thought, patched scope.t
to fix the problem, and rolled the snippet into a test for t/op/local.t.
All in a day's work http://xrl.us/k7oc
christian@pflanze tried to create an infinite stream with
closures but the only thing he managed to create were core dumps. Dave
Mitchell explained that it wasn't actually closure-related, but yet
another manifestation of the recursive implementation used to free
recursive data structures.
Make a deep structure, eat up all your C stack http://xrl.us/k7od
Christian returned with a follow-up, showing how he had managed to get it to almost but not quite work (but at least it no longer crashed).
http://xrl.us/k7oe
The thread returned under the title of Garbage collection issues later on, where Christian wondered about a couple of issues he was having with his code. Dave Mitchell explained precisely what was happening, and then asked Christian to explain what he was really trying to do. Once Dave understood what Christian wanted, he realised the proposed code was far too complex, and sketched out an alternative approach, with negligible memory requirements to boot.
Yves thought that what Christian was trying to do was shoehorn Lisp idioms into Perl, and suggested he take a look at Mark-Jason Dominus's Higher Order Perl.
http://xrl.us/k7of
HOP to it http://hop.plover.com/
Pod::Html generates incorrect href's in some html anchors (#39020) "bgstewart" filed a problem with some incorrectly generated HTML from POD.
Steve Peters explained that it had already been fixed. Tels thought there
was a bug in the source POD, and backed up his assertion with a report
from podcheck. Russ thought the POD was valid, just somewhat unusual.
Merely plain, not simple http://xrl.us/k7og
Tie::Memoize::EXISTS not caching the value (#39026) This was reported as a bug, but no-one commented on it.
http://xrl.us/k7oh
Another decline in the number of tickets noted this week.
13 created + 27 closed (yay!) = 1525 http://xrl.us/k7oi
What outstanding bugs http://rt.perl.org/rt3/NoAuth/perl5/Overview.html
The dor (//) patch was refreshed by H.Merijn Brand as dor-588-27985.diff,
following some tweaks to perly.y.
http://xrl.us/k7oj
Module::Build 0.28 released by Ken Williams.
http://xrl.us/k7ok
This followed a patch to blead from Steve Hay to fix up some problems on
Win32 following the upgrade of Test-Harness.
http://xrl.us/k7om
The select((select(OUTPUT_HANDLE), $| = 1)[0]) thread evolved into
a discussion about sequence points in Perl.
http://xrl.us/k2zs
Rafael looked at Marcus Holland-Moritz's patch to clean up 212
warnings emitted by gcc-4.2 and applied 4 out of 5 parts, the
last part seeming to be superfluous. Marcus commented in more
detail on the reasons behind the patches.
http://xrl.us/k2yk
Craig Berry made a belated followup to Ken Williams's File::Spec
changes for VMS, giving it a clean bill of health.
http://xrl.us/k2zr
Nicholas dropped some of recent tweaks to blead's version of
Test::Harness in order to keep it perfectly synchronised with
the latest CPAN version.
Stamping out gratuitous differences http://xrl.us/k2y4
The problem concerning multi-line attributes and Attribute::Handlers
noted by Bas van Sisseren (#38475), and its required fix to toke.c
was applied by Rafael, and the example was turned into a regression
test.
Won't see that again http://xrl.us/k7on
Rafael also applied the fix suggested in opening |- triggers unjustified
taint check. (#38709)
http://xrl.us/k7oo
Jerry D. Hedden continued to sync threads with blead . Applied.
http://xrl.us/k7op
And then he consolidated some XS functions. Also applied.
http://xrl.us/k7oq
Here's one that was missed in maint:
http://xrl.us/k7or
And another that was caught:
http://xrl.us/k7os
Nicholas Clark revived the POSIX build fails in bleadperl
and RTMAX patch bug (#36951). David Dyck confirmed that it is
still alive and kicking,
http://xrl.us/k7ot
and closed out the utf8 overload stringify bug (#34297), saying that
it was fixed in blead and that he hoped to backport it to 5.8.9.
http://xrl.us/k7ou
He then showed how uc/lc/ucfirst/lcfirst fail on typeglobs on
5.8.8 in bug #39019. The bug doesn't manifest itself on blead, but that is
because it's typeglob implementation is quite different these days.
http://xrl.us/k7ov
Joshua ben Jore wondered how to interpret the meanings of the MAD keys.
No-one else knew, either http://xrl.us/k7ow
Nicholas Clark wondered why lc and friends don't taint empty strings
when locales are in effect (bug #39028). Sadahiro Tomoyuki gave his thoughts
on the matter, explaining that perl was probably Doing The Right Thing.
http://xrl.us/k7ox
Abe Timmerman fixed up snsprintf and vsnsprintf problems for MSVC
on the Windows platform.
http://xrl.us/k7oy
This summary was written by David Landgren.
If you want a bookmarklet approach to viewing bugs and change reports, there are a couple of bookmarklets that you might find useful on my page of Perl stuff:
http://www.landgren.net/perl/
Weekly summaries are published on http://use.perl.org/ and posted on a mailing list, (subscription: perl5-summary-subscribe@perl.org ). The archive is at http://dev.perl.org/perl5/list-summaries/ . Corrections and comments are welcome.
If you found this summary useful or enjoyable, please consider contributing to the Perl Foundation to help support the development of Perl.