MP3::Mplib 0.01

ethan on 2002-12-02T12:35:31



The uploaded file

MP3-Mplib-0.01.tar.gz

has entered CPAN as

file: $CPAN/authors/id/V/VP/VPARSEVAL/MP3-Mplib-0.01.tar.gz size: 80543 bytes md5: 7acf4fa07a13648ecbc9e0509a27b6da

No action is required on your part Request entered by: VPARSEVAL (Tassilo von Parseval) Request entered on: Mon, 02 Dec 2002 12:15:17 GMT Request completed: Mon, 02 Dec 2002 12:15:57 GMT

Virtually Yours, Id: paused,v 1.81 2002/08/02 11:34:24 k Exp k



Another tough session today made the above possible. Perl5.8.0 complained about some readonly values that I tried set somewhere within my XS code. It took a while to find the delinquent. 5.005_03 and 5.6.1 did not have these objections.

In the last moment I decided to use Test::More instead of Test. That made a few rewrites necessary, too.

Also, h2xs obviously screwed up when I created the extension skeleton some months ago. This part was funny actually: When I tested all the constants from mplib.h, Perl failed to export UTF16...but only on this one. UTF16BE and all the other encodings worked ok. I located the error somewhere here:



static double constant_UTF1(char *name, int len, int arg) { if (4 + 1 >= len ) { errno = EINVAL; return 0; } switch (name[4 + 1]) { case '\0': if (strEQ(name + 4, "6")) { /* UTF1 removed */ #ifdef UTF16 return UTF16; #else goto not_there; #endif } case 'B': if (strEQ(name + 4, "6BE")) { /* UTF1 removed */ #ifdef UTF16BE return UTF16BE; #else goto not_there; #endif


It turned out that the first if-condition was messed up. It should have been:

if (3 + 1 >= len ) {

I have no explanation for this failure but I find it somehow amusing that even a machine can come up with an incorrect calculation of a string length. This puts my sometimes scaringly silly errors into the right perspective, I think. ;-)

<rant>

Who actually came up with this insane constant handling in XS code? A constant name is cut into strings of variable length and then control is delegated to another suitable constant-function that exclusively deals with constants of a particular prefix. If that is a performance tweak then it is silly since any constant is only retrieved once and after that compiled into a subroutine by AUTOLOAD.

</rant>


Insane XS constant handling code

nicholas on 2002-12-02T14:01:12

Who actually came up with this insane constant handling in XS code? A constant name is cut into strings of variable length and then control is delegated to another suitable constant-function that exclusively deals with constants of a particular prefix. If that is a performance tweak then it is silly since any constant is only retrieved once and after that compiled into a subroutine by AUTOLOAD.

Fair comment.

What you saw with the miscalculated if condition looks like the bug I found in the constant generation routines in perl 5.6.x. It pissed me off sufficiently to fix it (now in perl 5.8), and write regression tests to try to stop it happening again.

The h2xs constant generation was changed in 5.6.something. I don't remember who made the change, but I did find the code it generated extremely hard to follow, and wasn't convinced that it was a speed win, for the very reasons you observe.

The change made for 5.6.something only attempted to generate faster lookup code, without changing the basic idea - generate the switch table once when h2xs is first written. This makes it very hard to add new constants later on, if later versions of the module you're wrapping add new constants. It also means that you're stuck with whatever lousy autogenerated switch code was the style at the time of the h2xs you ran.

The change I made for 5.8 changes the approach taken, in an attempt to fix these deficiencies. Rather than generate the C code for the constants, h2xs now adds perl code to Makefile.PL that lists the names (and types, although h2xs can't work these out yet) of the constants, and a call to a new module, ExtUtils::Constant, which turns these into XS and C code. This conversion is done at module build time (with fallback C and XS files supplied in the module tarball in case the user doesn't have ExtUtils::Constant installed). So if you (the module developer) need to add more constants to the switch table, you just add them to the list in Makefile.PL and rerun it. And if anyone improves (or changes) the switch table generation code, everyone gets the benefit, even on existing modules. (ExtUtils::Constant 0.14 did improve the code slightly over 0.13). And I hope that the switch table code in ExtUtils::Constant compiles to a more compact form, and executes more quickly, than the code that h2xs used to generate. (It tries to make switch statements that the compiler will convert to branch tables, and tries to minimise the number of linear lookups needed after the switch tables)

ExtUtils::Constant is on CPAN, and works with perl at least as far back as 5.005_03. If you want to auto-generate Makefile.PLs that use it you'll need to run h2xs in perl5.8.0, specifying the -b flag to say what version of perl you want your module to be compatible back to. Oh, and ExtUtils::Constant can do Unicode constants correctly, which is more than the old h2xs code could do.

And the best bit of all this - if you think my autogenerated switch code sucks, you just write some perl to generate better switch code, wrap it with the same API that ExtUtils::Constant uses, and hey preso, all your constants are switched your way.

ExtUtils::Constant

ethan on 2002-12-02T15:22:07

Quite relieved to see that I am not the only one with these woes so I don't have to feel stupid. :-)

This makes it very hard to add new constants later on, if later versions of the module you're wrapping add new constants. It also means that you're stuck with whatever lousy autogenerated switch code was the style at the time of the h2xs you ran.

That was exactly my second thought after fixing constant_U(): What happens if the next release of mplib has additional constants? I don't want to re-run h2xs (and stuff my XSUBS into it) just to realize later that the constant-switchs weren't able to deal with a particular constant. It must be quite horrible to fix it if you have 50 or so constants. It was already hard enough in this pretty simple case with 15 or so #defines!

ExtUtils::Constant is on CPAN, and works with perl at least as far back as 5.005_03. If you want to auto-generate Makefile.PLs that use it you'll need to run h2xs in perl5.8.0, specifying the -b flag to say what version of perl you want your module to be compatible back to.

Sounds brilliant! Also, 5.005_03 is the very version which my module claims to be backwards compatible with. The next time anything constant-related changes (or next time I'll tackle an XS module) I'll switch to ExtUtils::Constant. Thanks for mentioning your module!