All the Perl that's Practical to Extract and Report

Malformed UTF8 Hoohah

TorgoX on 2003-10-12T21:14:28

Dear All,

I am puzzled. The current verison of Pod::Simple installs happily and quietly under 5.8. But under 5.6, trying to install it leads to banshee-like screaming about malformed UTF8 characters. I've no idea why. Can anyone offer any clues? I'm really not doing anything exotic in my Pod::Simple code.

details

jhi on 2003-10-14T14:22:49

Any chance of telling a little bit more about the banshee part?

Re:details

TorgoX on 2003-10-15T05:33:50

I think this is about the most minimal case I can produce: podsimpleutf8hell.pl

Re:details

jhi on 2003-10-15T05:49:57

I'm afraid it looks like 5.6.1 is out of luck here. In other words, looks like yet another Unicode bug in 5.6.1. What it looks like is that the 0xe9 or ISO Latin 1 eacute does not get correctly upgraded to UTF-8, and this fatally annoys s/// and ord().

You *may* be able to dodge this by forcing the data at appropriate spots into UTF-8: something along the lines of:

$ascertain_utf8 = $] < 5.008 ? sub { $_[0] .= chr(0x100); chop $_[0] } : sub { };