Reference Counts

Dom2 on 2004-12-08T00:57:01

I've been looking at writing stuff using XML::Genx. Unfortunately, I soon came across a rather nasty little bug. There's an API which lets you pass in a filehandle and it gets all output sent to it. XS has a typemap to help you along with this: T_STDIO. That takes a perl filehandle and gives you a FILE* which you can pass into all the usual C functions.

The trouble is that the typemap doesn't increase the refcount on the perl filehandle. I don't think that it should, because it doesn't know when or where you are likely to need to decrement the refcount.

The net effect is that the filehandle is only valid as long as it's open in Perl. Otherwise it'll get GC'd (and consequently closed) when it goes out of scope. So you have to keep the original filehandle lying around. Surprisingly, I'd managed to do this in all the tests that I had written... Adding a test for this situation provoked a segfault (under Linux, but not FreeBSD for some reason).

Anyway, I'm now faced with two choices.

  1. Rewrite the typemap so that it increments the refcount of the filehandle. The trouble with that is that I then have to decrement that refcount when I get to EndDocument(). And I have nowhere to store the information that I need to do that because my object is a blessed scalar reference.
  2. Alternatively, I deprecate StartDocFile() and make everything use StartDocSender() which makes it easier to wrap things up in.

I don't want to deprecate the interface; it's a useful thing to have. But it seems that the alternative is to create a static HV (not visible to Perl) with a key of "$self" and a value of the filehandle SV (or is that really GV?). Then I can check for that in EndDocument() and decrement it appropriately there. I assume that will work, I have no idea what point Perl runs its GC at.

Four years of pure Perl have really softened me up when it comes to C. :-)


dup?

Matts on 2004-12-08T13:16:33

How about duping the filehandle?

Re:dup?

Dom2 on 2004-12-08T13:23:26

That's not a bad idea. Although there's still too many layers going on here. The filehandle goes from a PerlIO* to a FILE* to an fd. And then I'd have to dup, and fdopen(). A touch complicated, but it should work well.

Thanks,
-Dom

Re:

Aristotle on 2004-12-16T09:20:17

I think it's worthwhile to think about what should happen if someone passes you a Perl filehandle, then closes it before EndDocument().

What would happen with a duped filehandle? Would the duped one stay open if the original one gets closed?

I think that it should behave the way a pure-Perl module mostly like would: closing the filehandle closes it out from under the module, too. The duping behaviour seems safer, but I worry that this “safety” net could veil bugs in client code, in which case I'd prefer dumb behaviour that blows up in a user's face as soon as they do something stupid.

Re:

Dom2 on 2004-12-16T12:01:58

Unfortunately, that behaviour caused a segfault in some cases, which is why I was so keen to avoid it. I've worked around it by storing the file handle in a hash in the latest version, and releasing it when EndDocument() is called.

-Dom

Re:

Aristotle on 2004-12-17T00:04:42

Which means closing the Perl filehandle while GenX has a copy of it will blow in my face if I try it? That would be good, if so, and is all I was arguing for.

Re:

Dom2 on 2004-12-17T11:35:10

It might do, it might not. The behaviour was extremely variable and core dumped on some boxes but not others. Given the unpredicatbility, I felt that it was the right thing to do to store a reference to the filehandle until we're done with it.

-Dom