All the Perl that's Practical to Extract and Report

Following convention across language boundaries

doubi on 2010-07-20T18:10:18

I've been meaning to write this post for a while. I'd like your opinion on it.

In my last post [0] I touched on the question of following the Python ctypes API in the context of Type objects. The issue of how closely to stick to the Python API has come up a few times. There just bits and pieces I'm not wild keen on. The whole WINFUNCTYPE / CFUNCTYPE function prototype factory thing [1] for callbacks is a good example (in our implementation creating callbacks is as simple as my $cb = Ctypes::Callback->new(\&perl_func, <returntype>, <argtypes>);).

Who is the audience?

The argument for of course is the assumption that some of the first and most important users of Perl's Ctypes module will be C library authors or porters who already have a Python binding and will be interested in doing the same thing for Perl. Obviously if the Perl Ctypes implementation followed Python's 100% then the friction for those users would be low as possible and we might get more library bindings written sooner.

I think that's a worthy objective. I'm just not completely decided on whether the best ordering of preference is, "Copy the Python API exactly and improve and add features where possible," or, "Write the best module possible, and follow the Python conventions where you can and it makes sense."

My personal preference is for the latter. I've always seen Perl authors as the main clients of the module, which I think makes sense, especially over the long term. So I'd preference doing things better to following Python's conventions to a fault.

Example: Functions and prototypes

A good example of an API change is the fact that Python doesn't have a simple way to set Function object properties using constructor arguments. A feature of ctypes is the ability to use auto-load and use functions like so: 

>>> print hex(windll.kernel32.GetModuleHandleA(None)) # doctest: +WINDOWS
0x1d000000

This is fine for simpler functions, but for more complex examples you might have to specify argument types and/or return type. The simple way to do this is in three separate steps [2]:

>>> strchr = libc.strchr
>>> strchr.restype = c_char_p
>>> strchr.argtypes = [c_char_p, c_char]

The other way is to use a [WIN|C|PY]FUNCTYPE factory function appropriate to (the C calling convention of) your system, which returns a prototype class which in turn must be instantiated in one of four different ways to get the actual function object you want to use. A cynical reader of the ctypes docs would also point out that the whole mechanism is sequestered into the Reference document, left out of the Tutorial part altogether (apart from the part on callbacks, because there's no other way to make them).


In Perl's Ctypes on the other hand, we have sensible, somewhat clever constructors for Functions, so we can combine the three lines above into one statement, and not have to worry about generating new bespoke classes/packages:

my $strchr = CDLL->libc->strchr({ restype => [ 'c_int' ],
  argtypes => ['c_char_p', 'c_char'] });

or even

my $result = CDLL->libc->strchr({ restype => [ 'c_int' ],
  argtypes => [qw(c_char_p c_char)] })->("abcdef", "d");    # "def"

I think this is a good example of Doing It Better. In the Perl module, you're of course still able to specify Function properties individually if you want with $strchr->argtypes(). And we'll almost certainly replicate the *FUNCTYPE shenanigans later too, if only to appease porters.

Out with the old?

That's fine where different interfaces can live alongside each other, but in the case of the fundamental behaviour of Type objects, what I consider an improvement in behaviour would represent a divergence from the Python way of doing things (see previous post [0]).

In these instances, which philosophy should win out? "Copy the Python API exactly and improve and add features where possible," or, "Write the best module possible, and follow the Python conventions where you can and it makes sense?" I think the answer needs to come logically from the expected clientèle of Perl's Ctypes module. Maybe there are other factors to think about as well. I've stated my leanings, but I'm very keen to hear more opinions on the matter.

[0] http://blogs.perl.org/users/doubi/2010/07/thoughts-on-ctypestype-object-api.html
[1] http://docs.python.org/library/ctypes.html#callback-functions
[2] http://docs.python.org/library/ctypes.html#specifying-the-required-argument-types-function-prototypes