Settling in and KQueue

Matts on 2005-02-16T20:38:20

Spent last week and this weekend settling in to the new house. Heather has done most of the unpacking, and I'm immensely grateful to her for that. This has allowed me to just get on with work while the chaos continues around me.

Today I've mostly been writing a perl module for calling the BSD/MacOSX kqueue() system call. I thought at first to try just using perl's syscall() command to do it, but I'm not entirely convinced that is possible due to requiring memory allocation and so forth. So instead I wrote a simple XS wrapper called IO::KQueue (I'm sure the module naming overlords will shoot me for that).

If anyone would like to test it please email me - I'd like to at least know it works on FreeBSD before uploading to CPAN. It requires perl 5.8 though (for ExtUtils::Constant).

The reason I've been doing this is that I've been working on highly scalable/parallel clients again. There's some modules for doing this kind of work (POE, Event, IO::Select, IO::Multiplex, etc) but none are really simple enough to achieve the kind of scalability I require. Instead I've been building on Brad Fitzpatrick's Danga::Socket code which he uses in perlbal - his perl based load balancer. Danga::Socket will transparently use epoll() if it's on your system, or poll() if not. I'll now be working on making it do kqueue() if it's available. Should be fairly simple to make that work, and then I'll be able to test my code on OSX that much easier.


Minium perl version

nicholas on 2005-02-16T20:49:22

It requires perl 5.8 though (for ExtUtils::Constant).

Why so? If ExtUtils::Constant isn't buggy, then I thought that it worked safely as far back as 5.005. You ship a pair of source files to fall back on if the user doesn't have it installed (which means that they can't tweak the list of defined cosntants) but anyone who does have it installed can tweak to their hearts content. Well, that was the plan. See Time-HiRes for an example of the fallback mechanism.

Re:Minium perl version

Matts on 2005-02-16T22:37:10

Just laziness. I do this stuff for work rather than for the benefit of the community now. Looks like a simple work around though.

I should also send you some doc patches for ExtUtils::Constant - the docs suck :-)

Re:Minium perl version

nicholas on 2005-02-18T21:10:06

I read them again recently when I was refactoring it, and yes, they do suck. Patches welcome, as it's unlikely that I'm going to get there first.

Event::Lib ?

dug on 2005-02-17T13:26:27

I think it's great that you're working on IO::KQueue (and I think the name is fine, in the vein of IO::Select and IO::Poll).

I've been quite happy with Event::Lib as a wrapper around various IO polling mechanisms (and although I don't use a system that supports kqueue, I hear libevent doesn't mind using it).

Was there something keeping you from using Event::Lib? The external library dependancy? I'm just curious, and like to hear about the selection process from others.

Congrats on settling in, I've been doing just that for about a year now [grin].

Re:Event::Lib ?

Matts on 2005-02-17T14:32:45

It's a number of things really - primarily that libevent is just more stuff than I need, and my code all integrates with Danga::Socket which uses low level calls to epoll/poll (and now kqueue) rather than going through another abstraction layer.

Also I know some people more knowledgeable than I am about this stuff who don't have high opinions of libevent :-)