All the Perl that's Practical to Extract and Report

Concurrency

excalibor on 2006-09-12T16:33:18

While I wait for my freshly installed-from-sources Perl 5.8.8 to run the cat-install script to install Catalyst, I'll mumble about concurrency in Perl, now that I am trying to learn Erlang.

Concurrency can be doen at the level of process (fork(), which is fork(2) and friends on UNIXen) or at the level of thread.

Since Perl 5.8.x we have threads, although a bit shakey. However, threads mean semaphores, locks, etc... Ugly.

CPAN to the rescue:

subs::parallel looks very nice. I will have to stress it to see if it's really usable, but it puts perl threads to the use in a very simple way, I like it.

Parallel::Simple also looks very nice, although it uses fork(). This has also problems, but nowadays fork() is very fast and cheap in UNIXen.

If you have to run the same thing many times, and it can run in parallel, Parallel::ForkControl is very nice. I've used it in the past to run control code on a big subnet, limiting the amount of bandwidth I sucked up, and it improved my partner's program by 3 orders of magnitude (yeah, that's right, 3 hours--counting the sleep we added on purpose--instead of 3 months) by improving the use of the IO bottleneck... :-)

There are other CPAN modules so you don't have to use threads or fork() yourself. Have a look on the module namespaces Proc::*, Parallel::* or forks::*, for example, and see which one suites your needs the best.

BTW, nothing of this is similar to Erlangs lightweight processess, which use message passing to work, and I don't think we could make a similar thing that worked easily enough.

Termite is a Scheme implementation of Erlang's concurrency model, and a couple of changes to the compiler/interpreter were necessary to implement it in Scheme, so I guess some changes would be needed in perl itself (i.e. in the compiler/interpreter).

Simulating Erlang's ability in perl is a different story, which I may tackle one of these days if I have time enough... :-P

In the meantime, I'm back to fighting with CPAN.pm and my proxy, which is stupid enough to be really annoying (argh!). BTW, any way to let CPAN know it has to use the HTTP methods instead of FTP? Using the fully qualified HTTP URI of my proxy doesn't seem to work... :-/

Best regards!

disable ftp in CPAN

diakopter on 2006-09-12T20:30:39

in the cpan shell...
! $CPAN::Config->{dontload_list} = ('Net::FTP');

Re:disable ftp in CPAN

diakopter on 2006-09-12T20:35:40

Of course, this assumes you already have LWP installed... which has a large number of prereqs that aren't in the 5.8.8 core.

Re:disable ftp in CPAN

excalibor on 2006-09-12T22:04:26

Well, heh, it says LWP is not installed...

I wish we had a properly ordered CORE... I still think that a layered approach to packaging would put some order to this little chaos of ours (see the comments section to my last journal post).

Okay, I don't have LWP... If I download the tarball, will it come with all the dependencies solved?

(anyway, I'll try it tomorrow, thanks for the tips!)

Re:disable ftp in CPAN

diakopter on 2006-09-12T22:35:23

No, it does not come with its dependencies. :(

Event::ThreadFarm

uri on 2006-09-13T07:32:14

i have a design for a module which will likely do what you want which is threads without perl threads. it will be called Event::ThreadFarm and it will create a farm of kernel/c level threads which interact with a perl level event loop via pipes. operations can be sent to the threads which will run plugins (which come with the module or custom). the perl side will be a callback api which interacts with the rest of the perl process. IO::AIO is a similar module in concept but narrower in scope as it doesn't do plugins and it is tied to one event loop style.

also stem is a message passing based system in perl which might help you with your apps. it is much higher level than the fork:: and proc:: modules. the cpan version is way out of date but you can get the beta of .11 at stemsystems.com/Stem-0.11.tar.gz.

when Event::ThreadFarm is out and integrated with stem you will be able to create a message passing framework in a single process that can handle events or dispatch all blocking operations to fast kernel threads. a good combination to look forward to IMNSHO. :)

uri