All the Perl that's Practical to Extract and Report

Pipelines

james on 2002-02-19T17:56:51

As more and more people have been looking at openframe the big thing everybody seems to be getting from it is that pipelines are a really nice way of doing things. We've known that at fotango for quite a while, which is why we wrote OpenFrame the way we did.

Anyway, lots of the feedback we've been getting around OpenFrame is that it seems like a really nice way to do things but its a bit heavy on the CPAN usage - so the first requirement for the rewrite is that we remove lots of the dependencies by bundling it differently, we don't want to change the fact that we don't reinvent the wheel, but we would like to make it a bit more convenient.

The second thing we want to do is come up with a generic pipeline module that people can easily used for different purposes.

I've spent most of the afternoon looking at the problem, and I figure that I've got the guts of it solved, and that the parts that are unsolvable, well, I won't try to solve them.

There is a very early version in the same repository as OpenFrame in the 'pipeline' module. If people are interested of course. And, if you don't know where the OpenFrame repository is, have a look at the website.

Pipelines

darobin on 2002-02-19T18:39:07

You're not alone thinking that pipelines are very neat :-) Perhaps we should have an OpenFrame-AxKit-AxKitB2B-SAX::Machines-Tessera-etc get together to talk about just how far we can get pipeline abstraction into a single reusable module? All of the above use different variations on the same principle, and not all can probably take advantage of common code (eg XML::SAX::Machines is probably too different) but I'm sure that something could come out of it.

And hey, we're on the same IRC server, it shouldn't be too hard ;-)

Re:Pipelines

james on 2002-02-19T19:34:06

Absolutly. I'm 100% in favour of this. As I've said in several places, ABW and I are quite a long way down defining a system for generic pipelines - the pipeline isn't the real problem, its the agreement on how to pass data along the pipe. But, I'm in favour in having a pipeline townhall. Have a look at the pipeline code that I've got going now, I'm pretty sure its generic, and I'm trying to add (right now) mechanisms to it so that pipelines and pipe segments can indicate what they can and cannot do, but in general the principles are all included.

CPAN deps

lachoy on 2002-02-19T18:44:35

Why is it a bad thing to have a number of CPAN dependencies? To me the CPAN shell makes this a non-issue. So the question may be: should we modify modules to have fewer dependencies or show people how to use the CPAN shell more effectively? (And for those modules that don't properly declare their prereqs, a polite email to the module author is usually sufficient to prompt a change.)

Re:CPAN deps

james on 2002-02-19T19:30:48

I completely agree - I'm all in favour of having module dependencies - it means CPAN is working, right? But regardless, I've had more complaints about the dependencies than compliments, so I'd rather unbundle core of OpenFrame from all the slots, and that way people can either get the whole shebang, or the parts they require.

Re:CPAN deps

djberg96 on 2002-02-20T03:28:28

I completely agree - I'm all in favour of having module dependencies - it means CPAN is working, right?

Yeah, but some people are stuck behind firewalls and can't use the shell (even though it seems I ought to be able to). Two other people have tried and failed to get by it as well.

You can always create a bundle.

Personally, I don't mind taking the time to go and get the modules off CPAN directly if I have to. I just start with Net::SSH::Perl with ssh1 and ssh2 support - that way, I've probably downloaded all the dependencies I'll need anyway. :)