The following should all do (almost!) the same:
pipe( 'zcat' ) | [ tar => x => '-O' ] Pipe->new( 'zcat' ) | Pipe->new( tar => x => '-O' ) Pipe->new( 'zcat' )->std( Pipe->new( tar => x => '-O' ) ) Pipe->new( 'zcat' )->stdout( stdin => Pipe->new( tar => x => '-O' ) ) Pipe->new( 'zcat' )->pair( stdout => stdin => Pipe->new( tar => x => '-O' ) )
Thoughts? (Aside from module naming.)
(Rationale: setting up complex pipes manually with IO::Pipe/socketpair + fork results in brain-melting code. Shell shows, however, that pipes can easily be described declaratively. Why is there, 20 years after the inception of Perl, no module that provides a piping API congruent with shell syntax?)
Have you have a look at mugwump's IO::Plumbing? Maybe with a little more syntactic sugar it could do the kind of thing you're talking about.
Re:IO::Plumbing?
Aristotle on 2009-02-12T09:45:27
Thanks for the pointer. That looks mostly like what I was thinking of, indeed, just with a far clumsier API. Getting the API right is the tricky part – the implementation isn’t very exciting.
Re:IO::Plumbing?
educated_foo on 2009-02-13T03:47:37
Indeed -- that's one for the Hall of Lousy Interface Fame. I've toyed with extending IO::All to do this kind of thing, but never produced anything worth sharing.
Re:IO::Plumbing?
Aristotle on 2009-02-13T16:44:55
I’m not touching IO::All with a 10ft pole anyway.
:-) Re:IO::Plumbing?
educated_foo on 2009-02-17T00:22:04
The flashbacks stop... eventually.
Re:IPC::Run?
Aristotle on 2009-02-13T05:52:51
I saw it when it was first released and got really excited. The API just doesn’t feel natural, though. It’s almost purely function-based, has this pumping concept which is an obvious use case cast into a weird API shape, and its use of strings as faux operators is at once too much and too little syntactic sugar: because it deals in strings, it requires a lot of noise characters and leaves the user responsible to account for the lack of operator precedence. So the clarity advantage of special syntax is nullified, while also passing on the explicitness advantage of more verbose identifiers.
In summary, ugh.
Honestly, I’m not trying to be a curmudgeon.
:-( I really wanted to like at least one of the existing modules.
Maybe you could write a Text::Pipe plugin that does what you need? See this test file for an example using syntactic sugar of how text pipes are stackable.
Re:Text::Pipe?
Aristotle on 2009-02-13T16:43:03
The API in Text::Pipe is nice. However, the module addresses a much simplified problem to begin with. There are no cases like connecting the stdout from one process to the stdin of another process but fd3 of the first process to the stdin of a third – to cater to such use cases, an Unix piping module will need a solid object model for use by desugared code doing more complicated things. Text::Pipe’s design does not fit this bill.
Then there is also the need to provide amenities for real IPC stuff like picking up exit statuses. I am not sure exactly what support is needed for these things in a convenient Unix piping API. Depending on the shape, Text::Pipe’s design may not be adequate. I can’t say yet.
Text::Pipe would be a good wrapper for something simplified, at the level of IPC::Filter, though.
I looked into doing something very much along those lines some time ago (during OSCON::EU 2006 in Brussels, I think).
One of the problems is that callers which do use warnings will get lots of “Useless use of bitwise or (|) in void context” warnings. I considered whether this sort of thing would alleviate the problem:
package Whatever::Name::It::Gets;
use strict;
use warnings;
sub import {
warnings->unimport(qw<void>);
}
It would, through the surprising scope of *^H, but there’s an obvious disadvantage: you don’t get any other void-context warnings.
I was additionally trying to get <, > , and >> to work as redirection operators. That turned out to be even trickier: things like
Pipe('zcat') | [qw<tar x -O>] > $filename;
won’t work because of operator precedence. Perl conveniently warns you about this: “Possible precedence problem on bitwise | operator”; whether that’s actually better than the alternative is debatable in this case.
Finally, I believe it’s also impossible to use pipe as the convenience function name for these purposes:
$ cat foo.pl
use strict;
use warnings;
sub pipe { 1 }
pipe('foo') | pipe('bar');
$ perl foo.pl
Ambiguous call resolved as CORE::pipe(), qualify as such or use & at foo.pl line 4.
Ambiguous call resolved as CORE::pipe(), qualify as such or use & at foo.pl line 4.
Not enough arguments for pipe at foo.pl line 4, near "'foo') "
Not enough arguments for pipe at foo.pl line 4, near "'bar')"
Execution of foo.pl aborted due to compilation errors.
That’s by no means an insuperable objection, of course. For that matter, the other issues can all be worked around to produce a good interface, but it’s particularly annoying that what seems initially to be an ideal interface isn’t apparently implementable.
Re:Operator overloading to describe pipelines
Aristotle on 2009-02-17T00:06:12
I wouldn’t particularly care about the redirection operators. I would rather let the desugared API handle the more complex cases than try to stuff them all into symbols.
The warning is an annoyance, I agree.
