All the Perl that's Practical to Extract and Report

Some Fun Numbers

chromatic on 2008-01-08T21:04:16

With a production Perl 5.8.8, production Python 2.5.1, and unoptimized Parrot built for debugging from r24673, running each benchmark three times in a row to populate disk caches and the like and reporting the best numbers:

$ time perl examples/benchmarks/fib.pl
fib(28) = 317811

real 0m0.602s
user 0m0.596s
sys 0m0.004s

$ time python examples/benchmarks/fib.py
fib(28) = 317811

real 0m0.367s
user 0m0.364s
sys 0m0.004s

$ time parrot examples/benchmarks/fib.pir
fib(28) = 317811 1.949330s

real 0m1.965s
user 0m1.960s
sys 0m0.004s

That's not great for Parrot, but it's a debugging build without optimizations. However, there are some optimizations built in and enabled with special command-line flags:

$ time parrot -Cj -O3 examples/benchmarks/fib.pir
fib(28) = 317811 0.007451s

real 0m0.023s
user 0m0.016s
sys 0m0.004s

That's better. Again, this is with a debugging build. With an optimized, non-debugging build (that is, no debugging symbols, no assertions, and GCC's -O2 $ time parrot examples/benchmarks/fib.pir
fib(28) = 317811 1.156776s

real 0m1.170s
user 0m1.164s
sys 0m0.000s

That's better. Now the punchline:

$ time parrot -Cj -O3 examples/benchmarks/fib.pir
fib(28) = 317811 0.007305s

real 0m0.020s
user 0m0.012s
sys 0m0.004s

Of course, it's important not to get too cocky. There are languages specifically optimized for generating the Fibonacci sequence:

$ time ./haskell_fib
"fib(28) = 317811"

real 0m0.003s
user 0m0.000s
sys 0m0.004s

Still, for a development version of Parrot that's not bad.

Sort of a lower bound

mdiep on 2008-01-08T22:01:41

I think it's important to note that the PIR example represents something of a lower bound for languages targeting Parrot.

The PIR code is designed to be fast and is most likely simpler than the generated code will be from any HLL compiler (and it doesn't need to load any whole language runtimes). So while I hope that Perl-6-on-Parrot will generate such fast code, in all likelihood I think it'll be a fair bit slower than that.

It's a good thing that the PIR is that fast, but I don't think it's much of a metric. If the PIR were slower than Perl/Python, we'd know that Perl 6/etc was going to be slower, but the fact that it's faster doesn't mean that Perl 6/etc will be faster – it just means that it could conceivably be faster.

Re:Sort of a lower bound

chromatic on 2008-01-09T00:37:20

That's a fair point, but it's also important to note that it's possible to write efficient and cross-platform PIR by hand to use from any HLL in case you need additional speed. Compare that to dropping down to C (and then trying to distribute that C across multiple platforms).

Even if Perl 6 on Parrot is an order of magnitude slower than PIR on Parrot, it's still faster than Perl 5 on C.

Good enough for me

muixirt on 2008-01-09T03:07:32

Ok, I won't complain anymore ... but wait, what about der perl6 binary?

Apples and Oranges

leo on 2008-01-09T23:51:50

With ...

$ time parrot -Cj -O3 examples/benchmarks/fib.pir
fib(28) = 317811 0.007305s

... you are just measuring parrot startup time, which isn't optimized in any way.

Please try something real ;-) like fib 38.

$ time ./parrot -Cj -O3 examples/benchmarks/fib.pir 38
fib(38) = 39088169 0.671501s

(on my ancient 2GHz Athlon). Wirh perl 5.8.8 it's consuming some time:

$ time perl examples/benchmarks/fib.pl 38
fib(38) = 39088169

real    2m3.206s

which is exactly what I've always said: compiler to interpreter ops performance is about 200:1 (which doesn't say anything about any program performance, e.g. perl regexes are ~equally fast compared to the pcre library)

$ echo '(2*60+3)/0.67'|bc
183

And a final note: these numbers don't say anything about perl6 or OO code or whatever - these are peak performance possibilities of the parrot engine with JITted native types.

leo