How light is Lite?

brian_d_foy on 2002-11-20T15:16:45

Randal gave me an assignment last night to benchmark loading SOAP::Lite in a CGI script versus mod_perl. The SOAP stuff, which uses XML::Parser somewhere behind the scenes, is not the fastest thing in the world and takes a bit to load. This can kill CGI scripts under high load.

The Benchmark module, which comes with Perl, makes this easy (which I also write about in Computer Science & Perl Programming: The Best of The Perl Journal). I hit the web address a couple hundred times for each implementation method.

How do you think vanilla CGI, Apache::Registry, and Apache::PerlRun will compare to a mod_perl module? I start with a "Hello World" trial, and then I add in SOAP::Lite to see what sort of penalty I pay just to load the module.

#!/usr/bin/perl

use Benchmark;
use LWP::Simple;

print "-" x 73, "\n", "Running Hello World\n";

timethese( 300, {
        'CGI'      => sub { get( 'http://localhost/cgi-bin/hello.cgi' ) }, 
        'Registry' => sub { get( 'http://localhost/perl/hello.cgi'    ) },
        'Run'      => sub { get( 'http://localhost/perlrun/hello.cgi' ) },
        'Native'   => sub { get( 'http://localhost/helloperl'         ) },
        } );

print "-" x 73, "\n", "Loading SOAP::Lite\n";

timethese( 300, {
        'CGI'      => sub { get( 'http://localhost/cgi-bin/soap.cgi' ) }, 
        'Registry' => sub { get( 'http://localhost/perl/soap.cgi'    ) },
        'Run'      => sub { get( 'http://localhost/perlrun/soap.cgi' ) },
        'Native'   => sub { get( 'http://localhost/mysoap'           ) },
        } );

These benchmarks are a bit different than the ones I normally look at. All I care about is the wallclock time. The implementations could be really fast, but what I care about is how fast I see the result in my browser, not how long it took to run or how much CPU it used. In this case, the CPU measurements are for LWP fetching the page, not the work the web server is doing.

The results only mean anything relative to each other because they depend on what else I am doing on my machine. I did not bother to stop the real audio stream, for instance, so the numbers turned out larger than they did this morning. You will get different numbers. The relative size stays about the same, though.

-------------------------------------------------------------------------
Running Hello World
Benchmark: timing 300 iterations of CGI, Native, Registry, Run...
       CGI: 60 wallclock secs ( 1.76 usr +  0.00 sys =  1.76 CPU) @ 170.45/s (n=300)
    Native:  8 wallclock secs ( 1.42 usr +  0.00 sys =  1.42 CPU) @ 211.27/s (n=300)
  Registry:  8 wallclock secs ( 1.43 usr +  0.00 sys =  1.43 CPU) @ 209.79/s (n=300)
       Run:  8 wallclock secs ( 1.39 usr +  0.00 sys =  1.39 CPU) @ 215.83/s (n=300)
-------------------------------------------------------------------------
Loading SOAP::Lite
Benchmark: timing 300 iterations of CGI, Native, Registry, Run...
       CGI: 240 wallclock secs ( 1.50 usr +  0.00 sys =  1.50 CPU) @ 200.00/s (n=300)
    Native:  7 wallclock secs ( 1.44 usr +  0.00 sys =  1.44 CPU) @ 208.33/s (n=300)
  Registry:  5 wallclock secs ( 1.39 usr +  0.00 sys =  1.39 CPU) @ 215.83/s (n=300)
       Run:  8 wallclock secs ( 1.20 usr +  0.00 sys =  1.20 CPU) @ 250.00/s (n=300)

See anything surprising?