Creating Web Services with XML-RPC

jjohn on 2001-02-05T15:29:59

XML-RPC. SOAP. CORBA. Buzzwords galore, but possibly useful ones: this is, essentially, technology that allows you to use web sites as a big API. Below we have an article about XML-RPC; you might also want to check out an article by Paul Kulchenko on www.perl.com, "Quick Start with SOAP".

Web Services Are More Than HTTP

One of Perl's enduring strengths is in allowing the programmer to easily manipulate UNIX system resources, like /etc/passwd files, syslog and the filesystem. Perl is also a great tool for building network applications (for more on this, see Lincoln Stein's excellent Network Programming with Perl). As Nathan Torkington pointed out at YAPC 19100, the Perl community hasn't yet fully embraced a component model of programming. Components are libraries that can be used by programs written in any language. If you've dealt with Windows programming particularly ASP coding, you are probably familiar with COM objects already. But COM is just one vendor's implementation of components. Jon Udell argues that web applications can be used in a component fashion. How many of you have stolen the HTML from your favorite search engine's FORM and pasted it into your homepage? (I hope I'm not the only one raising my hand!)

Although LWP is a powerful tool for "page scraping", you probably don't want to be parsing HTML every time you make an object call. Fortunately, others have been working on this problem. XML-RPC is a protocol for Remote Procedure Calls whose conversation over TCP port 80 is encoded into XML. There are XML-RPC implementations written in many languages for many platforms, including Perl.

Every XML-RPC application must have two parts, but there's a third that ought to be required. If your script is making an XML-RPC request to some web service, you are making client calls. If your script is answering XML-RPC requests, it is a listener. This seems to describe a typical client-server system; what could be the third piece? Because XML-RPC is language and platform neutral, it is essential that the listener's API be adequately documented. Listing what procedures are expecting for input and the datatypes of the return values is a necessity when dealing with sub-Perl languages, like Microsoft's VBScript.

The XML-RPC protocol is a call and response system, very much akin to HTTP. One consequence of this is that XML-RPC doesn't support transactions or maintaining state. Another problem is the conversation between listener and client is in clear-text. Despite these limitations, there are still a number of applications for which XML-RPC is well suited.

Building a Client

The first quandary the novice XML-RPC Perl programmer will encounter is the name of the module. Ken MacLeod's implementation is called Frontier::RPC, because XML-RPC was the brain child of UserLand's Dave Winer, also of Frontier Naming issues aside, you can find this module on your local CPAN. Installation follows the normal "perl Makefile.PL && make test && make install" cycle.

Let's assume there's an existing XML-RPC service you want to talk to. It defines the following API:

Procedure Name | Input    | Output
-------------------------------------
hello_world    | <STRING> | <STRING>
-------------------------------------
sum            | <INT>,   |
               | <INT>    | <INT>
--------------------------------------

Remember, XML-RPC is designed to be language neutral. Every language's implementation of XML-RPC translates the RPC into an XML description that tags each argument with a specific XML-RPC datatype. Although Perl's DWIM-FU is strong, other languages are strongly typed and need this additional information. When the listener responds, Frontier::RPC translates the XML back into Perl datatypes.

Besides the API, we need to know the URL of the listener. For this example, we will use an NT machine on my private network.

Enough yakkin'. Here's a simple test of this web service.

     1	#!/usr/bin/perl --
     2
     3	use strict;
     4	use Frontier::Client;
     5
     6	my $rps = Frontier::Client->new(
     7		   url => "http://durgan.daisypark.net/jjohn/rpc_simple.asp",
     8					);
     9
    10	print "Calling hello_world()\n";
    11	eval { print $rps->call("hello_world", "jjohn"), "\n" };
    12
    13	print "\n=-----------------=\n";
    14	print "Calling sum()\n";
    15	eval { print $rps->call("sum", "1024", "128"), "\n" };
    16
    17	print "\ndone\n";

After including the library, we instantiate a new Frontier::Client object by passing the URL of the web service. We can now make our RPC by using the call() method, which expects the name of the remote procedure followed by a list of its arguments. If all goes well, call() converts the return value of the procedure into a normal Perl datatype. Why am I wrapping the method calls in evals? According to the XML spec, an XML parser is supposed to halt as soon as it finds a parsing error. Since Frontier::RPC is encoding and decoding XML, this is a precaution. Sometimes, bad things happen to good network packets.

The output looks like this:

[jjohn@marian xmlrpc]$ ./test_simple
Calling hello_world()
Hello, jjohn

=-----------------=
Calling sum()
1152

done

Sure, strings and numbers are useful but Real Functions ™ use collection types like arrays and dictionaries. XML-RPC can handle those datatypes as well.

Here's an example of a procedure that returns an array of hashes. It is getting all the records from an Access database stored on the aforementioend NT system. Notice that we are talking to a different XML-RPC listener now.

     1	#!/usr/bin/perl --
     2
     3	use strict;
     4	use Frontier::Client;
     5	use Data::Dumper;
     6
     7	my $rps = Frontier::Client->new(
     8		 url => "http://durgan.daisypark.net/jjohn/rpc_addresses.asp",
     9					);
    10
    11	print "\nCalling dump_address_book()\n";
    12
    13	eval { my $aref = $rps->call("dump_address_book");
    14	       print Dumper($aref);
    15	     };
    16
    17	print "\ndone\n";

All the usual Frontier::RPC suspects are here. This time, we are explicitly assigning the call method's return value. A collection type is always returned as a reference by this library. We can use this reference as we would an normal Perl variable, but Data::Dumper is a simple way to verify that the call succeed. For the record, here's a pared down version of the output.

[jjohn@marian xmlrpc]$ ./use_perl_taddrs

Calling dump_address_book()
$VAR1 = [
          {
            'email' => 'mc@nowhere.com',
            'firstname' => 'Macky',
            'phone' => '999 555 1234',
            'lastname' => 'McCormack'
          },
          {
            'email' => 'jjohn@cs.umb.edu',
            'firstname' => 'Joe',
            'phone' => 'no way, dude!',
            'lastname' => 'Johnston'
          }
        ];
done

Building a Listener

For those system administrators out there who want to build a monitoring system for the health of each machine on their network, installing simple XML-RPC listeners on each machine is one easy way to collect system statistics. Here is the code for a listener that returns a structure (really a hash) that is a snapshot of the current system health.

#!/usr/bin/perl --
use strict;
use Frontier::Daemon;

Frontier::Daemon->new( methods => {
				   status => sub {
				                   return {
						     uptime =>
						        (join "<BR>",`uptime`),
						     df     =>
							(join "<BR>", `df`),
							  };
				                 },
                                  },
		       LocalPort => 80,
		       LocalAddr => 'edith.daisypark.net',
		     );

The Frontier::Daemon class is a sub-class of HTTP::Daemon. The new method doesn't return; it waits to service requests. Because HTTP::Daemon is a sub-class of IO::Socket::INET, we can control the TCP port and address to which this server will bind (ain't Object Orient Programming grand?). The procedures that XML-RPC clients can call are contained in the hash pointed to by the methods parameter. The keys of this hash are the names of the procedures that clients call. The values of this hash are references to subroutines that implement the given procedure. Here, there is only one procedure that our service provides, status. To reduce the display burden on the clients, I'm converting newlines into HTML <BR> tags. Here's a screenshot of an ASP page that is making XML-RPC calls to two machines running this monitoring service.

Documenting Your API

Like the weather, everyone talks about documentation but no one ever does anything about it. Without documenting the API to your XML-RPC web service, no one will be able to take advantage of it. There is no official way to "discover" the procedures that a web service offers, so I recommend a simple web page that lists the procedure names, the XML-RPC datatypes expected for input and the XML-RPC datatypes that are returned. Something like Figure 1 is good a minimum. Of course, it might be nice to explain what those procedures do. POD is very amenable to this purpose.

Links to More Information

XML-RPC is a great tool for creating platform independent, network application gateways. Its simplicity is its strength. For more information on XML-RPC, check out the homepage at www.xmlrpc.com or wait for the O'Reilly book Programming Web Applications with XML-RPC, due out this summer.

No credit!

jjohn on 2001-02-08T02:27:59

Pudge, I think you left out the author's name on that piece. :-)

-"Shameless plug" Joe Johnston

Re:No credit!

pudge on 2001-02-13T14:20:18