Making Journals from Emacs

jjohn on 2002-10-25T09:00:00

Like journaling at use.perl.org but hate composing entries in the HTML widget TEXTAREA? Tired of the Web Services hype and what to see a real application? Hold on to your hats, true believers, because the answer to both problems lies in the article below.

Emacs, Perl and SOAP: The New Axis of Evil

By Joe Johnston

Darkness Gathers: The Truth about Journaling

Online Journaling: what's not to like? Instantly publish your considered opinions to the entire world through the modern marvel of web logging. Fed up with the prices at Starbucks? Pissed off at the W3C? Need to vent spleen about those meddling kids? Now you can do all this and more for free on many web sites including use.perl.org. From any browser on any computer, you can log on and wax vitriolic, poetic or just plain nutty on any topic that resonsates strongly with you. Of course you will have to contend with your journal site's user interface, which is likely to consist mostly of an HTML TEXTAREA box.

Although beginners and casual users won't really notice the limitations of this HTML widget, experienced hackers will be slowly driven mad by the lack of Real Editting© features, such as kill rings, spell checking and file insertion to name a few. Some browsers, like Internet Explorer, try to help by at least allowing cutting and pasting in the TEXTAREA buffer via a menu that appears with a right-mouse click. For a real hacker, this is a meager editting repast. Why can't you use your favorite editor for journaling?

At least on use.perl.org, now you can.

A New Hope: The SOAP Journal API

Back in March, 2002, Chris Nandor got bitten by the Web Services bug. Lucky for use.perl.org, the infection wasn't fatal and he soon revealed a Simple Object Access Protocol API that allows users to manipulate journals here on use.perl.org. For easy reference, that API is reproduced in Table 1.

Table 1: Journaling Serivce API

Proxy: http://use.perl.org/journal.pl
URI: http://use.perl.org/Slash/Journal/SOAP
Methods
Method Name Input Output Description
add_entry string,
string
or record
integer Given the subject line and body, add this entry to your journal. Returns the new entry ID if successful or a false value otherwise. In place of two strings, add_entry accepts a structure with the fields 'subject', 'discuss', 'body', 'posttype', 'tid'.
modify_entry integer,
structure
integer Given a entry ID and the same structure listed in add_entry() update an existing journal entry. Instead of a structure, key-value pairs can be passed as a list. Return the entry ID if successful or a false value otherwise. You must be the owner of the entry to make changes to it.
delete_entry integer boolean Given an entry ID, return true if the delete was successful. Otherwise, return false. You must be the owner of the entry to delete it.
get_entry integer record Given a journal entry ID, return the record associated with it.
body           => the body of the entry
subject        => the subject line of the entry
discussion_id  => the ID for this discussion
discussion_url => the url to just the comments
                      on this entry
posttype       => an integer describing how the
                      entry should be render
date           => an ascii datestamp
tid            => Topic ID (not used for journals)
nickname       => the nickname of the entry's owner
uid            => the owner's UserID
url            => the URL to this entry
get_entries integer,
integer
array of records Given a UserID and Limit, return a list of that user's most recent journal entries. Limit is optional. The record format is:
url     => URL of this journal entry
id      => the ID of this entry
subject => the subject line of this entry
get_uid_from_nickname string integer Given a nickname, return the associated use.perl.org UserID

This API provides all the functions needed to add, modify, delete and list journal entries. Of course you don't want just anybody changing your journal so there must by an unlisted authentication method, right? Wrong. Slash itself uses HTTP cookies to authenticate clients. Since SOAP is built on top of HTTP, many SOAP clients can also pass cookies along and SOAP::Lite is not exception. To authenticate to the jounraling service, SOAP clients fabricate authentication cookies, as shown in Listing 13.

The journaling service API is still growing and should be considered alpha code, in that features are still being added. The most notable example of this is modify_entry(). After you get an entry, you will notice several metainformation fields that you can't alter, like the discussion_id and date. In the perl client below, only the subject line and body are ever submitted to modify_entry(). I'm unsure how much control the API allows over these housekeeping fields.

A typical way of using this API is to first list your most recent entries with list_entries(), retrieve an entry you wish to update with get_entry() and send the changes back with modify_entry(). Adding new entries can be done by supplying add_entry() with the subject line and body of the new journal entry.

As useful as the API is, it doesn't solve the problem of getting journal entries from your editor on to use.perl.org. What's needed is a SOAP client that can talk to this web service.

A Simple Perl Client

Before muddying the waters with editor-specific macros, let's build a command line perl script that is a SOAP client. Those who haven't used SOAP::Lite before may want to read this article I wrote for developerWorks. A quick look at Listing 1 reveals the modules required by this client, all of which can be found on CPAN. You will need to change two constants at the top of the program when you use this script yourself. You will need your UserID on use.perl.org and your password. Your UserID often appears as a number in parentheses next to your username on the info page of your account. Go to http://use.perl.org/user/[YOUR_USERNAME_HERE] to see this. The URI and PROXY constants are used to let SOAP::Lite how to find the use.perl.org journaling web service.

Listing 1: perl journal client, pt. 1

     1	#!/usr/bin/perl --
     2	#
     3	# Make SOAP calls to use.perl.org
     4	# Based on N.A.N.D.O.R work
     5	#
     6	# jjohn@taskboy.com 10/02
     7	# $Id: journal_client,v 1.3 2002/10/22 23:53:45 jjohn Exp $
     8
     9	use strict;
    10	use HTTP::Cookies;
    11	use SOAP::Lite;
    12	use File::Basename;
    13	use Digest::MD5 'md5_hex';
    14	use Data::Dumper;
    15
    16	use constant DEBUG   => 0;
    17	use constant UID     => 777; # your UID here
    18	use constant PW      => 's3cr3t'; # your password here
    19	use constant URI     => "http://use.perl.org/Slash/Journal/SOAP";
    20	use constant PROXY   => "http://use.perl.org/journal.pl";

This one perl script, spartanly called journal_client, will make any of the six jounraling service API calls, depending on how the script is invoked. I created symlinks to this script with the names that appear as keys in the ALLOWED hash (see Listing 2). These keys are mapped to subroutine references that will assemble the parameters and make the SOAP calls. This dispatch system is similiar to the way I write CGI programs. For those kinds of scripts, I also use a hash of possible actions mapped to implementing subroutines. I then look at an 'action' parameter to determine what to do. There will be more similiars to CGI in this script, as we'll see.

Listing 2: perl journal client, pt. 2

    21	use constant ALLOWED => {
    22				 # invoked as...local function
    23				 get_entry    => \&get_entry,
    24				 list_entries => \&list_entries,
    25				 add_entry    => \&add_entry,
    26				 modify_entry => \&modify_entry,
    27				 delete_entry => \&delete_entry,
    28				 whois        => \&whois,
    29				};
    30

Listing 3: symlinks to journal_client

  lrwxrwxrwx    1     14 Oct 21 13:29 add_entry -> journal_client
  lrwxrwxrwx    1     14 Oct 21 14:07 delete_entry -> journal_client
  lrwxrwxrwx    1     14 Oct 21 13:29 get_entry -> journal_client
  -r-xr-xr-x    1   4217 Oct 22 19:53 journal_client
  lrwxrwxrwx    1     14 Oct 21 14:07 list_entries -> journal_client
  lrwxrwxrwx    1     14 Oct 21 13:29 modify_entry -> journal_client
  lrwxrwxrwx    1     14 Oct 22 19:02 whois -> journal_client

In Listing 4, the special variable that contains the name of the program as it was invoked, $0, is passed to basename to remove additional path information. This is how the script know what action to perform. The listing below shows I used symbolic links to this program, journal_client.

Listing 4: perl journal client, pt. 3

    31	# How was I called?
    32	my $action = basename($0);
    33	my $in     = parse_input();
    34
    35	if (DEBUG) {
    36	  print "I was called $action with the following args:\n",
    37	    (map {"$_\n"} @ARGV), "\n";
    38	  print "Parsed Input: ", Dumper($in), "\n";
    39	}
    40

Although some parameters will be passed on the command line (similiar to HTTP GET requests), some parameters appear on standard input, like an HTTP POST request. The format of what comes in through standard input is very simple: key-value pairs. Keys are defined as left-justified non-space characters followed by a colon and a space. Values are everything after that until a new key appears. Some may ask why I didn't use XML for this. After all, XML is the premier data interchange format. One reason I choose this more primative format is that it is easier to parse. Second, I know I'll be dealing with HTML tags inside values and I don't want to bother escaping them to appease the XML parser. Input is passed to parse_input(), which returns a hash reference of any key-value pairs found. For your convenience debugging code is left in this script, but is disable by default.

Recall that this client needs to authenticate itself using a cookie. The easiest way to create HTTP cookie strings is to use the module HTTP::Cookie. Listing 5 shows that the cookie, which contains one user defined key-value, has a key called 'user' that is set to a string created with user's credentials. Chris Nandor is the author of the bakeUserCookie() function, so kudos to him (from Chris: "I didn't write it (though I probably modified it at one point or another): I just ripped it out of Slash"). You may also be able to use your browser's cookie file instead of baking your own, but this is left as an excerise for the reader.

A new SOAP::Lite object is created with the values specified in the constants section. Notice that the authentication cookie is passed into the proxy method. Although SOAP::Lite isn't a subclass of LWP::UserAgent, it does have an instantiated LWP::UserAgent object and that's what gets the $cookie_jar variable. You can set other LWP::UserAgent fields, like "timeout" and "agent" from here as well.

Next comes the dispatch section. Based on the name used to invoke this script, an action is performed. All subroutines that implement an action get the SOAP::Lite object, a reference to the command line arguments and the reference to the hash of key-value pairs found in standard input. If you are having CGI flashbacks right now, there's good reason. I've already mentioned that the command line arguments are somewhat like GET requests and you may have noticed that the getting values from standard input is a bit like an HTTP POST request. Had I been particularly twisted, I might have made the input to this script identical to HTTP GET and POST. Then I could have used the CGI module to fetch my parameters! Not only would that have been silly but it would have perhaps had the unfortunately side-effect of insinuating that web services must always have some kind of CGI component.

Listing 5 ends with a call to exit(). Although programs like to determine the success or failure of subroutines by looking at the boolean return value of the function, operating systems like Unix treat non-zero process exit values as an indication of failure. Therefore the return values of the action subroutines are remembered and their inverse is returned to the operating system. That's the end of the main line. Remarkably short, don't you think?

Listing 5: perl journal client, pt. 4

    41	# Everything is ready for the SOAP call now
    42	my $cookie_jar = HTTP::Cookies->new;
    43	$cookie_jar->set_cookie(0,
    44				user => bakeUserCookie(UID,PW),
    45				"/",
    46				"use.perl.org");
    47
    48	my $c = SOAP::Lite->uri(URI)->
    49	                    proxy(PROXY, cookie_jar => $cookie_jar);
    50
    51	my $rc = 0;
    52	if (exists ALLOWED->{$action}) {
    53	  $rc = ALLOWED->{$action}->($c, \@ARGV, $in);
    54	} else {
    55	  die "Oops: no such action: $action\n";
    56	}
    57
    58	exit ($rc ? 0 : 1);
    59

Listing 6 shows the first of the subroutines that implements the script's actions. This one expects to get a single integer from the command line. This integer should be a journal entry ID, however there is no way to verify this before making the SOAP call (line 68). SOAP calls can fail because of transport errors. For instance the site may be down, or you called a non-existant method. There's not much to do to recover from those kinds of errors, so I simply return from the subroutine sullen and rejected. Sematic errors, like asking or 1/0, still need to be watched for by examining the method's return value. This is no different than the precautions needed for ordinary functions.

Assuming all went well, it is time to print out the hash reference returned by the web service. Cleverly, the hash is printed out in key-value pairs that parse_input() can understand.

Listing 6: perl journal client, pt. 5

    60	#---------------------------#
    61	# subs                      #
    62	#---------------------------#
    63	# API-implementation
    64	sub get_entry {
    65	  my ($c, $argv, $in) = @_;
    66
    67	  my $id = $argv->[0] || die "get_entry requires an ID\n";
    68	  my $ret = $c->get_entry($id);
    69
    70	  return if had_transport_error($ret);
    71
    72	  if (my $hr = $ret->result) {
    73	    while (my ($k, $v) = each %{$hr}) {
    74	      print "$k: $v\n";
    75	    }
    76	    return 1;
    77
    78	  } else {
    79	    warn "Couldn't find journal: $id\n";
    80	    return;
    81	  }
    82	}
    83

Listing 7 shows how the API method get_entries is called. Again arguments are pulled off of the command line and results are printed in a key-value pairs.

Listing 7: perl journal client, pt. 6

    84	sub list_entries {
    85	  my ($c, $argv, $in) = @_;
    86
    87	  my ($uid, $limit) = @{$argv};
    88
    89	  $uid ||= UID;
    90
    91	  my $ret = $c->get_entries($uid, $limit);
    92
    93	  return if had_transport_error($ret);
    94
    95	  my $ar = $ret->result;
    96	  for my $row (@{$ar}) {
    97	    while (my ($k,$v) = each %{$row}) {
    98	      print "$k: $v\n";
    99	    }
   100	    print "\n";
   101	  }
   102
   103	  return 1;
   104	}
   105

Listing 9 shows how more a complex form is submit to the web service. Creating a new jounral entry requires a subject line consisting of one or more words and perhaps several paragraphs of text. This kind of data is a poor fit for the command line, so the user instead submits a document to the script like the following:

Listing 8: Example log message

subject: Cats are Crazy

body: Cats are driving me crazy! Why do they lick the butter?!
<p>My cat was biting the business end of my laptop's power cable
that was still <b>plugged-in</b>
<p>WHAT'S WRONG WITH THESE BEASTIES?!

Fortunately for this subroutine, parse_input has already done the heavy lifting of breaking apart this document so now it can feed the appropriate parts to the web service. On success, the SOAP API returns the entry ID of the newly created journal -- a helpful detail in case you need to immediately make changes to the entry.

Listing 9: perl journal client, pt. 7

   106	sub add_entry {
   107	  my ($c, $argv, $in) = @_;
   108
   109	  my $ret;
   110
   111	  if (keys %{$in} > 1) {
   112	    # expect 'subject' and 'body' here
   113	    $ret = $c->add_entry($in->{subject},
   114				 $in->{body});
   115	  } else {
   116	    $ret = $c->add_entry( "random thought #$$", $in->{all} );
   117	  }
   118
   119	  return if had_transport_error($ret);
   120
   121	  print "add_entry got articleID: ", $ret->result, "\n";
   122
   123	  return $ret->result;
   124	}
   125

Listing 10 uses much of same techniques shown in Listing 9. Here we see that the SOAP API can take named parameters. Actually, SOAP doesn't know anything about named parameters, it just sees a list of values, just like Perl does.

Listing 10: perl journal client, pt. 8

   126	sub modify_entry {
   127	  my ($c, $argv, $in) = @_;
   128
   129	  my $ret = $c->modify_entry($in->{id},
   130				     subject => $in->{subject},
   131				     body => $in->{body}
   132				    );
   133
   134	  return if had_transport_error($ret);
   135
   136	  if ($ret->result) {
   137	    return 1;
   138	  } else {
   139	    warn "modify_entry appears to have failed\n";
   140	    return;
   141	  }
   142	}
   143

Listing 11 shows delete_entry(). Notice that it doesn't prompt the user for confirmation. Perhaps a front-end that calls this script could do that? (hint, hint).

Listing 11: perl journal client, pt. 9

   144	sub delete_entry {
   145	  my ($c, $argv, $in) = @_;
   146
   147	  my ($id) = $argv->[0] || die "delete_entry requires an ID\n";
   148	  my $ret = $c->delete_entry($id);
   149
   150	  return if had_transport_error($ret);
   151
   152	  return $ret->result;
   153	}
   154

Listing 12 demostrates the latest addition to the use.perl.org journaling API, get_uid_from_nickname(). What's the point of this method? Let's say that you want to read other poeple's journals from your favorite editor. The only thing stopping you is that you probably don't recall TorgoX's or Gnat's User ID. With this function, that problem is quickly remedied.

Listing 12: perl journal client, pt. 10

   155	sub whois {
   156	  my ($c, $argv, $in) = @_;
   157
   158	  my ($nick) = $argv->[0] || die "whois requires a nickname\n";
   159	  my $ret    = $c->get_uid_from_nickname($nick);
   160
   161	  return if had_transport_error($ret);
   162
   163	  if ($ret->result) {
   164	    print "$nick has the UID ", $ret->result, "\n";
   165	  } else {
   166	    print "Can't find the UID for $nick\n";
   167	  }
   168
   169	  return 1;
   170	}
   171

Listing 13 has three subroutines that have nothing to do with SOAP. The first subroutine shown is parse_input() and it doesn't do anything fancy. It's an example of Practical Extraction. As was mentioned, Chris Nandor supplied bakeCookie, which makes a weird string of the user's credentials. While probably not cryptographically secure, it's not all that bad either. The last routine works on the Response object from SOAP::Lite and it checks for transmission errors.

Listing 13: perl journal client, pt. 11

   172	#-------------------#
   173	# Utility functions #
   174	#-------------------#
   175
   176	# parse STDIN of colon terminated attributes
   177	sub parse_input {
   178	  my %record;
   179	  my $last_field = 'all';
   180	  while (<STDIN>) {
   181	    if (/^(\w+): (.*)/) {
   182	      $last_field = $1;
   183	      $record{$last_field} = $2;
   184	    } else {
   185	      $record{$last_field} .= $_;
   186	    }
   187	  }
   188
   189	  return \%record;
   190	}
   191
   192	# Thanks Pudge
   193	sub bakeUserCookie {
   194	    my($uid, $passwd) = @_;
   195	    my $cookie = $uid . '::' . md5_hex($passwd);
   196	    $cookie =~ s/(.)/sprintf("%%%02x", ord($1))/ge;
   197	    $cookie =~ s/%/%25/g;
   198	    return $cookie;
   199	}
   200
   201	sub had_transport_error {
   202	  my ($ret) = @_;
   203
   204	  if ($ret->fault) {
   205	    warn "Oops: ", $ret->faultstring, "\n";
   206	    return 1;
   207	  }
   208
   209	  return;
   210	}

That's the command line perl script. While functional, it's user interface is a bit terse for direct use. It's utility is apparent when married to a programmer's editor like emacs, vi or BBEdit.

And In The Lisp-ness Bind Them

I'll come out of the closet: I've been using emacs for four years now. At first, I just used it recreationally for quick editing jobs. As my addiction deepened, I learned about syntax-highlighting, controlling windows, symbol completion, auto-indenting, mail-mode, cperl-mode and perldb. I didn't know I had a problem until I found the doctor (neé Eliza) program that ships with emacs. Rather than admit that I need help, I starting tinkering with creating my own extensions to emacs and that lead to the subject of this article.

The perl client has been dealt with above and now it's time to look at that crazy, prefix, functional code that is emacs lisp, the macro language of emacs. I'm not an expert at lisp, but that's not needed here. All the lisp that I (and you) need to know to make many emacs extensions can be grasp from the lisp below. For a more complete reference on emacs lisp, check out O'Reilly's Writing GNU Emacs Extensions or the ever-verbose online help in emacs itself.

The strategy I used to create this emacs extension is very simple. Since I don't know lisp (and lisp isn't trivial to pick up), write just enough lisp to scrap data out of emacs and shell out to the perl script for the real work. It's almost as if I'm treating emacs like a web browser (yes I know emacs already has a real web browser and spreadsheet program).

Listing 15 is the start of a lisp file that I've labeled use.perl.el on my system. To load this file when emacs start, add the following line to your .emacs file.

Listing 14: loading use.perl.el at emacs start-up

   (load-file "/path/to/use.perl.el")

The lisp begins by creating a global variable to hold the full path to the directory that holds the symlinks to the perl SOAP client (whew!). The defvar function allows variables to be overriden by calling packages, if needed. Here, defvar could be replaced safely with the humble setq.

Listing 15: Emacs LISP module, pt. 1

     7	(defvar progpath
     8	        "/path/to/journal/client/directory"
     9		"use_perl_journal: Default path to switchbox perl script"
    10	)
    11
    12	(defun get-entry (n)
    13	   "Get journal entry from use.perl.org"
    14	   (interactive "sJournal ID: ")
    15	   (setq buffer (generate-new-buffer "*use_perl_journal*"))
    16	   (switch-to-buffer buffer)
    17	   (setq cmd (concat progpath (concat "/get_entry " n)))
    18	   (shell-command-on-region (point-min) (point-max) cmd 1 nil nil)
    19	)
    20

Line 12 begins a function called get-entry that prompts a user for an entry ID to fetch. All of these user-defined functions will have keystrokes associated with them later in this file. Only one parameter is required by get-entry, the entry ID represented by the parameter n. The double quoted string is the documentation string that is used by the emacs help system. The next procedure is interactive and it tells the emacs lisp interpreter that this function can be called interactively (via Meta-x). interactive is also used to create prompts for function arguments. Somewhat like printf, there are special format characters that precede the prompt string that indicate how the user-data should be stored. In this case, I use 's' for 'string'. The data type isn't all that important to me since Perl will Do The Right Thing later. The full list of interactive format strings can be found in emacs with M-x describe-function interactive.

If you're still with me, you'll be happy to know that the lisp code gets easier from here. The result of this function will be to populate a buffer with the record of the desired entry. Since you probably don't want to overwrite your current buffer, this function creates a new buffer with the name "*use_perl_journal* and changes to it on line 16.

Line 17 constructs string that will invoke our perl client with the appropriate command line argument. Lisp's concatenation operator isn't as terse as perl's but at least it works. Because we want the perl script to call it's get_entry() subroutine (take another look at Listing 2) we need to invoke the script through the proper symlink. Line 18 pipes the output from the executed perl script into the current buffer.

Shazam!

The path from emacs to use.perl.org is now complete. The rest of this document consists of devilish details.

Listing 18 shows the function list_entries which requires two paramenters, UserID and the number of entries to fetch. Notice on line 32 that multiple prompts are separated by a newline and can be given in one string. Each prompt is prefixed with an interactive format code. The rest of the functions are merely variations on a common theme, except for save-entry and modify-entry which use the widen function to grab all the characters in the current buffer and pass them through standard input to the perl script.

Listing 16: Emacs LISP module, pt. 2

    21	(defun list-entries (uid limit)
    22	   "Get jounral entries"
    23	   (interactive "sUser ID: \nsLimit: ")
    24	   (setq buffer (generate-new-buffer "*use_perl:list_entries*"))
    25	   (switch-to-buffer buffer)
    26	   (setq cmd (
    27		      concat progpath
    28			     (
    29			      concat "/list_entries "
    30				  (
    31				   concat uid (concat " " limit)
    32				  )
    33			      )
    34		     )
    35	   )
    36
    37	   (shell-command-on-region (point-min) (point-max) cmd 1 nil nil)
    38	)
    39
    40	(defun save-entry()
    41	   "Add journal entry"
    42	   (interactive)
    43	   (setq cmd (concat progpath "/add_entry"))
    44 ;; grab the whole buffer!
    45	   (widen)
    46	   (shell-command-on-region (point-min) (point-max) cmd)
    47	)
    48
    49	(defun modify-entry ()
    50	  "Modify an entry"
    51	  (interactive)
    52	  (setq cmd (concat progpath "/modify_entry"))
    53	  (widen)
    54	  (shell-command-on-region (point-min) (point-max) cmd)
    55	)
    56
    57	(defun delete-entry (jid)
    58	  "Delete a journal entry"
    59	  (interactive "nEntry ID: ")
    60	  (setq cmd (concat progpath
    61			    (concat "/delete_entry"
    62				    (concat " " jid)
    63			    )
    64	            )
    65	  )
    66	  (shell-command-on-region (point-min) (point-max) cmd 1 nil nil)
    67	)
    68
    69	(defun whois (nick)
    70	  "Look the UID of a user by his nickname"
    71	  (interactive "sNickname: ")
    72    (setq buffer (generate-new-buffer "*use_perl:whois*"))
    73    (switch-to-buffer buffer)
    74	  (setq cmd (concat progpath (concat "/whois" (concat " " nick))))
    75	  (shell-command-on-region (point-min) (point-max) cmd 1 nil nil)
    76	)

Listing 19 shows how to bind keystrokes to function names. The key sequence is "Control-x t" followed by an easily remembered letter.

Listing 17: perl journal client, pt. 3

    77	(global-set-key "\C-xtl" `list-entries)
    78	(global-set-key "\C-xtg" `get-entry)
    79	(global-set-key "\C-xts" `save-entry)
    80	(global-set-key "\C-xtm" `modify-entry)
    81	(global-set-key "\C-xtd" `delete-entry)
    82	(global-set-key "\C-xtw" `whois)

Perhaps you are wondering: Why CTRL-x t? A year ago, I wanted to designed a content management system for Taskboy.com. The twist here was that I want to maintain all the pages from emacs and I used a similiar perl client to make the web service calls. Although that project didn't pan out (I found that Taskboy's needs were met better with rsync) my discussions with Chris Nandor about Taskboy eventually lead to the journaling service on use.perl.org (he did the heavily lifting while I drank beer).

Now How Much Would You Pay?

You can find both the perl client, the lisp module and this article here. This client works, but there is much room for improvement. No doubt users of other editors will have different solutions to journaling remotely, but this is MWTDI (My Way To Do It).

Exclesior!

About the Author

The dessicated, withered husk that answers to the name Joe Johnston spends his days in filth and delerium. In rare moments of lucidity, he journals his madness here on use.perl.org. Learn more about this gentle, retiring creature of Perl at Taskboy.


Woohoo!

jdavidb on 2002-10-25T13:01:13

Joe, you are my new hero of darkness! Thanks for helping me grow deeper in my understanding of dark magics like SOAP. I've been looking for just such an article.

Hey!

jordan on 2002-10-25T14:00:35

Maybe this should be reworked to genericize it to work for all Slash, sites, like Slashdot, for example and submit this to the attention of the Slashdot editors.

I know a lot of people there would appreciate it, as well.

Re:Hey!

jordan on 2002-10-25T14:06:02

Hmmm... Reading this more carefully, it appears that this SOAP access may be a 'local' mod to Slash supported by pudge.

Well, if that's the case, then maybe we should just use the heck out of this, show it off, and encourage the Slashdot editors to get in sync with pudge's good work.

Re:Hey!

pudge on 2002-10-25T14:13:50

Well, I *am* a Slashdot editor. :-)

SOAP is not ready to be used on a site such as Slashdot. When we have more time to button it down and do more to prevent abuse, then we'll see.

Re:Hey!

jdavidb on 2002-10-25T16:04:49

Meanwhile I think someone who is interested could turn WWW::UsePerl::Journal into a general Slash client module.

Re:Hey!

russell on 2002-10-28T11:03:12

I've been meaning to do this, but haven't had the necessary tuits. Patches welcome. I've also been meaning to move to the SOAP interface, so that the module doesn't break every time the HTML changes. When I digest this article, I think I'll try out SOAP.

Xemacs wonkyness

Fletch on 2002-10-25T15:12:16

My xemacs is griping that shell-command-on-region doesn't take as many arguments as you're trying to pass it. Trimming off the last nil makes it happy, though.

jjohn++ jjohn++ jjohn++

Re:Xemacs wonkyness

Fletch on 2002-10-25T15:51:41

Share and enjoy.

(defun new-journal (subj)                 
  (interactive "sSubject: ")
  (let ((buffer (generate-new-buffer "*use_perl:new_entry*")))
    (goto-char (point-min))
    (switch-to-buffer buffer)
    (insert-string (concat "subject: " subj "\nbody: <p>\n\n</p>") buffer)
    (goto-char (+ (point-min) 20 (length subj)))))

(global-set-key "\C-xtn" `new-journal)

Re:Xemacs wonkyness

mako132 on 2002-10-25T20:00:11

mee too - xemacs 21.1.14. So I deleted the last "nil" parameter on every instance of shell-command-on-region in the code.

I'm able to post, list...didn't seem to able to delete, but I haven't investigated why yet.

Harumph!

hexmode on 2002-10-25T15:27:21

Why not use the XML-RPC APIs that are out there so that you could leverage all the clients written for, say, the Blogger API and the metaWeblogAPI? (For those heathen who don't like to use Emacs.) Or you could just use a SOAP version of the XML-RPC API if you wanted.

Seriously, though, this is a great addition.

For future reference:

Emacs package to update weblogs/journals using XML-RPC APIs.

SOAP.el (to avoid forking off a call from emacs):

The more mature xml-rpc.el.

Re:Harumph!

pudge on 2002-10-25T15:35:32

Which APIs? I looked at the Blogger API and was unimpressed. There's no standard out there that I could find.

Re:Harumph!

gav on 2002-10-25T15:49:05

There is the MetaWeblog API which is supported in Manilla and MovableType (and probably others). This is much more useful than the Blogger API.

On a somewhat related note, Net::Blogger does a great job, supporting Blogger, Manilla, MovableType, Radio, Slash, and Userland.

Re:Harumph!

hexmode on 2002-10-25T16:15:18

metaWeblog API allows you to do more than the Blogger API. Second, don't trust the offical docs on the Blogger API. Simon Kittle has a more complete set of docs. I know, its annoying.

Still, though the blogger API may not be impressive, it (along with the metaWeblog API) maps pretty closely to the API you've specified.

  • add_entry ~ blogger.newPost or metaWeblog.newPost (for titles/subjects)
  • modify_entry ~ blogger.editPost or metaWeblog.editPost
  • delete_entry ~ blogger.deletePost
  • get_entry ~ metaWeblog.getPost (stuff the struct with the info you want, map as much of it as possible to RSS)
  • get_entries ~ blogger.getRecentPosts

I know this probably isn't a perfect fit, but it gives you 95% of the functionality you have here and the added benefit that there are several clients already written for the API.

Re:Harumph!

pudge on 2002-10-25T21:38:56

But it doesn't support everything I need, it doesn't fully apply in what it does have, and this has been implemented for many months and a lot of people rely on it. So it isn't changing. :-)

Re:Harumph!

hexmode on 2002-10-26T02:46:55

I never suggested that you should change it. That would be silly, especially since you have other people using it. I appreciate the work done on this -- it gives me some great ideas.

Still, as far as I can tell, there is only one missing call (get_uid_from_nickname) and there's no reason you couldn't bolt that on somewhere. (But your api is better with respect to authentication.)

Would it be possible to get some introspection or even an availableMethods call (like the MT API has)?

Re:Harumph!

pudge on 2002-10-26T04:38:16

At some point, perhaps. The SOAP development is on hold while we work on other things. We hope to come back to it.

Re:Harumph!

russell on 2002-10-28T11:05:09

WWW::UsePerl::Journal has this function, so you could just use that in the meantime. I'd be better to see it within the SOAP interface though...

Re:Harumph!

pudge on 2002-10-30T04:47:47

If you mean get_uid_from_nickname, it is in the SOAP interface now.

bakeUserCookie()

pne on 2002-10-25T16:08:24

That reminds me... why are user IDs URI-encoded twice? It seems bizarre to me to replace, say, '@' with '%40' and then replace the '%' with '%25', leading to '%2540'. A bit like '&amp;eacute;' for 'é'.

Code bug at the beginning that couldn't be changed because of backwards compatibility?

Re:bakeUserCookie()

pudge on 2002-10-30T04:48:17

Yeah, it is just legacy.

Use w3m!

mary.poppins on 2002-10-26T12:15:21

The fantastical web browser w3m invokes your $EDITOR to edit textareas. Really! Obviously, this doesn't give you nearly as much flexibility as a fully general journal submission API. However, it *does* at least let you use your favorite editor, the same way any (reasonable) mail or news client does.

Oh, and w3m displays images in xterms. Really. I kid you not -- it is really wild. It even works inside screen, and over ssh connections!

Re:Use w3m!

pne on 2002-10-28T13:18:21

Lynx will let you use an external editor as well, though you have to invoke it explicitly. (^X^E IIRC.)

Re:Use w3m!

mary.poppins on 2002-10-28T19:30:37

Cool! When do we get the feature in the X-based browsers? :)