Accessing Amazon S3 from the command line

polettix on 2008-08-31T17:49:21

I started playing with Amazon Simple Storage Service and acme's Net::Amazon::S3, finding both quite useful and easy to use.

The obvious next step has been re-inventing the wheel. Well, sort-of, because the type of wheel I had in mind wasn't anywhere I was able to look for. In particular, I was looking for some "operating-system command" a-la git that was able to support subcommands for common operations.

So, I concentrated on two sets of commands: the "pure" ones, that focus on Amazon's way of seeing data, and the "filesystem" ones, that support thinking S3 stuff as files with a Unix-like path.

Among the first group you can find the basic S3 functions like add, remove etc. The second group has more "convenient" commands like ls, cp, rm and mv, that try to do the right thing in the spirit of the corresponding Unix commands (e.g. cp can both send or get data, depending on the order of the parameters). I find this useful because the metaphor is somewhat "hardcoded" in me after years of using those commands.

If you're interested or just want to find something to start a flamewar, you can find it at http://repo.or.cz/w/s3.git. Enjoy!

Update: thanks to grink's tip below, I found Term::ShellUI that allowed me to add an interactive shell to call all of the commands, with on-line help! I love CPAN.


commands with subcommands

rjbs on 2008-08-31T21:56:14

I hate to be the guy* who just posts "use my code to write your code," but for writing this kind of "cmd subcmd --opt" program, have a look at App::Cmd. It takes away a lot of drudgery.

* this is not actually true

Re:commands with subcommands

polettix on 2008-08-31T23:18:54

No reason at all! I'll surely take a look. Anyway, I didn't anything particular in this case... so I'm probably in time to use it anyway, thanks. Just in case, I'm wondering about adding a "--interactive" to start some shell... do you have/know anything to do this as well? :D

Re:commands with subcommands

grink on 2008-08-31T23:57:09

I haven't had a chance to use it, but Term::ShellUI looks promising.

Re:commands with subcommands

polettix on 2008-09-01T01:27:34

Well, it doesn't simply look promising, it's actually awesome. I added the interactive mode and it seems to work... just right out of the box and with a minor refactoring of code.

Thanks!