Amazon Web Services

acme on 2006-09-10T06:23:06

It was quite short notice (and on a Saturday!) but Jeff Barr presented a talk on Amazon Web Services to London.pm yesterday. It was a good run through the many services that Amazon can provide to programmers. Many people were convinced to back photos up to S3 instead of tape and were intrigued by the compute cloud. Recently I have been saying that I'm mostly IO-bound, not CPU-bound, but I realise I've been wrong: with EC2 and S3 the balance changes. I have some ideas and I'll benchmark some interesting things later on. Thanks Jeff!


But how...

Alias on 2006-09-10T08:17:25

The question then becomes HOW to back up the photos...

Is there a simple usable program you/he recommend(ed)?

Re:But how...

acme on 2006-09-10T14:32:31

At the moment I'd say the tools aren't entirely there. I do quite like http://www.rjonna.com/ext/s3fox.php for inspecting S3 but otherwise it's not amazing. I just knocked up something which uses Net::Amazon::S3.

Re:But how...

Aristotle on 2006-09-11T00:37:55

I wonder whether Brackup’s usable yet… Brad’s not touched it since spring, which means either it works well enough to scratch his itch or he got sidetracked.

Re:But how...

binky on 2006-09-13T08:59:23

S3Backup (http://s3backup.blogspot.com/) is one possible answer, but only available for Linux and Windows.

Jungle Disk (http://www.jungledisk.com/) is another. My plan is to see if I can make that work with Adobe Lightroom's library feature, so I can search my archive locally, but pull the images from S3 if I need them.

- Neil.

or use rsnapshot

drhyde on 2006-09-12T17:08:05

Or, instead of paying Amazon, use rsnapshot, which also doesn't use tape. Tape bad.

Re:or use rsnapshot

Alias on 2006-09-16T11:35:18

The difference being that rsnapshot doesn't come with bundled enterprise grade disaster planning.

The main reason I'm interested in S3 is I'm a shit sysadmin who avoids having to control my own physical resources, and if Amazon trusts S3 enough to put their OWN data inside of it, then I'm willing to slap an "Enterprise-Grade Backup" mental sticker on it and trust it with my stuff too.

That's not to say I necesarily trust them to keep it always secret (although Brackup might go a bit far to prevent it in that regard) but I trust them to store it in a way which means if I get caught in another earthquake, I don't have to worry about backups or boxes.

For high value assets, that's well worth it.

Re:or use rsnapshot

drhyde on 2006-09-18T23:03:12

I used to be all paranoid about how safe my backups were. Then I realised that if my house burns down, I've got bigger things to worry about than the fact that I didn't have backups safely squirrelled away on another continent.

Anyway, everything important is backed up on the CPAN.