All the Perl that's Practical to Extract and Report

Google File System

acme on 2004-04-27T22:08:57

It took me a while to understand how important the Google file system (PDF) is to Google. Has anyone reimplemented the system they describe? Anyone interested?

Update: Livejournal's MogileFS looks interesting.

not as important as you might think

perrin on 2004-04-28T18:00:36

People have commented about this on Slashdot saying that they use this file system for internal work, not for the web site. There's an example here.

You have to take Slashdot comments with a pound of salt, but this sounds reasonable.

Re:not as important as you might think

iburrell on 2004-04-29T03:46:29

It sounds like they use it for the production of the indexes. I think that is pretty important.

Re:not as important as you might think

acme on 2004-04-29T09:22:17

We've seen at work that large storage systems are very expensive and quite fragile. That's mostly why I was interested in something cheap (commodity hardware) and very reliable. It's not for everyone, of course.

Re:not as important as you might think

perrin on 2004-04-29T15:53:22

It sounds like a good system to me. I just meant that it's probably not used for what most people assume it is used for, i.e. directly running the Google site. A distributed system for storing large data sets does sound valuable.