In my house there is a central media-server holding our collective digital music collection. When at home we mount the server over NFS, which works wonderfully. I copy music over to my laptop for coffee-shop work, but always wanted something a bit more transparent. Enter Fuse.pm.
What I've built is a caching filesystem specifically designed for this situation, and I named it MobileFS. When I listen to a song it is cached locally. Then if I later try to listen to the song but the NFS is unavailable the cached version is used. To construct this, I took Fuse.pm's example dummy tool as my initial code. The example mounts one directory on another, providing all of the basic filesystem manipulations (such as open, read, write, and various stat commands). It was very straightforward to insert code that additionally caches all the files that are opened, and if a file being read disappears then to fall back to the cache. In a lot of ways I actually designed this for my wife to use, so transparency of operation was important.
I had to solve a couple problems. The first was that the song I was listening to would skip a bit at the beginning when the copy-to-cache started. This was fixed by calling out to rsync with bandwidth limit instead of plain 'cp'. The second problem was that many mp3 players (xmms for example) read a chunk at the beginning and the end of the file to get the ID3 information and song length. So instead of immediately caching a file when it is opened, I instead delay until a block in the file a little ways in is read. About 15 seconds in on my 192bps songs.
This is also not too difficult. In the open function I look for mp3/ogg files, and then do:
$read_action->{$bare_file} = sub {
my ($file,$bufsize,$off) = @_;
if($off > 500000 && $off < 1000000) {
print STDERR "Read at offset $off triggering action...\n";
cache_file($bare_file);
delete $read_action->{$bare_file};
}
};
Where $read_action is a hashref to hold callbacks like this per-file, and the filename is $bare_file. You can see that I have some constant offsets in there, I got those through experimentation with my own music, and it is a somewhat relative measure depending on the bitrate of the file. Notice also that once we trigger the callback it deletes itself, stopping it from being triggered multiple times. In the read function I then do:
if($read_action->{$bare_file}) {
$read_action->{$bare_file}->($bare_file, $bufsize, $off);
}
One thing is missing, and I haven't coded it yet, which is that mp3/ogg files that are opened but never read in the given offset range leave their callbacks around filling up memory. Interestingly, as far as I can tell there is no 'close' function in FUSE, otherwise that would be a great place. I can't just delete the callback on the next file open in case more than one file is being accessed through this filesystem. Perhaps I'll add in some sort of timestamp and then just kill the old ones after some time.
In any case... it works! It was impressively easy to adapt the Fuse.pm example, as I said. I have now unmounted and remounted my NFS while in the middle of a cached song, and it transparently switches from the real source, to the cache, and back. Loading songs into the XMMS playlist doesn't trigger them being cached, but playing them does. Now I just need to code up a cron job that goes through and clears out unused things in the cache, perhaps based on a fixed cache size.
Future features? Well, one that comes immediately to mind is that, when I'm in the coffee shop and try to play a song that isn't cached, I could actually have a GUI popup asking me if I'd like to queue this song for (possibly slow) HTTP retrieval. Or I could just do it without asking. Another thing would be to pre-cache files, for a directory containing an entire album of songs, for example. Got any other ideas?
You can see the project page for MobileFS at
That's a cool use of Fuse. Fuse is a very useful technology with many potentially cool uses.
I've made my own use of Fuse for debugging purposes, but I used C++ because I started out with a C++-based Fuse-backend - LoggedFS, which seemed to be a good basis for that. As it turned out, it had a small problem that prevented it from being used on $HOME/.kde/, which required a lot of debugging on my part, until I found it and added the appropriate flag which fixed the problem.