Thursday, March 5, 2009

Tahoe-LAFS Distributed, Secure, Fault-Tolerant Filesystem

I have long been looking for a viable off-site backup solution for both my business and personal data. The business part is pretty easy. I only back up a couple hundred megabytes, and even allowing for weekly full backups + daily incrementals, I can keep a few months of backups available in a basic 4 GB volume from (For what it's worth, a shout-out to Duplicity here, which is what I use to manage the backups to It handles everything, including encryption and an rsync-like only-send-the-differences algorithm.)

Backing up my personal files, though, is an entirely different problem. Like most people, this includes a huge amount of media: over 100 GB of audio and more than 200 GB of photographs. Backing up to any of the popular online backup companies will either cost me a fortune per month or run the risk of crossing some fuzzy "acceptable use" line.

The other day I ran across the Tahoe-LAFS and this has me really excited. It is a distributed, secure and fault-tolerant filesystem. You create a node, point it at some disk space and join a grid. Everyone in the grid shares the pool of storage. Every file is broken up in many pieces and stored redundantly such than the loss of up to 70% of the nodes does not lose any data. (Redundancy and fault-tolerance values are adjustable, too, so you can tune it to your liking.)

While this doesn't solve the problem of a slow initial backup (I am limited by my own upstream bandwidth), I love the idea of not relying on a single company for access to my data, and knowing that what I store on the grid is opaque to anyone else. Disks are getting cheaper every month, so it doesn't seem too unreasonable to round up a bunch of people, buy a TB disk or two each and build a grid for everyone to use.

No comments:

Post a Comment

Raspberry Pi Zero W - Wireless Configuation

create the file under "boot" folder wpa_supplicant.conf country=GB ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev u...