"permanent" backup solution

Started Feb 5, 2013 | Discussions thread
GideonW Contributing Member • Posts: 822
Re: debated often, I use hard drives

kelpdiver wrote:

rio911 wrote:

I would have to say my main requirement would be to back it up and forget about it. All this checksumming monthly or yearly business, I would like to avoid it.

the danger you face here is data corruption via silent bit rot. You think you have a good data file, and the thumbnail may even display correctly, but when you open it up, you lose one 8x8 pixel block and your image has that offset somewhere in the middle. You can't just trust that you wrote it once and it's fine forever.

Because of this I use a self-built NAS with the ZFS filesystem as my main backup solution. ZFS is a Unix/Linux filesystem that emphasizes data integrity. It stores a checksum of each data-block (at a location away from the data-block itself) and with every read-operation, the integrity of the data is verified using this checksum. When you add a level of redundancy (think RAID, but handled completely by the filesystem), any checksum errors can be fixed on-the-fly by using the redundant information.

This means that I have a system that mimics RAID-5 in most ways, with the automatic checksumming and verification in place. If bit rot occurs, the error is fixed. The RAID-like nature of the setup means that a single disk can fail completely and I can replace it without data loss. In addition, I could replace my disks with higher capacity models one by one and rebuild the array after each replacement and end up with a higher capacity volume at the end.

(I use an off-site external harddisk as well for backups of course. But my NAS/server machine has the highest level of confidence when it comes to data integrity.)

Post (hide subjects) Posted by
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark MMy threads
Color scheme? Blue / Yellow