Storage strategy. RAID 1 or 5? Backup? Both?

Started Oct 28, 2012 | Questions
(unknown member) Forum Pro • Posts: 14,338
Re: Both imaging and incremental backup are needed

kseidman wrote:

I use both Windows 7 Backup and Filesync to several different external drives, all of which are now becoming full. So, I'm in the market for a couple of new 2TB external drives... and I'm not happy with the user reports I've been reading for any of these external drives.

Yes, I'm not thrilled with USB drive quality either. There are alternatives:

1. Look at the Lacie Quadra drives. A little more expensive but definitely much better build quality than the regular WD/Seagate offerings.

2. Buy bare SATA (internal) drives and a good quality enclosure. WD Greens are ideal IMO.

3. Buy bare SATA (internal) drives and a dock like the BlacX. However, I do worry about the reliability of the SATA connectors which are not designed for repeated insertions.

kelpdiver Veteran Member • Posts: 3,484
Re: For me, neither.

Michael Firstlight wrote:

You are right, for what you need the extra speed would be overkill. But there are those of us that do disk and CPU intensive work, like myself that constantly processes 36MP RAW files and stitched images that are over 1 Gigapixel each in size - waiting 5 minutes for a single Photoshop action isn't fun - so some of us need that kind of speed and are not just worshipers - it comes down to time and time is profit

If that's the case, shouldn't you have replaced thos X25s already?  Single drives exceed the performance (esp on writes) of your R0 pairing.

kelpdiver Veteran Member • Posts: 3,484
Re: RAID cannot replace "real" backup

Osvaldo Cristo wrote:

RAID cannot replace "real" backup as it can provide only fault tolerance. It will not provide you any kind of protection for software or operator error.

Though snapshots do not actually require RAID, most (acceptable) RAID solutions offer this capability.  ZFS lets me create a snapshot in seconds to protect against software/operator error, and this is trivially scripted to happen on a automated basis.

If something gets deleted and I later have regrets, I can retrieve it instantly.

Michael Firstlight Veteran Member • Posts: 3,445
Re: For me, neither.

Definitely!  I am planning to, but waiting for Haswell next Spring to replace my motherboard as I need a new board with a sufficient number of SATA3 ports and I want a bunch of them. I'll move to a pair of Intel 500 series drives in RAID 0 for my system disk and double my RAM from 12GB. It will be worth it.  I OC my CPU as well to the max on an H100 cooler (I don't want to get into custom cooling and the H100 is pretty good for a contained OOTB liquid solution. I should be able to reuse that as well as many other components.  I try to get about 2-3 years out of each major Toc upgrrade.  Coming from anOCed i7 920 at 4GHz with the X25Ms to this new spec should be a major leap.  I'll keep my Sapphire 5850 4GB card; it seems good enough for the occasional video editing I do. Its all housed in an HAF932 tower.

Regards,
Mike

 Michael Firstlight's gear list:Michael Firstlight's gear list
Nikon D800 Nikon AF-S Nikkor 70-200mm f/2.8G ED VR Nikon AF-S Nikkor 24-70mm f/2.8G ED Nikon AF-S Micro-Nikkor 105mm f/2.8G IF-ED VR Sigma 10-20mm F3.5 EX DC HSM +28 more
GideonW Contributing Member • Posts: 822
Re: RAID cannot replace "real" backup

kelpdiver wrote:

Osvaldo Cristo wrote:

RAID cannot replace "real" backup as it can provide only fault tolerance. It will not provide you any kind of protection for software or operator error.

Though snapshots do not actually require RAID, most (acceptable) RAID solutions offer this capability. ZFS lets me create a snapshot in seconds to protect against software/operator error, and this is trivially scripted to happen on a automated basis.

If something gets deleted and I later have regrets, I can retrieve it instantly.


Yes, another ZFS user here on DPR! The future of storage lies with more powerful filesystems, because the current filesystems of Windows/Mac/Linux boxes are inflexible and can't detect file corruption. ZFS is a great step forward. Data-security through distributed checksums, raid-support, flexibility with upgrading the storage pool.

I have a self-built NAS that runs ZFS on 4x2 TB in RAID-Z1 (equivalent to RAID-5), so 6 TB net storage. It's fast (close to saturating a gigabit network) and reliable. Coupled with an external disk for offsite backup and all is well.

gkreth Veteran Member • Posts: 3,122
+1 for Free File Sync

malch wrote:

Anfy wrote:

Any thoughts on WD's own Smartware software, which - as I happen to read - in the last versions should let you to perform a complete drive/folders backup, not only of given files categories?

Personally, I think it's horrible. I tend to stick to programs I know and trust versus some crap that comes bundled.

For making disk images I like Macrium Reflect. There are lots of good File Sync packages. This is a good freeware program:

http://sourceforge.net/projects/freefilesync/

Just to add another thumbs-up: I use Free File Sync, and I quite like it.

Greg

Sean Nelson
Sean Nelson Forum Pro • Posts: 12,944
Re: For me, neither.

kelpdiver wrote:

If that's the case, shouldn't you have replaced thos X25s already? Single drives exceed the performance (esp on writes) of your R0 pairing.

Not for random writes they don't, especially small random writes. And writes to the system drive are normally small, random, and make up only a tiny proportion of the overall workload.

Jim Cockfield Forum Pro • Posts: 16,338
How's RAID Z1 working for you?

GideonW wrote:

kelpdiver wrote:

I have a self-built NAS that runs ZFS on 4x2 TB in RAID-Z1 (equivalent to RAID-5), so 6 TB net storage. It's fast (close to saturating a gigabit network) and reliable. Coupled with an external disk for offsite backup and all is well.

I bought an external eSATA attached drive enclosure and controller card supporting Port Multiplier features on sale a while back (no RAID features, I'd just use software RAID instead), that I still haven't bothered to setup, and I've been considering the available options for it.

For example, FreeNAS 8.3 now supports ZFS version 28 features and it's a very low resource requirement solution for a NAS server.

Raid Z1 and Raid 5 both scare me, because I've seen lots of "horror stories" about another drive failing after the rebuild starting when replacing a failed drive, resulting in the loss of the entire array.

Apparently, many modern drives just don't cope with RAID 5 (or, RAID Z1 if using ZFS) very well, as it's very common to have timeout issues with another drive being removed from an array during that rebuild process from what I can find out about those setups.

So, I've been wondering if a RAID Z2 config (equivalent to RAID 6) may be a better option, since it allows for more than one drive failure. Sure, you lose some of the available space with a given number of drives. But, reliability would be a more important consideration from my perspective.

So, any comments are welcome.

-- hide signature --

JimC
------

kelpdiver Veteran Member • Posts: 3,484
Re: For me, neither.

Sean Nelson wrote:

kelpdiver wrote:

If that's the case, shouldn't you have replaced thos X25s already? Single drives exceed the performance (esp on writes) of your R0 pairing.

Not for random writes they don't, especially small random writes. And writes to the system drive are normally small, random, and make up only a tiny proportion of the overall workload.

the X25 was much better at that than the competition....3 years ago.  Current drives smoke it.  And R0 doesn't increase IOPS anyway.  Two 830s (or 840 pros) would be a substantial improvement in performance.

kelpdiver Veteran Member • Posts: 3,484
Re: How's RAID Z1 working for you?

Jim Cockfield wrote:

Raid Z1 and Raid 5 both scare me, because I've seen lots of "horror stories" about another drive failing after the rebuild starting when replacing a failed drive, resulting in the loss of the entire array.

Apparently, many modern drives just don't cope with RAID 5 (or, RAID Z1 if using ZFS) very well, as it's very common to have timeout issues with another drive being removed from an array during that rebuild process from what I can find out about those setups.

So, I've been wondering if a RAID Z2 config (equivalent to RAID 6) may be a better option, since it allows for more than one drive failure. Sure, you lose some of the available space with a given number of drives. But, reliability would be a more important consideration from my perspective.

For the concerns you have, R6/Z2 doesn't actually change matters, and introduces a new point of failure - what if the two parity stripes are in conflict?  Which one is authoritative?   If you believe in how zfs solves the write hole problems and if you believe that weekly scrubbing will fix bad blocks and identify a silent failing drive, then Raid Z1 gets the job done.  Z2 just lets you suffer an outage without falling into degraded performance and running without a safety net.

The TLER issue has been around for a while.  I've never had to do a full rebuild, so I don't know how serious it really is.  Again, scrubbing would identify bad blocks in advance of a rebuild event.  You can mitigate it by biasing towards the drives that have settable parameters, or in the case of WD, the Red line of drives.  I have two as part of a Z filer and they seem to be working out fine, though they do command a premium over the EARX which also seem to be working fine.

How many drives will you put into this enclosure?  For 4 drives, raid 10 would be better than Z2.  While read performance is the same, write performance is much better with the mirrored setup.

Keyboard shortcuts:
FForum MMy threads