Windows 10 Storage spaces / Mirror / ReFS

[primeshooter]

Im basically building from scratch a PC, (same case etc, but nearly everything else new including PSU) new mb etc but using some old hdd and bought x2 new ones. I planned to intially start off using the new x2 10TB in a mirror in the tower I have using ReFS but just noticed MS only have this option (ReFS) in workstation and i and currently running home!

Spec will be fairly high level - ryzen 9 3.8ghz 12 core, 128gb ddr4 3600mhz ram, 1070nvidia gtx 8gb gpu, 1tb samsung evo 970 m.2 card (win10 going here) and a 1200watt psu.

Im going to use some old drives (only 3 years old) and eventually upgrade to larger ones (x2 6tb and x2 2tb and a 500gb ssd currently owned) but like I mentioned ive bought x2 seagate ironwolf nas drives (10tb) to add into the setup now i have more sata ports. People keep saying raid isnt a backup - something i dont understand because it is surely, as long as you have a copy offsite too? Considering this and raid ready to find and fix any corruption i hope ill have a good setup. Ps using the excellent dark base pro 900.

What do you recommend, do I just shell out for the workstation version to get ReFS or another option? I like the sound of the ReFS option as it will find and fix / prevent corruption being copied across the drives. I then plan to replace the other drives with 10TB or more, mirroring of course. I have no real experience with ReFS or storage spaces or proper backups (been really bad for this and manually copy stuff) but i want to get better. Because i have a huge pc that can take loads of drives i want to keep it all in here for now. Advice re ReFs and backups locally first?

 

[Sean Nelson]

People keep saying raid isnt a backup - something i dont understand because it is surely, as long as you have a copy offsite too?

RAID (in a configuration that includes redundancy) isn't a backup because it doesn't protect against a lot of data risks. For example, if you accidentally delete a file, or if ransomware encrypts your data files, a mirrored RAID set will obediently delete or encrypt the files on all of the mirror copies instantly. Voila, your data is gone.

A true backup is a copy of the data which is offline, where it can't be screwed around with.

If you need RAID for performance reasons, you ought to think about staging your more frequently used files on an SSD, which is orders of magnitude faster than even the fastest HDD-based RAID.

Otherwise, my own opinion is that for most typical personal use RAID is an unnecessary bother. You need a true offline backup anyway, so the only thing that RAID really buys you is uptime while you replace a failed disk. Considering that in 30 years of continuous home computer use I've never had a disk fail on me (and that includes all my offline backup drives) I'm willing to live with the v-e-r-y occasional period of downtime where I need to restore my data from a backup, should a drive ever fail.

As far as ReFS goes, my same comments regarding drive reliability and the need for backup no matter what you're doing apply. ReFS is a new file system which is still in a niche within the Windows universe. I would be concerned that you might run into issues with it that you wouldn't with NTFS, which has been around for ages and is pretty rock-solid in terms of reliability. There aren't a whole lot of beta testers running ReFS to detect problems with weird configurations or software patches, whereas anything that affects NTFS is going to be known far and wide very quickly.

ReFS has some nice reliability features, but again for personal use my experience is that NTFS has never let me down so I'm not particularly tempted to jump off a proven horse.

 

[primeshooter]

People keep saying raid isnt a backup - something i dont understand because it is surely, as long as you have a copy offsite too?

RAID (in a configuration that includes redundancy) isn't a backup because it doesn't protect against a lot of data risks. For example, if you accidentally delete a file, or if ransomware encrypts your data files, a mirrored RAID set will obediently delete or encrypt the files on all of the mirror copies instantly. Voila, your data is gone.

A true backup is a copy of the data which is offline, where it can't be screwed around with.

If you need RAID for performance reasons, you ought to think about staging your more frequently used files on an SSD, which is orders of magnitude faster than even the fastest HDD-based RAID.

Otherwise, my own opinion is that for most typical personal use RAID is an unnecessary bother. You need a true offline backup anyway, so the only thing that RAID really buys you is uptime while you replace a failed disk. Considering that in 30 years of continuous home computer use I've never had a disk fail on me (and that includes all my offline backup drives) I'm willing to live with the v-e-r-y occasional period of downtime where I need to restore my data from a backup, should a drive ever fail.

As far as ReFS goes, my same comments regarding drive reliability and the need for backup no matter what you're doing apply. ReFS is a new file system which is still in a niche within the Windows universe. I would be concerned that you might run into issues with it that you wouldn't with NTFS, which has been around for ages and is pretty rock-solid in terms of reliability. There aren't a whole lot of beta testers running ReFS to detect problems with weird configurations or software patches, whereas anything that affects NTFS is going to be known far and wide very quickly.

ReFS has some nice reliability features, but again for personal use my experience is that NTFS has never let me down so I'm not particularly tempted to jump off a proven horse.

Thanks for the input. I had the odd .nef file corrupt over the years so wondered if ReFS would prevent this going forward...

Would you agree i should consider just using the second 10tb drive as a backup drive without raid then? If so which software backs up to another drive regularly in the background and is safe? I could then buy another large external as offline copy. Should i use storage spaces for example and make the 2nd disk for that?

A lot of thinking to do here!

 

[primeshooter]

People keep saying raid isnt a backup - something i dont understand because it is surely, as long as you have a copy offsite too?

RAID (in a configuration that includes redundancy) isn't a backup because it doesn't protect against a lot of data risks. For example, if you accidentally delete a file, or if ransomware encrypts your data files, a mirrored RAID set will obediently delete or encrypt the files on all of the mirror copies instantly. Voila, your data is gone.

A true backup is a copy of the data which is offline, where it can't be screwed around with.

If you need RAID for performance reasons, you ought to think about staging your more frequently used files on an SSD, which is orders of magnitude faster than even the fastest HDD-based RAID.

Otherwise, my own opinion is that for most typical personal use RAID is an unnecessary bother. You need a true offline backup anyway, so the only thing that RAID really buys you is uptime while you replace a failed disk. Considering that in 30 years of continuous home computer use I've never had a disk fail on me (and that includes all my offline backup drives) I'm willing to live with the v-e-r-y occasional period of downtime where I need to restore my data from a backup, should a drive ever fail.

As far as ReFS goes, my same comments regarding drive reliability and the need for backup no matter what you're doing apply. ReFS is a new file system which is still in a niche within the Windows universe. I would be concerned that you might run into issues with it that you wouldn't with NTFS, which has been around for ages and is pretty rock-solid in terms of reliability. There aren't a whole lot of beta testers running ReFS to detect problems with weird configurations or software patches, whereas anything that affects NTFS is going to be known far and wide very quickly.

ReFS has some nice reliability features, but again for personal use my experience is that NTFS has never let me down so I'm not particularly tempted to jump off a proven horse.

Thanks for the input. I had the odd .nef file corrupt over the years so wondered if ReFS would prevent this going forward...

Would you agree i should consider just using the second 10tb drive as a backup drive without raid then? If so which software backs up to another drive regularly in the background and is safe? I could then buy another large external as offline copy. Should i use storage spaces for example and make the 2nd disk for that?

A lot of thinking to do here!

Advice from photographylife is to use raid 1 as i thought then another ext hdd? The thought of doing this manually sounds like a pain.

 

[Morris0]

People keep saying raid isnt a backup - something i dont understand because it is surely, as long as you have a copy offsite too?

RAID (in a configuration that includes redundancy) isn't a backup because it doesn't protect against a lot of data risks. For example, if you accidentally delete a file, or if ransomware encrypts your data files, a mirrored RAID set will obediently delete or encrypt the files on all of the mirror copies instantly. Voila, your data is gone.

A true backup is a copy of the data which is offline, where it can't be screwed around with.

If you need RAID for performance reasons, you ought to think about staging your more frequently used files on an SSD, which is orders of magnitude faster than even the fastest HDD-based RAID.

Otherwise, my own opinion is that for most typical personal use RAID is an unnecessary bother. You need a true offline backup anyway, so the only thing that RAID really buys you is uptime while you replace a failed disk. Considering that in 30 years of continuous home computer use I've never had a disk fail on me (and that includes all my offline backup drives) I'm willing to live with the v-e-r-y occasional period of downtime where I need to restore my data from a backup, should a drive ever fail.

As far as ReFS goes, my same comments regarding drive reliability and the need for backup no matter what you're doing apply. ReFS is a new file system which is still in a niche within the Windows universe. I would be concerned that you might run into issues with it that you wouldn't with NTFS, which has been around for ages and is pretty rock-solid in terms of reliability. There aren't a whole lot of beta testers running ReFS to detect problems with weird configurations or software patches, whereas anything that affects NTFS is going to be known far and wide very quickly.

ReFS has some nice reliability features, but again for personal use my experience is that NTFS has never let me down so I'm not particularly tempted to jump off a proven horse.

Thanks for the input. I had the odd .nef file corrupt over the years so wondered if ReFS would prevent this going forward...

Would you agree i should consider just using the second 10tb drive as a backup drive without raid then? If so which software backs up to another drive regularly in the background and is safe? I could then buy another large external as offline copy. Should i use storage spaces for example and make the 2nd disk for that?

A lot of thinking to do here!

RAID dose not protect agents bitrot(aged data and a bit flip in the data). It will not throw an error on most controllers. Windows dose not have a fix for this risk. In the BSD environment there is RAIDZ1 and RAIDZ2 that can recover from bitrot if you regularly scrub data so that the problem is detected. As you are going to run windows the simplest path is to build a TrueNAS system for your storage. You will need a minimum of 3 dirsk for RAIDZ1 and 4 for RAIDZ2. Connect via 10-GB and the drives will be faster than local.

As Sean stated, you still need backup. Minimum of two copies of your backup's must be maintained, perforable to be on two separate media.

Morris

 

[primeshooter]

People keep saying raid isnt a backup - something i dont understand because it is surely, as long as you have a copy offsite too?

RAID (in a configuration that includes redundancy) isn't a backup because it doesn't protect against a lot of data risks. For example, if you accidentally delete a file, or if ransomware encrypts your data files, a mirrored RAID set will obediently delete or encrypt the files on all of the mirror copies instantly. Voila, your data is gone.

A true backup is a copy of the data which is offline, where it can't be screwed around with.

If you need RAID for performance reasons, you ought to think about staging your more frequently used files on an SSD, which is orders of magnitude faster than even the fastest HDD-based RAID.

Otherwise, my own opinion is that for most typical personal use RAID is an unnecessary bother. You need a true offline backup anyway, so the only thing that RAID really buys you is uptime while you replace a failed disk. Considering that in 30 years of continuous home computer use I've never had a disk fail on me (and that includes all my offline backup drives) I'm willing to live with the v-e-r-y occasional period of downtime where I need to restore my data from a backup, should a drive ever fail.

As far as ReFS goes, my same comments regarding drive reliability and the need for backup no matter what you're doing apply. ReFS is a new file system which is still in a niche within the Windows universe. I would be concerned that you might run into issues with it that you wouldn't with NTFS, which has been around for ages and is pretty rock-solid in terms of reliability. There aren't a whole lot of beta testers running ReFS to detect problems with weird configurations or software patches, whereas anything that affects NTFS is going to be known far and wide very quickly.

ReFS has some nice reliability features, but again for personal use my experience is that NTFS has never let me down so I'm not particularly tempted to jump off a proven horse.

Thanks for the input. I had the odd .nef file corrupt over the years so wondered if ReFS would prevent this going forward...

Would you agree i should consider just using the second 10tb drive as a backup drive without raid then? If so which software backs up to another drive regularly in the background and is safe? I could then buy another large external as offline copy. Should i use storage spaces for example and make the 2nd disk for that?

A lot of thinking to do here!

RAID dose not protect agents bitrot(aged data and a bit flip in the data). It will not throw an error on most controllers. Windows dose not have a fix for this risk. In the BSD environment there is RAIDZ1 and RAIDZ2 that can recover from bitrot if you regularly scrub data so that the problem is detected. As you are going to run windows the simplest path is to build a TrueNAS system for your storage. You will need a minimum of 3 dirsk for RAIDZ1 and 4 for RAIDZ2. Connect via 10-GB and the drives will be faster than local.

As Sean stated, you still need backup. Minimum of two copies of your backup's must be maintained, perforable to be on two separate media.

Morris

Hi. Yeah i know raid doesnt on its own. Its why im talking about ReFS.

 

[Morris0]

People keep saying raid isnt a backup - something i dont understand because it is surely, as long as you have a copy offsite too?

RAID (in a configuration that includes redundancy) isn't a backup because it doesn't protect against a lot of data risks. For example, if you accidentally delete a file, or if ransomware encrypts your data files, a mirrored RAID set will obediently delete or encrypt the files on all of the mirror copies instantly. Voila, your data is gone.

A true backup is a copy of the data which is offline, where it can't be screwed around with.

If you need RAID for performance reasons, you ought to think about staging your more frequently used files on an SSD, which is orders of magnitude faster than even the fastest HDD-based RAID.

Otherwise, my own opinion is that for most typical personal use RAID is an unnecessary bother. You need a true offline backup anyway, so the only thing that RAID really buys you is uptime while you replace a failed disk. Considering that in 30 years of continuous home computer use I've never had a disk fail on me (and that includes all my offline backup drives) I'm willing to live with the v-e-r-y occasional period of downtime where I need to restore my data from a backup, should a drive ever fail.

As far as ReFS goes, my same comments regarding drive reliability and the need for backup no matter what you're doing apply. ReFS is a new file system which is still in a niche within the Windows universe. I would be concerned that you might run into issues with it that you wouldn't with NTFS, which has been around for ages and is pretty rock-solid in terms of reliability. There aren't a whole lot of beta testers running ReFS to detect problems with weird configurations or software patches, whereas anything that affects NTFS is going to be known far and wide very quickly.

ReFS has some nice reliability features, but again for personal use my experience is that NTFS has never let me down so I'm not particularly tempted to jump off a proven horse.

Thanks for the input. I had the odd .nef file corrupt over the years so wondered if ReFS would prevent this going forward...

Would you agree i should consider just using the second 10tb drive as a backup drive without raid then? If so which software backs up to another drive regularly in the background and is safe? I could then buy another large external as offline copy. Should i use storage spaces for example and make the 2nd disk for that?

A lot of thinking to do here!

RAID dose not protect agents bitrot(aged data and a bit flip in the data). It will not throw an error on most controllers. Windows dose not have a fix for this risk. In the BSD environment there is RAIDZ1 and RAIDZ2 that can recover from bitrot if you regularly scrub data so that the problem is detected. As you are going to run windows the simplest path is to build a TrueNAS system for your storage. You will need a minimum of 3 dirsk for RAIDZ1 and 4 for RAIDZ2. Connect via 10-GB and the drives will be faster than local.

As Sean stated, you still need backup. Minimum of two copies of your backup's must be maintained, perforable to be on two separate media.

Morris

Hi. Yeah i know raid doesnt on its own. Its why im talking about ReFS.

If you implement your mirror in Storage Spaces with ReFS overplayed, the in theory if you enable the scrubbing features you would be OK. Then there was the failure of Storage Spaces under the Spring Update of Windows. Not only did Parity Spaces not mount, any space that filed was not repairable as you could not add a new drive. I'm glad I had backups!

Also, what do you do if you accidently overwrite or permanently delete something?

It's your data.

Morris

 

[primeshooter]

People keep saying raid isnt a backup - something i dont understand because it is surely, as long as you have a copy offsite too?

RAID (in a configuration that includes redundancy) isn't a backup because it doesn't protect against a lot of data risks. For example, if you accidentally delete a file, or if ransomware encrypts your data files, a mirrored RAID set will obediently delete or encrypt the files on all of the mirror copies instantly. Voila, your data is gone.

A true backup is a copy of the data which is offline, where it can't be screwed around with.

If you need RAID for performance reasons, you ought to think about staging your more frequently used files on an SSD, which is orders of magnitude faster than even the fastest HDD-based RAID.

Otherwise, my own opinion is that for most typical personal use RAID is an unnecessary bother. You need a true offline backup anyway, so the only thing that RAID really buys you is uptime while you replace a failed disk. Considering that in 30 years of continuous home computer use I've never had a disk fail on me (and that includes all my offline backup drives) I'm willing to live with the v-e-r-y occasional period of downtime where I need to restore my data from a backup, should a drive ever fail.

As far as ReFS goes, my same comments regarding drive reliability and the need for backup no matter what you're doing apply. ReFS is a new file system which is still in a niche within the Windows universe. I would be concerned that you might run into issues with it that you wouldn't with NTFS, which has been around for ages and is pretty rock-solid in terms of reliability. There aren't a whole lot of beta testers running ReFS to detect problems with weird configurations or software patches, whereas anything that affects NTFS is going to be known far and wide very quickly.

ReFS has some nice reliability features, but again for personal use my experience is that NTFS has never let me down so I'm not particularly tempted to jump off a proven horse.

Thanks for the input. I had the odd .nef file corrupt over the years so wondered if ReFS would prevent this going forward...

Would you agree i should consider just using the second 10tb drive as a backup drive without raid then? If so which software backs up to another drive regularly in the background and is safe? I could then buy another large external as offline copy. Should i use storage spaces for example and make the 2nd disk for that?

A lot of thinking to do here!

RAID dose not protect agents bitrot(aged data and a bit flip in the data). It will not throw an error on most controllers. Windows dose not have a fix for this risk. In the BSD environment there is RAIDZ1 and RAIDZ2 that can recover from bitrot if you regularly scrub data so that the problem is detected. As you are going to run windows the simplest path is to build a TrueNAS system for your storage. You will need a minimum of 3 dirsk for RAIDZ1 and 4 for RAIDZ2. Connect via 10-GB and the drives will be faster than local.

As Sean stated, you still need backup. Minimum of two copies of your backup's must be maintained, perforable to be on two separate media.

Morris

Hi. Yeah i know raid doesnt on its own. Its why im talking about ReFS.

If you implement your mirror in Storage Spaces with ReFS overplayed, the in theory if you enable the scrubbing features you would be OK. Then there was the failure of Storage Spaces under the Spring Update of Windows. Not only did Parity Spaces not mount, any space that filed was not repairable as you could not add a new drive. I'm glad I had backups!

Also, what do you do if you accidently overwrite or permanently delete something?

It's your data.

Morris

I don't mean raid on it's own, i'd keep offline backups too. I'm just talking about a raid mirror and has ReFS ability to repair itself.