Logical and economical storage of image archive

[jamesdak]

Alright I have just bit the bullet and purchased a new system to use for my photo/video imaging.

Now I want to finally get smart about my storage.

Currently I maintain one copy of the image file on an internal HD and another copy one of several external USB type drives.

I have purchased a Dell XPS 8500 which is coming with only one 2 TB HD. I plan on adding at least one other 2 TB HD and then running some sort of Raid to protect the data.

Can someone suggest how I best address a safe data storage solution? I am not adverse to putting several hundred more dollars into a solution.

I also have several older PCs sitting around the house. Various Pentium 4 machines running XP with smaller hard drives. So maybe some kind of NAS is an option as part of the solution? Two of the machines have jacked up OS due to teenagers doing who knows what on them. But I can reformat and reload XP. The other is the system I am replacing but I think the MB is dying. I keep losing the USB ports, have boot-up problems and the on board video flacked out for first time the other day.

Anyway, I'd really appreciate any advice on how to properly set up the system for storage. The new computer is coming with Windows 8 but I may go back to Windows 7 if I can't stand the new interface.

Thanks in advance...

 

[docvince]

if your willing to spend 200 bucks more buy an ssd drive for your boot drive, maybe 120gb then buy another 2tb hard drive same as the one that comes with dell xps and do raid 1 for your storage. Make sure that the second 2 tb drive is same size, brand and spec as the one that comes with dell to make a problem free raid 1. Then you're all set for years. This is only if you have a lot of image to archive.




Vince

 

[Chris Noble]

docvince wrote:

do raid 1 for your storage. Make sure that the second 2 tb drive is same size, brand and spec as the one that comes with dell to make a problem free raid 1. Then you're all set for years. This is only if you have a lot of image to archive.

Vince

RAID is not an archiving system. This comes up a lot in DPReview threads and has been thoroughly covered.

 

[jamesdak]

Ok, point is storing the data to keep is safe from corruption is important and what I am after. I see data integrity as a vital part of the archive. So, anything helpful to aid? My search of raid did not yield much...

 

[kcbeatty]

jamesdak wrote:

Alright I have just bit the bullet and purchased a new system to use for my photo/video imaging.

Now I want to finally get smart about my storage.

Currently I maintain one copy of the image file on an internal HD and another copy one of several external USB type drives.

I have purchased a Dell XPS 8500 which is coming with only one 2 TB HD. I plan on adding at least one other 2 TB HD and then running some sort of Raid to protect the data.

Actually your current backup plan is not bad. You need to save at least one good drive image stored offsite and one copy onsite. Then save data backups done every other day or so to portable hard disks, storing one copy offsite rotating weekly. You can substitute the data backups to use an online cloud system. I currently store my offsite backups to a good fireproof safe located in my house. The only problem with this is that the safe isn't water proof. I will be replacing that soon.

Offsite storage can be a real problem. I can tell you from first hand witnessing, safe deposit box's at banks are not safe from flooding. A lot of people learned that lesson the hard way in hurricanes katrina, Rita, and Sandy.

If your data is valuable to you, you need to develop a "Disaster Recovery Plan".

 

[MrScary]

Buy a NAS and put a 2TB or a 3TB HDD in it. Format it in the NAS box and use as JBOD.

When ready, put another 2TB or 3TB in the NAS box, format it and use as JBOD.

Internally in your Tower Case have a 1TB or 2TB HDD and put/store your images on them so you got EASY access every time you start your PC.

Download "Karen's Replicator" FREE and Backup your images with it to your NAS.

Very Easy, No Hassle, No problems. Don't knock it till you try it..

 

[Chris Noble]

jamesdak wrote:

Ok, point is storing the data to keep is safe from corruption is important and what I am after. I see data integrity as a vital part of the archive. So, anything helpful to aid? My search of raid did not yield much...

 

[malch]

Chris Noble wrote:

Archive tips:

1) One copy onsite, updated daily and automatically.

2) One copy off-site, swapped with the on-site copy every few months.

External USB drives are useful for this.

Very sensible.

I think building a NAS and/or implementing RAID will only burn a hole your pocket and get you into trouble.

I might consider a third copy depending on the value of the data. And swap the off-site copy a little frequently depending on how easy that is going to be in practice.

 

[jamesdak]

malch wrote:

Chris Noble wrote:

Archive tips:

1) One copy onsite, updated daily and automatically.

2) One copy off-site, swapped with the on-site copy every few months.

External USB drives are useful for this.

Very sensible.

I think building a NAS and/or implementing RAID will only burn a hole your pocket and get you into trouble.

I might consider a third copy depending on the value of the data. And swap the off-site copy a little frequently depending on how easy that is going to be in practice.

What if one of my existing old computers can be configured as a NAS. Then just remove the HD from my external My Book USB drives and consolidate them in the NAS. I'm asking this because two of my USB drives have already had the power supplies die on them despite not being used much.

 

[malch]

jamesdak wrote:

What if one of my existing old computers can be configured as a NAS. Then just remove the HD from my external My Book USB drives and consolidate them in the NAS. I'm asking this because two of my USB drives have already had the power supplies die on them despite not being used much.

Hmmm, I think I detect an itch that needs scratching

Hey, if you want to try building something based on FreeNAS or whatever, I certainly won't try to talk you out of that as a fun learning experience.

I don't think it's quickest, easiest solution to the immediate problem. But I bet you'll learn a lot.

Just one caution: a NAS isn't that useful if it isn't reasonably robust and reliable. If you're building one with a lot of components that are 5+ years old it might fall short in that regard. Just keep that in mind as you look at the items you're hoping to utilize.

 

[Scott Eaton]

Having hard drives in RAID 1 won't hurt anything and delivers immediate redundancy in case one drive fails. In the respect, it solves the problem - in the short term. It will not however guard against data corruption because the corruption will just be written immediatley across both drives. RAID 1 is just a mirror that occurs real time to protect against drive failure. It only works for archiving as long as the drives are present. When the drives fill up it's no longer usefull. Plus, if your drives are online in a RAID 1 all the time the are getting wear and tear, have the chance of getting deleted due to some OS glitch, and possible damage caused by lighting strikes, power surges, etc. So, that's why RAID isn't practical for home archiving.

Easiest and most convenient back-up is just to use an external drive where you can easily swap out the drive. When the drive fills up, label it and store it someplace dry and safe.

However, there is still chance of drive failure even if the drive is offline and sitting in a safe. Small as it is, it's still a possibility. In that case....use two drives with the same data on each. Not much of a conceptual leap there although some people make it out to be.

A big NAS box isn't a bad idea for back-ups, but has the same scary pitfalls when we talk about actual archiving. NAS boxes are typically online, running off a power supply, typically have poor fault notification, etc. 1 TB and 2TB drives are cheap right now. Nothing wrong with using a Blue Ray burner either, but I just don't trust the media for archiving and would still make duplicate copies if I did.

 

[Sean Nelson]

malch wrote:

I think building a NAS and/or implementing RAID will only burn a hole your pocket and get you into trouble.

I totally agree. My archiving system is very simple: everything is available instantly online on an individual 2TB in my system, and I have two external 2TB drives that I sync to it on a weekly basis (or more often if I have some special need). One of the two external drives is stored offsite and I swap it with my onsite drive once a month.

My daily system incremental backups include any changes made to the archive drive since the last offsite swap, and I don't delete files or videos from my camera memory cards until those incremental backups have been run. So I'll never loose any files if a hard drive crashes and if an unimaginable disaster should befall me the worst case is that I'll loose the last month's worth of files (which, in that kind of circumstance, will be the least of my worries).

The more complex you make your backup strategy the more likely it is to screw up. KISS (Keep It Simple, Stupid) is the best policy, IMHO.

 

[jamesdak]

malch wrote:

jamesdak wrote:

What if one of my existing old computers can be configured as a NAS. Then just remove the HD from my external My Book USB drives and consolidate them in the NAS. I'm asking this because two of my USB drives have already had the power supplies die on them despite not being used much.

Hmmm, I think I detect an itch that needs scratching

Hey, if you want to try building something based on FreeNAS or whatever, I certainly won't try to talk you out of that as a fun learning experience.

I don't think it's quickest, easiest solution to the immediate problem. But I bet you'll learn a lot.

Just one caution: a NAS isn't that useful if it isn't reasonably robust and reliable. If you're building one with a lot of components that are 5+ years old it might fall short in that regard. Just keep that in mind as you look at the items you're hoping to utilize.

Not an itch at all. Just looking at options for using what I got. Storage would not be such an issue if I had more time to scrub my digital files. But I'd rather shoot and PP what I want versus culling out all the bad stuff, LOL!!

"5+ years old" is my new stuff, LOL. My newest system was built in 2005.

 

[malch]

jamesdak wrote:
"5+ years old" is my new stuff, LOL. My newest system was built in 2005.

The reliability of items like power supplies and disk drives plummets pretty badly after 5 years. So building systems out of components that old is unlikely to work out well without a great deal of good luck.

P.S. I generally dispose of disks once they get to 5 years old. The small capacity, high power consumption, and poor reliability makes them more of a liability than an asset.

 

[JimPearce]

More than 1tb?

 

[jamesdak]

JimPearce wrote:

More than 1tb?

 

[BobSC]

malch wrote:

jamesdak wrote:
"5+ years old" is my new stuff, LOL. My newest system was built in 2005.

The reliability of items like power supplies and disk drives plummets pretty badly after 5 years. So building systems out of components that old is unlikely to work out well without a great deal of good luck.

P.S. I generally dispose of disks once they get to 5 years old. The small capacity, high power consumption, and poor reliability makes them more of a liability than an asset.

Depends on what they were in the first place. Our Dell business class machines seem to just keep running.

 

[BobSC]

If you can get a decent size drive on one of the old machines, then reformat and put XP on it, then get Retrospect, and use it to back up the new machine. You can configure it for incremental backups. Occasionally swap out the back up disk and store it at some other location, preferably in a different city.




This isn't perfect, but it is easy and simple.

 

[JimPearce]

Is it just me, or was that a confusing answer? Data doesn't include duplicates.

 

[malch]

BobSC wrote:

The reliability of items like power supplies and disk drives plummets pretty badly after 5 years. So building systems out of components that old is unlikely to work out well without a great deal of good luck.
P.S. I generally dispose of disks once they get to 5 years old. The small capacity, high power consumption, and poor reliability makes them more of a liability than an asset.

Depends on what they were in the first place. Our Dell business class machines seem to just keep running.

AFAIK, Dell use the same basic disk drive models in their business class machines that they do in their consumer products.

However, the power supplies in the business class machines are definitely superior.