Warning. Honest.

Started Mar 26, 2013 | Discussions
chuhsi Contributing Member • Posts: 999
Re: I keep five copies of every file

DotCom Editor wrote:

Files are never, ever stored locally on any computer's hard drive at my home. All files are stored on an Ethernet-connected NAS array running in RAID 5 configuration. This array is mirrored to a second NAS RAID array. The first array is also mirrored to yet another NAS RAID array that's located across town. That's three copies. All files are backed up to a series of USB-connected external drives that are in rotation and stored in the safe deposit box at my bank. That's four copies. Finally, all files are stored -- NOT backed up, but stored -- at an online cloud service that I consider reliable. That's five copies. It is essential to understand that storage and back up are NOT the same thing.

The one thing that screws this up is that Lightroom catalogs must be stored locally. I do that, but keep backups of the catalogs in all five of my aforementioned storage locations.

I sleep very well at night.

What cloud service do you use for your NAS? I haven't found an easy way to automatically back up my NAS to the cloud.

Thanks

 chuhsi's gear list:chuhsi's gear list
Nikon Df Nikon D4S Nikon AF-S Nikkor 24-70mm f/2.8G ED Nikon AF-S Nikkor 50mm f/1.4G Nikon AF-S Nikkor 70-200mm f/2.8G ED VR II +4 more
Steve Bingham
OP Steve Bingham Forum Pro • Posts: 25,114
Re: I keep five copies of every file
1

DotCom Editor wrote:

Files are never, ever stored locally on any computer's hard drive at my home. All files are stored on an Ethernet-connected NAS array running in RAID 5 configuration. This array is mirrored to a second NAS RAID array. The first array is also mirrored to yet another NAS RAID array that's located across town. That's three copies. All files are backed up to a series of USB-connected external drives that are in rotation and stored in the safe deposit box at my bank. That's four copies. Finally, all files are stored -- NOT backed up, but stored -- at an online cloud service that I consider reliable. That's five copies. It is essential to understand that storage and back up are NOT the same thing.

The one thing that screws this up is that Lightroom catalogs must be stored locally. I do that, but keep backups of the catalogs in all five of my aforementioned storage locations.

I sleep very well at night.

. . . maybe with your pants on . . . along with your belt and suspenders? Just joking. Most folks aren't THAT serious.

-- hide signature --

Steve Bingham
www.dustylens.com
www.ghost-town-photography.com

 Steve Bingham's gear list:Steve Bingham's gear list
Nikon D810 Nikon D5500 Nikon D7200 Nikon AF-S DX Nikkor 18-105mm f/3.5-5.6G ED VR Nikon AF Nikkor 85mm f/1.8D +30 more
wireless
wireless Contributing Member • Posts: 508
Re: Warning. Honest.

PHXAZCRAIG wrote:

[...]I'd say I've lost at least 30 hard drives in the last 10 years.   It gets your attention.
-- hide signature --

Craig
www.cjcphoto.net

It only took one failure to get my attention. 

David

 wireless's gear list:wireless's gear list
Nikon D800E Nikon AF-S Nikkor 14-24mm f/2.8G ED Nikon AF-S Nikkor 70-200mm f/2.8G ED VR II Sigma 105mm F2.8 EX DG Macro Nikon AF-S Nikkor 50mm f/1.8G +5 more
desidown Regular Member • Posts: 107
Re: I keep five copies of every file

chuhsi wrote:

DotCom Editor wrote:

Files are never, ever stored locally on any computer's hard drive at my home. All files are stored on an Ethernet-connected NAS array running in RAID 5 configuration. This array is mirrored to a second NAS RAID array. The first array is also mirrored to yet another NAS RAID array that's located across town. That's three copies. All files are backed up to a series of USB-connected external drives that are in rotation and stored in the safe deposit box at my bank. That's four copies. Finally, all files are stored -- NOT backed up, but stored -- at an online cloud service that I consider reliable. That's five copies. It is essential to understand that storage and back up are NOT the same thing.

The one thing that screws this up is that Lightroom catalogs must be stored locally. I do that, but keep backups of the catalogs in all five of my aforementioned storage locations.

I sleep very well at night.

What cloud service do you use for your NAS? I haven't found an easy way to automatically back up my NAS to the cloud.

Thanks

Assuming your NAS storage can be mapped as Windows  Shared drives on a windows box , you can use CrashPlan. That is what I use to backup my Windows Home Server data.

John King
John King Forum Pro • Posts: 14,941
Re: Warning. Honest.

Gidday Dwight

I have been in the computer industry for over 30 years.

Most people do not understand the importance of backup until about 30 seconds (roughly ... ) after they suddenly discover a need for it.

dwight3 wrote:

I back up my photos regularly.

However, I don't back up automatically.

I've been looking around for a program that will do that. I'm open to suggestions. How do you like the Acronis? Why did you chose that one? Are there some to avoid? Anyone else have favorites? (Windows here, not Apple or Unix).

Syncback is freeware (or more capable if you get a paid version - thie latter will backup open files such as Outlook .PST files ... ).

Even the free version does:
 - It is capable of doing binary comparions, if you want it to (takes a very long time ... ).
 - Data verification; safe copies; verified copies;
 - It can easily be scheduled to do backups as often as you like.
 - Extremely configurable.
 - ABSOLUTELY non-proprietary ... 
 - Can do backups; synchronisation; include/exclude files and folders, etc.

Available here:
http://www.2brightsparks.com/

I currently download my photos, do the initial processing, then back everything up. I won't reformat the card until all the photos have been put onto at least two drives.

I don't wipe the card until I have the images on my internal HDD, at least one external HDD, and a DVDR that is readable on the flakiest DVD drive amongst my computers. I also use Nero s/w; have the verify after write switch set (an unverified b/u is not worth anything ... ); and then I run Nero SCANDISK on the DVD to check for flaky sectors/files. Any yellow bits, that one goes in the bin, and I burn another. I have never had a failure with reading any of my archive DVDs.

At least one HDD is kept in a safe at all times.

I never leave external HDDs connected to my computers with either power or data cables unless actually in use.

One set of DVDs lives in a local bank safety deposit box ...

When I started digital photography I backed stuff up on CD's. After not too long a period of time the pile of CD's got to be a management problem. Finding things was difficult. And after about 3 years I found some CD's that were unreadable. I quit backing up on CD.

When I had a paying job I had some software on the work computer that would backup everything daily. However, it compressed the data using a proprietary format. I can't read the files now. Fortunately I did my own manual backups so I didn't lose anything.

I have a bias against using the cloud. (1) I don't know how reliable it is; (2) it's not under my direct control; (3) there times when I'm not online.

I feel the same. Any guarantee that the cloud company will be there tomorrow?

I currently back up using several external hard drives. One is on the computer all the time. Another is stored in a cabinet in the basement. A third is in my barn a mile away. That one is not online so I can't mirror it. I have to pick it up, bring it home, update it, and return it to its safe hiding spot.

For the laptop I have a 2 TByte USB drive that I can carry along.

Although I'm pretty good about backing up my photos, my other data don't fare so well. My wife shares the desktop computer and her files, along with my word processing files and email and other useful files get backed up manually once every month or two. An automated system would really be useful there.

Hope the above is of some use to you.

-- hide signature --

Regards, john from Melbourne, Australia.
(see profile for current gear)
Please do not embed images from my web site without prior permission
I consider this to be a breach of my copyright.
-- -- --
.
The Camera doth not make the Man (nor Woman) ...
Perhaps being kind to cats, dogs & children does ...
.
I am a Photography Aficionado ... and ...
"I don't have any problems with John. He is a crotchety old Aussie. He will smack you if you behave like a {deleted}. Goes with the territory." boggis the cat
.
Gallery: http://canopuscomputing.com.au/gallery2/v/main-page/

Bird Control Officers on active service.

 John King's gear list:John King's gear list
Olympus E-1 Olympus E-510 Olympus E-30 Olympus E-M1 Olympus Zuiko Digital 14-54mm 1:2.8-3.5 II +17 more
digitalshooter
digitalshooter Forum Pro • Posts: 19,604
Just cost me 900 bucks to have drive recovered

Daughters college hard Drive crashed during spring break.  Three years of college work, pics and music!

As a dad, had to do it for the memories.  I'm liking this idea combined with Dropbox for everyday files between laptop, iPad, and desktop!

-- hide signature --

Thanks,
Digitalshooter
PS: all posts are just my opinion!

 digitalshooter's gear list:digitalshooter's gear list
Sony Alpha DSLR-A700 Canon Pixma Pro9000 Mark II
sshoihet Senior Member • Posts: 2,595
Re: Ideally...

mnodonnell wrote:

...you'd have both a local backup and cloud-based backup and they'd be running at or near real time.

My local backup is managed by Time Machine and I use CrashPlan to create a second copy in the cloud.  Both run continuously.

-- hide signature --

www.michaelneilodonnell.com

I do something similar, automatic backup to local USB drive and secondary backup of the same data to Crashplan. Both run continuously and keep multiple file versions.

 sshoihet's gear list:sshoihet's gear list
Nikon D7000 Canon EOS M Nikon D600 Nikon D7100 Nikon AF-S DX Nikkor 16-85mm f/3.5-5.6G ED VR +13 more
Roger Engelken
Roger Engelken Veteran Member • Posts: 5,180
Re: Warning. Honest.

Four hard drives, an offsite location, and the cloud, those are my places for 208 gigabytes worth of photographs extending back eleven years to when I first joined the digital age.  Sound advice.

 Roger Engelken's gear list:Roger Engelken's gear list
Olympus PEN E-P5 Olympus E-M1 Olympus OM-D E-M10 Panasonic Lumix DMC-GX8 Olympus E-M1 II +1 more
MOD TOF guy Forum Pro • Posts: 14,340
Re: same here: I don't trust cd or dvd

FractalFlame wrote:

And it's funny how technology bring new, unwanted, thing.

Before CD's we had never heard of the aluminium eating fungus - then the alu CD's arrived and the fungus started eating the layers.

Now we have gold, silver, platinum, etc layers - soon we may have gold eating fungi.. or a bacteria that thrives on silver..

Oxidation has been around way before bacteria (or fungi, for that matter) even started to show up on earth

The idea behind using gold is, precisely, because it's the metal most resistant to oxidation.

-- hide signature --

Thierry - posted as regular forum member

 TOF guy's gear list:TOF guy's gear list
Nikon D810 Nikon D850 Nikon AF-S Nikkor 70-200mm f/2.8G ED VR Nikon AF-S DX Nikkor 12-24mm f/4G ED-IF Nikon AF-S Nikkor 24-70mm f/2.8G ED +6 more
Horshack Veteran Member • Posts: 6,137
Re: Warning. Honest.

orchidblooms wrote:

i have never had a HD fail - I have always build my own PC's with good - enterprise class HD's

Maybe i have been lucky

currently running raid array plus external usb HD

-- hide signature --

Phillip "Iggy" Murphy http://phillipmurphy.com
ILM Sandra Murphy - "The Orchid Lady" http://orchidblooms.com

The only truly enterprise-level HDs are lower-capacity SAS drives, typically with areal densities  far below the the consumer market. These drives are expensive and infrequently found outside of corporate/business use. Everything short of that is marketing.

KZEE Regular Member • Posts: 202
Re: Warning. Honest.

I used to have one external hard drive that I cloned to, but after on a couple of occasions having failures during the cloning process itself, I now have two external hard drives that I alternatively clone to so that if there's a problem during the cloning process, I have a back up. I also have a couple of flash drives that I store data only on.

In regard to the poster that says he has little sympathy for those that don't do back ups and lose their data, I have a lot of sympathy for those folks because if someone doesn't use computers a lot and isn't real computer savvy, they're not always always aware of the things they should be doing like backing up their data. A friend of mine fit into that category and her hard drive crashed she lost all of her photos and was devastated. I had another friend that was a college student and not really into computers all that much and not aware of the importance of backing her data up, and she wrote her thesis on a laptop and the laptop crashed and it was goodbye thesis. I have a lot of sympathy for them.

Ernie Misner
Ernie Misner Veteran Member • Posts: 3,814
Re: Warning. Honest.

>>> I've been looking around for a program that will do that. I'm open to suggestions. How do you like the Acronis? Why did you chose that one? Are there some to avoid? Anyone else have favorites? (Windows here, not Apple or Unix). >>>

Anyone, like me, using the Western Digital backup software that comes free with any WD drive purchase?   I've used it for a couple of years and it works great, and preserves the original folder system, etc.

If you have a WD external (or internal?) drive check out the software CD that comes with it.

 Ernie Misner's gear list:Ernie Misner's gear list
Nikon D810 Nikon AF-S Nikkor 24-70mm f/2.8G ED Nikon AF-S Nikkor 50mm f/1.4G Nikon AF-S Nikkor 17-35mm f/2.8D ED-IF Nikon AF Nikkor 85mm f/1.8D +4 more
aldoc Regular Member • Posts: 122
Re: A solution or: don't trust RAIDs too much

robot 1 wrote:

Mojn wrote:

Can only recommend a NAS with RAID 5 (2-4 mirrored disks). Even better: Get two, place one of them in your grandmothers attic and set up a mirror between the two NAS units.

A RAID is not a replacement for a back-up. Yes, you get more protection from failed drives, but you are still relying on a single piece of hardware (the RAID controller), and if that fails all your data is gone.

Yes, that's why I prefer several independent drives, in my case 5 numbered 1 or 2 TB drives which are used and exchanged on a weekly basis, each with full backup. After the one month full cycle, I format the oldest drive and start all over. Twice a year I buy a new drive, make a full backup and store it for a long time in a safe place outside.

This works for me, of course other solutions are as good or better and YMMV

Alfred

 aldoc's gear list:aldoc's gear list
Sony Alpha NEX-6 Nikon D7100 Sony Alpha DSLR-A850 Panasonic Lumix DMC-GH2 Nikon D750 +67 more
SeanU Senior Member • Posts: 2,070
Re: Warning. Honest.

I just finished building a nas for my home. I went with raidz for keeping the system up if a single drive fails. Its important to remember that raid is not a backup. My backup plan is to have 2 drives in rotation with full backups. One should always be off site. I am also using the ZFS file system. Zfs verifies the data on every read or write. It also supports snapshots. Snapshots allow you to keep track of any file that is modified or deleted,allowing you to recover files from the point in time that the snapshot was made. They also make incremental backups simple. I am going to set up a small old laptop to do remote off site syncing at some point to do away with moving the drives back and forth. I recommend reading the Tao of backup: taobackup.com/index.html .
--
http://www.dustandrust.com

FractalFlame Regular Member • Posts: 159
Re: same here: I don't trust cd or dvd

TOF guy wrote:

FractalFlame wrote:

And it's funny how technology bring new, unwanted, thing.

Before CD's we had never heard of the aluminium eating fungus - then the alu CD's arrived and the fungus started eating the layers.

Now we have gold, silver, platinum, etc layers - soon we may have gold eating fungi.. or a bacteria that thrives on silver..

Oxidation has been around way before bacteria (or fungi, for that matter) even started to show up on earth

The idea behind using gold is, precisely, because it's the metal most resistant to oxidation.

-- hide signature --

Thierry - posted as regular forum member

Oh true - the problems have existed in some form or other.

My point was that until the CD came into mass use, this fungus was not known. CDs arrive, fungus happens along.

Gold is resistant to pretty much anything of this nature (corrosive/consuming/etc - not meaning heat or physical damage)..

But - what if somewhere in the Amazon rainforests there's a gold eating bacteria. It's only that no one has been there with a gold dvd or piece of golf/ring/etc.
One day someone goes there and brings it back in a dvd movie they made of their trip.. and BAMF!!  we get our gold dvds eaten away!

Fantasy scenario, and just put up for discussion; but that is how the aluminium fungus got known, pure random happen-stance.

(We think we know so much about this world.. the reality is that we know so little - theorise a lot, yes, know little)

FractalFlame Regular Member • Posts: 159
Re: Warning. Honest.

rgolub wrote:

The major problem with 'cloud' storage is that the owner of said storage can decide they don't want to play anymore and they go home.  With your data.

Now, a reputable firm like Google is at least going to give you a couple of months to deal with it. Kim Dotcom, maybe not.  It's certainly not a good primary storage medium but it's OK for yet another storage system.

The key to backups is redundancy and automaticity.  Several backups, in different times, different hardware and different places.  Run by scripts or programs or whatnot.  In this day and age there is really no excuse not to have a decent backup strategy.

-- hide signature --

Just look at what rapidshare and filefactory have done recently..

Unlimited or 10 gig store suddenly reduced to 5g, 1G, etc - and you get a few days or weeks to collect your data and do something with it.

Cloud store is NOT safe - it's just got different reasons WHY it's unsafe..

nunatak Senior Member • Posts: 2,739
Re: Warning. Honest.

time machine only works on HFS+ formatted drives. what's worse, it was failing to backup one of my external drives since last October. recently, my Seagate 1.5 TB portable blew up on me. if it wasn't for my NAS backups I'd have lost 5 months of work.

redundancy is key. it's more important than trusting your critical work to a single backup. the more, the better.

-- hide signature --

design guy

husky92 Forum Member • Posts: 61
Re: A solution

Mojn wrote:

Can only recommend a NAS with RAID 5 (2-4 mirrored disks). Even better: Get two, place one of them in your grandmothers attic and set up a mirror between the two NAS units.

Yes. A second backup off site is critical. If your house burns down you lose all local backups. I don't like the cloud for 2 reasons, broadband upload is way too slow. I'm going to hit a terabyte of images at some point. It's just not practical to get that many images online. Second, I just don't trust it. Google just published the number of National Security Letters they receive from the government:

http://www.mintpress.net/google-applauded-for-disclosing-national-security-requests-for-user-data-to-public/

These are requests for user information that do not require a search warrant or subpoena.

michaeleclark Regular Member • Posts: 301
Acronis can fail... Other Win8 options

Used Acronis for several years, but it can be tempermental and I do not like the drive "lock down" procedure where utility access to the drives is limited due to Acronis controls implemented at boot.

Tried Ease-US Todo back up due to its high ratings. Offers much versatility, but like Acronis it can limit drive use and, in addition, its documentation is poor.

Since switching to Win8, now use the built in recovery manager for scheduled back ups. Very versatile, includes ability to gang SSDs used for drive acceleration as part of the primary BU, works seamlessly in the background, and in 6 months have not had a single problem. Another adcvantage is that it is available at preboot to access without shutting off low level drive access the way that Acronis and other BU utilities do. Also use MS SyncToy, a little-known free utility to sync files (including new or changed files) from one drive to another. Very easy to use, straight forward, and fast. If there is a failure, simple to reverse the sync to restore the files to a new drive.

Use the Win8 recovery routine to back up my system drives and SyncToy to BU photo files.

Michael E. Clark

 michaeleclark's gear list:michaeleclark's gear list
Nikon Coolpix AW100 Nikon D300 Nikon D700 Nikon 1 V1 Nikon D4 +11 more
Steve Bingham
OP Steve Bingham Forum Pro • Posts: 25,114
Re: Acronis can fail... Other Win8 options

michaeleclark wrote:

Used Acronis for several years, but it can be tempermental and I do not like the drive "lock down" procedure where utility access to the drives is limited due to Acronis controls implemented at boot.

Tried Ease-US Todo back up due to its high ratings. Offers much versatility, but like Acronis it can limit drive use and, in addition, its documentation is poor.

Since switching to Win8, now use the built in recovery manager for scheduled back ups. Very versatile, includes ability to gang SSDs used for drive acceleration as part of the primary BU, works seamlessly in the background, and in 6 months have not had a single problem. Another adcvantage is that it is available at preboot to access without shutting off low level drive access the way that Acronis and other BU utilities do. Also use MS SyncToy, a little-known free utility to sync files (including new or changed files) from one drive to another. Very easy to use, straight forward, and fast. If there is a failure, simple to reverse the sync to restore the files to a new drive.

Use the Win8 recovery routine to back up my system drives and SyncToy to BU photo files.

Michael E. Clark

Nice post! And yes, Acronis has been a PITA at times - in the beginning! I am not sure it is a great choice for a novice. Unfortunatly, Ease-US Todo was not available t the time (years ago).

-- hide signature --

Steve Bingham
www.dustylens.com
www.ghost-town-photography.com

 Steve Bingham's gear list:Steve Bingham's gear list
Nikon D810 Nikon D5500 Nikon D7200 Nikon AF-S DX Nikkor 18-105mm f/3.5-5.6G ED VR Nikon AF Nikkor 85mm f/1.8D +30 more
Keyboard shortcuts:
FForum MMy threads