backup strategies

Started Apr 14, 2013 | Discussions
chironNYC
Senior MemberPosts: 1,431
Like?
backup strategies
Apr 14, 2013

This may not quite belong in this forum, but I don't see another better place to get a range of well-informed answers.

What strategies, technologies, and software do people use to backup their photographs?

My particular situation is this: I have moved all of my photographs from multiple sources onto an external Western Digital 2 TB usb drive. I have located my Lightroom catalog onto this drive also. One of my purposes in doing this is to permit me to edit photographs on multiple computers by connecting the 2 TB external drive to whatever computer I am using.

What would be the best current ways to back up, more or less continuously, this 2 TB external usb drive? Remember that the drive will be connected to 2 or 3 different computers for editing and for importing pictures from sd and cf cards that will change the files on the drive and the Lightoom catalog.

Thanks for the help!

Peter

Carbonman
Regular MemberPosts: 340
Like?
Re: backup strategies
In reply to chironNYC, Apr 14, 2013

Backups are a critical part of your plan and many overlook this critical function.  There are several major items that should be looked at in your planning:

1.) Backup software - you need good backup software because if you're going to rely on yourself to remember to the backups, they will soon be out of date.  I like Nova Backup because it's very feature rich, supports external and network drives and is easy to use.

2.) Main storage - where are you going to store your originals?  What's the capacity and what is your expansion plan.  I prefer internal drives because of speed and you can still access them from other locations if needed, although the external access will certainly be slower.  My computer has 3 - 2TB drives for photo storage and another 2TB drive for the Lightoom catalog and other documents.

3.) Backup media - what are you going to store on?  I like Network Attached Storage (NAS) because it allows me to place the backup drives away from the rest of the house and out of harms way.  I have a series of three NAS drives, each with 2-2TB drives and a quad drive with 4-2TB drives.  Each of the source drives does an incremental backup to one of the NAS drives.  There's also a weekly backup to a separate NAS drive.  That way each source disk is backed up daily to one NAS and weekly to another.  Finally, I keep an off-site backup to external USB drives.  Since two of my three internal drives are basically static (as they get close to capacity, I add an additional drive internally) the off-site backups are current.

4.) Expansion - To add more storage, I just add another internal drive, add another NAS (I use DLink Duos) and buy another external USB drive for the offsite storage.

It may sound like overkill (it costs about $600 to add 2TB of drive space with all the backup equipment) but it keeps things backed up very diligently and I don't sweat equipment failures and that cost is easy to absorb compared to a data loss scenario.

The only downfall I see is that everything on on magnetic media, but with image sizes and the size of modern storage, there's not really a good option these days.

-- hide signature --

Carbonman
Flame alert: I wear Ztex underwear so if you flame me, not only don't I care, but you can't toast what matters.

Reply   Reply with quote   Complain
rbarta
Regular MemberPosts: 276Gear list
Like?
Re: backup strategies
In reply to chironNYC, Apr 14, 2013

Right now I have all my photos on my local drive using Time Machine to backup, but soon that solution will be impossible as I am running out of local drive space.  I also back up photos to a separate drive that I leave at my work in case of any fire at home.

My plan (after I get back from a 2 week Galapagos trip) is to transition to a Western Digital 4TB RAID ($290) and set it to level 1 (2TB mirrored drives) and put this on my Airport as a shared drive.  I will still copy to a separate drive to save at work once a month.

 rbarta's gear list:rbarta's gear list
Canon EOS-1D X Canon EOS 5D Mark III Canon EF 15mm f/2.8 Fisheye Canon EF 50mm f/1.2L USM Canon EF 16-35mm f/2.8L II USM +7 more
Reply   Reply with quote   Complain
RedFox88
Forum ProPosts: 22,122Gear list
Like?
Re: backup strategies
In reply to chironNYC, Apr 14, 2013

Just what do you do for pay that you need to be so into "backup strategies" ?

Reply   Reply with quote   Complain
chironNYC
Senior MemberPosts: 1,431
Like?
Re: backup strategies
In reply to Carbonman, Apr 14, 2013

Carbonman wrote:

Backups are a critical part of your plan and many overlook this critical function.  There are several major items that should be looked at in your planning:

1.) Backup software - you need good backup software because if you're going to rely on yourself to remember to the backups, they will soon be out of date.  I like Nova Backup because it's very feature rich, supports external and network drives and is easy to use.

I will take a good look at NovaBackup. It is a bit pricey, but is very well-reviewed and seems to be easy to use, which is important to me.

2.) Main storage - where are you going to store your originals?  What's the capacity and what is your expansion plan.  I prefer internal drives because of speed and you can still access them from other locations if needed, although the external access will certainly be slower.  My computer has 3 - 2TB drives for photo storage and another 2TB drive for the Lightoom catalog and other documents.

My current plan has been to store the originals on a 2TB external usb drive and have them also backed up on another 2TB external usb drive. The Lightroom catalog for the photos would also reside on the external drive. This would let edit them in Lightroom on various computers that I might be using.

I have a feeling that this storage solution would make you cringe. I may reconsider and go to a different arrangement. I am not a professional, so the amount of redundancy that makes sense is less than if my livelihood depended on the files.

3.) Backup media - what are you going to store on?  I like Network Attached Storage (NAS) because it allows me to place the backup drives away from the rest of the house and out of harms way.  I have a series of three NAS drives, each with 2-2TB drives and a quad drive with 4-2TB drives.  Each of the source drives does an incremental backup to one of the NAS drives.  There's also a weekly backup to a separate NAS drive.  That way each source disk is backed up daily to one NAS and weekly to another.  Finally, I keep an off-site backup to external USB drives.  Since two of my three internal drives are basically static (as they get close to capacity, I add an additional drive internally) the off-site backups are current.

If I don't go with the external drive storage option, I would probably create a tower system with three 2TB internal drives: One drive for photo originals; one drive for photo backups; the third drive for everything else I do on the computer, with my documents backed up to Skydrive and iDrive. I might add to this a NAS 3TB drive to back up the photographs on the network in addition to their location on the 2 internal drives. That all should be plenty for me.

4.) Expansion - To add more storage, I just add another internal drive, add another NAS (I use DLink Duos) and buy another external USB drive for the offsite storage.

It may sound like overkill (it costs about $600 to add 2TB of drive space with all the backup equipment) but it keeps things backed up very diligently and I don't sweat equipment failures and that cost is easy to absorb compared to a data loss scenario.

The only downfall I see is that everything on on magnetic media, but with image sizes and the size of modern storage, there's not really a good option these days.

-- hide signature --

Carbonman
Flame alert: I wear Ztex underwear so if you flame me, not only don't I care, but you can't toast what matters.

Thanks so much for such a detailed and well-informed answer! It is very helpful.

Peter

Reply   Reply with quote   Complain
Jason Rickerby
Contributing MemberPosts: 712
Like?
Lightroom Catalog makes life tricky
In reply to chironNYC, Apr 14, 2013

I use a Synology 2-drive NAS (RAID 1) to store all my images. In turn, the Synology runs two backup jobs, to two USB connected drives. So effectively there 4 physical copies of my images. When I travel, I relocate one of the USB drives to another location, providing some protection in the unlikely event of a fire.

Lightroom on Windows does not want to store the catalog on a network mapped drive. I therefore have my catalogs on a local SSD on my PC, then run a weekly backup job to copy the catalogs, without the previews, to the Synology.

Given the OPs requirements, my method is not friendly for easily moving catalogs between computers. I suppose I could put the catalog on a USB drive connected to the PC, leaving the images on the Synology NAS.

It should be mentioned the the NAS can be a bottleneck with regard to ultmiate performance, but this is a trade off I live with for the peace of mind. I can also connect to the Synology NAS from anywhere with an Internet connection.

Reply   Reply with quote   Complain
TimR32225
Veteran MemberPosts: 3,117Gear list
Like?
Re: backup strategies
In reply to RedFox88, Apr 14, 2013

RedFox88 wrote:

Just what do you do for pay that you need to be so into "backup strategies" ?

Are you suggesting that someone's images are not worth protecting if he is not doing it for pay?    Everyone needs a backup strategy, regardless...

-- hide signature --
 TimR32225's gear list:TimR32225's gear list
Canon EOS 7D Canon EOS-1D Mark IV Canon EOS-1D X Canon EF 300mm f/2.8L IS USM Canon EF 500mm f/4.0L IS USM +11 more
Reply   Reply with quote   Complain
chironNYC
Senior MemberPosts: 1,431
Like?
Re: backup strategies
In reply to RedFox88, Apr 14, 2013

RedFox88 wrote:

Just what do you do for pay that you need to be so into "backup strategies" ?

Nothing at all for pay, but almost 10 years of photographs that are quite important to me. I would like a strategy that is safe, convenient, relatively inexpensive, automatic, and otherwise easy. It should also be likely to transfer easily to evolving technologies.

Reply   Reply with quote   Complain
chironNYC
Senior MemberPosts: 1,431
Like?
Re: backup strategies
In reply to rbarta, Apr 14, 2013

rbarta wrote:

Right now I have all my photos on my local drive using Time Machine to backup, but soon that solution will be impossible as I am running out of local drive space.  I also back up photos to a separate drive that I leave at my work in case of any fire at home.

My plan (after I get back from a 2 week Galapagos trip) is to transition to a Western Digital 4TB RAID ($290) and set it to level 1 (2TB mirrored drives) and put this on my Airport as a shared drive.  I will still copy to a separate drive to save at work once a month.

The Western Digital RAID set up to 2X2 TB sounds and put on the network sounds pretty good to me, though I think access might be slow if the networked drives are the main location for working.

Reply   Reply with quote   Complain
chironNYC
Senior MemberPosts: 1,431
Like?
Re: Lightroom Catalog makes life tricky
In reply to Jason Rickerby, Apr 14, 2013

Jason Rickerby wrote:

I use a Synology 2-drive NAS (RAID 1) to store all my images. In turn, the Synology runs two backup jobs, to two USB connected drives. So effectively there 4 physical copies of my images. When I travel, I relocate one of the USB drives to another location, providing some protection in the unlikely event of a fire.

Lightroom on Windows does not want to store the catalog on a network mapped drive. I therefore have my catalogs on a local SSD on my PC, then run a weekly backup job to copy the catalogs, without the previews, to the Synology.

Given the OPs requirements, my method is not friendly for easily moving catalogs between computers. I suppose I could put the catalog on a USB drive connected to the PC, leaving the images on the Synology NAS.

It should be mentioned the the NAS can be a bottleneck with regard to ultmiate performance, but this is a trade off I live with for the peace of mind. I can also connect to the Synology NAS from anywhere with an Internet connection.

Sounds like a strong system. I think your needs are different from mine--I would like to move easily between my laptop and desktop for editing by using the external usb drives. I do like the idea of a networked NAS backup in addition to a backup on a 2nd physical drive.

I think the question for me is whether I can have the portability I want and still have a secure and automatic backup system. Maybe two external usb drives with the Lightroom catalog on the drive and one drive backing up the other, and then a NAS drive as a third backup that is physically separate from the two external usb drives?

Thanks for your help!

Peter

Reply   Reply with quote   Complain
harrygilbert
Senior MemberPosts: 2,799Gear list
Like?
Re: backup strategies
In reply to chironNYC, Apr 14, 2013

I use a very simple backup strategy; after transferring images from the camera to my PC, I copy the newly created folder with images to 4 additional separate hard drives, and to a fifth network-attacked drive. After post-processing, I re-copy from the work drive to the backup drives. If a drive fails, I still have 4 others containing my images. Less hassle than any backup software, and I control when and what I backup. These days, hard drives are really cheap on a $/Gb basis. My main PC is always on, and connected to my network, so I have access from other computers on the network.

I also do this with downloaded software, along with a copy of the invoice and serial numbers.

 harrygilbert's gear list:harrygilbert's gear list
Canon EOS 5D Canon EOS 50D Canon EOS-1D Mark IV Canon EOS-1D X Canon EF 15mm f/2.8 Fisheye +17 more
Reply   Reply with quote   Complain
El Wray
Junior MemberPosts: 49
Like?
Re: backup strategies
In reply to harrygilbert, Apr 14, 2013

My backup system isn't sophisticated, but it's simple and works for my needs. For me, simple wins out because it means I'll actually do it.

I have my pictures on three drives: the PC's internal drive, an external USB drive always attached the PC, and another USB drive which I keep off site.

I use FreeFileSync to keep the drives synced and up-to-date.

FreeFileSync is fast and easy, so I can quickly update the always-attached USB drive frequently. It compares the two drives (or folders if you prefer) and only backs up new or changed files.  It's a neat little piece of software.  It's so easy and quick to use, I find myself running it a lot- which is good.

In addition to the always-attached USB drive,  every so often, I bring the off-site drive home and update it too.

Reply   Reply with quote   Complain
RedFox88
Forum ProPosts: 22,122Gear list
Like?
Re: backup strategies
In reply to chironNYC, Apr 14, 2013

chironNYC wrote:

It should also be likely to transfer easily to evolving technologies.

Unlikely.  Even camera makers generally make you buy a new product in order to get new features and compatibilities.

Reply   Reply with quote   Complain
Graham Meale
Senior MemberPosts: 1,093Gear list
Like?
Re: backup strategies
In reply to chironNYC, Apr 14, 2013

I back up to an external drive and make two copies on DVD, one of which is kept off-site.

-- hide signature --
 Graham Meale's gear list:Graham Meale's gear list
Canon EOS 5D Mark II Canon EOS 5D Mark III Canon EF 50mm f/1.8 II Canon EF 100mm f/2.8 Macro USM Canon EF 70-300mm f/4-5.6L IS USM +5 more
Reply   Reply with quote   Complain
PeterK
Contributing MemberPosts: 874
Like?
Re: backup strategies
In reply to Graham Meale, Apr 14, 2013

You've had a lot of excellent advice. Please note that one poster mentioned a periodic (weekly?) backup.

Let me emphasize the value of weekly/monthly and quarterly backups. When you copy over your updated work every day, including mirroring deletions, you might accidentally delete wanted files that you have deleted accidentally. On a few occasions I have needed that removable drive with the backup from a month or two or three ago. In one case, I had accidentally deleted a good image file for a client right after posting her images to my web site. I then made sure I ran all my backups, and I was content.  It was a couple of months later that she asked for prints, and I found out the file didn't exist any more. Luckily it was on a quarterly backup disk.

Reply   Reply with quote   Complain
Jason Rickerby
Contributing MemberPosts: 712
Like?
Have a strong backup and test plan
In reply to chironNYC, Apr 15, 2013

chironNYC wrote:

Sounds like a strong system. I think your needs are different from mine--I would like to move easily between my laptop and desktop for editing by using the external usb drives. I do like the idea of a networked NAS backup in addition to a backup on a 2nd physical drive.

I think the question for me is whether I can have the portability I want and still have a secure and automatic backup system. Maybe two external usb drives with the Lightroom catalog on the drive and one drive backing up the other, and then a NAS drive as a third backup that is physically separate from the two external usb drives?

Thanks for your help!

Peter

My desktop PC monitor is large, wide gamut and calibrated, so it doesn't make a lot of sense for me to post-process on a laptop screen. When doing event photography, I will work on my laptop, but then import the catalog and images into the PC / NAS environment when I get home.  Plus, my desktop PC is way faster than any laptop.

The beauty of the NAS is the automation. The Synology units have a huge amount of functionality and include software to automate backup of data on your computer. With external USB drives, it's hard to know when they are going too fail, verses the NAS with Western Digital Red drives, SMART, etc.

Whatever method works for you: Make sure you're performing regular backups; do not trust the backup - test that you can successfully recover data off the backup regularly. I have first hand knowledge of discovering that the backup is corrupt when trying to perform a restore. One trick - have the back software perform a full scheduled backup, not an incremental. The full backup will take longer, cause more wear and tear, but will alert you that the backup drive is failing. It's possible to do incremental backups for months before finding out that corrupt sectors on the backup drive will prevent data recovery. As a rule of thumb, replace drives every 3 years.

Reply   Reply with quote   Complain
chironNYC
Senior MemberPosts: 1,431
Like?
Re: Have a strong backup and test plan
In reply to Jason Rickerby, Apr 15, 2013

Jason Rickerby wrote:

chironNYC wrote:

Sounds like a strong system. I think your needs are different from mine--I would like to move easily between my laptop and desktop for editing by using the external usb drives. I do like the idea of a networked NAS backup in addition to a backup on a 2nd physical drive.

I think the question for me is whether I can have the portability I want and still have a secure and automatic backup system. Maybe two external usb drives with the Lightroom catalog on the drive and one drive backing up the other, and then a NAS drive as a third backup that is physically separate from the two external usb drives?

Thanks for your help!

Peter

My desktop PC monitor is large, wide gamut and calibrated, so it doesn't make a lot of sense for me to post-process on a laptop screen. When doing event photography, I will work on my laptop, but then import the catalog and images into the PC / NAS environment when I get home.  Plus, my desktop PC is way faster than any laptop.

The beauty of the NAS is the automation. The Synology units have a huge amount of functionality and include software to automate backup of data on your computer. With external USB drives, it's hard to know when they are going too fail, verses the NAS with Western Digital Red drives, SMART, etc.

Whatever method works for you: Make sure you're performing regular backups; do not trust the backup - test that you can successfully recover data off the backup regularly. I have first hand knowledge of discovering that the backup is corrupt when trying to perform a restore. One trick - have the back software perform a full scheduled backup, not an incremental. The full backup will take longer, cause more wear and tear, but will alert you that the backup drive is failing. It's possible to do incremental backups for months before finding out that corrupt sectors on the backup drive will prevent data recovery. As a rule of thumb, replace drives every 3 years.

Thank you again for such good advice. I had not thought about the issue of monitor calibration--working on a laptop would certainly lead to occasional unexpected results. I am just beginning to actual use editing software, and so the calibration will matter more than it did before for me. I may rethink my working plans. I also have a nice desktop station to work from. If i did a drive on the desktop backed up to an external usb drive and to a NAS drive, that would be quite secure. I guess I would have to give up working on the laptop, except perhaps as a first pass when the images come from the camera on a cf/sd card.

Thank you also for the good advice on best practices of making and testing the backups. I am sure many other readers will also use your advice.

Peter

Reply   Reply with quote   Complain
hodown55
Regular MemberPosts: 143
Like?
Re: backup strategies
In reply to chironNYC, Apr 16, 2013

Some really great advice so far. One thing that many people do not think about is future compatibility. Many back-up programs use their own compression algorithms, to save drive space. If the company goes out of business or when new operating systems are introduced, there is no guarantee that your backup software or the back-up files will be readable. That is why I use Microsoft 'Synchtoy' for backing up my image files. This program was written specifically for photographers & is free. It uses no compression, so your back-up files are identical to the originals. Of course, this means that your back-ups will require more drive space than compressed back-ups, but HDs are getting cheaper ...

Details & download are available here: http://www.microsoft.com/en-us/download/details.aspx?id=15155

As with all back-up programs, select your type of back-up carefully as in some modes files deleted from one of the drives, will also be deleted from the other. I use 'Contribute', to avoid this.

Hope this helps.

Kev

Reply   Reply with quote   Complain
rstone
Regular MemberPosts: 266
Like?
Re: backup strategies
In reply to PeterK, Apr 16, 2013

I also use FreeFileSynch and absolutely love it. If you run in in "mirror" mode, you can't go wrong.

Do all your editing on C:, then mirror to E: . FFS will copy and/or overwrite the E: drive as needed as well as deleting all the files from the E: if you have deleted them from C:

Did I say it's fast and easy..? You simply set up the folders you want to back up on each side then run it.. Here are some of the folders I have set up for example:

C: My Documents ----> E: My Documents

C: Photos -----> E: Photos

C: MP3 ------> E: MP3

etc..

Reply   Reply with quote   Complain
chironNYC
Senior MemberPosts: 1,431
Like?
Re: backup strategies
In reply to hodown55, Apr 16, 2013

hodown55 wrote:

Some really great advice so far. One thing that many people do not think about is future compatibility. Many back-up programs use their own compression algorithms, to save drive space. If the company goes out of business or when new operating systems are introduced, there is no guarantee that your backup software or the back-up files will be readable. That is why I use Microsoft 'Synchtoy' for backing up my image files. This program was written specifically for photographers & is free. It uses no compression, so your back-up files are identical to the originals. Of course, this means that your back-ups will require more drive space than compressed back-ups, but HDs are getting cheaper ...

Details & download are available here: http://www.microsoft.com/en-us/download/details.aspx?id=15155

As with all back-up programs, select your type of back-up carefully as in some modes files deleted from one of the drives, will also be deleted from the other. I use 'Contribute', to avoid this.

Hope this helps.

Kev

This is a great suggestion. I had never heard about Microsoft's synchtoy. It sounds like that might do a lot of what I want to do since I would like to have the backups not depend on my remembering to do them every week or so. I am going to give it a trial run. Thank you for the link!

Peter

Reply   Reply with quote   Complain
Keyboard shortcuts:
FForum MMy threads