Building a photo PC

He said 3G. Without going to the 64-bit version of Windows (either XP or Vista), can standard 32-bit Windows XP handle more than 3 gig of memory? I really don't know, since I tend to focus on Linux more than Windows, and certainly in the server side of things, 32/64 gigabytes are doable. Even if Windows doesn't handle it right now, it would make sense to have a motherboard that has as many memory slots as possible for future growth.

If you went a little lighter on the graphics card (save it for later when you commit to 3D CAD), you could use the extra cash to buy a color calibration device.

--
Jeff

Having recently made the jump to an external RAID 1 drive, I'd enjoying hearing more about your thoughts and experience on configuring RAID for photo use.

My goal was a to have a redundant hard drives for holding a working copy of my photo library/archive, and then use a second external drive (non-RAID) for backup. I went external so that I could use the library with either a desktop or laptop machine, and be able to easy transfer the library to a new machine at a later date.

But there sure is a lot of jargon and conflicting advice out there regarding RAID setups. Would it have been better to go with RAID 5? If the primary goal is reliability and a backup, what's the best way to allocate a fixed amount of money?

My sole interest is to support photography, so no need to worry about databases, gaming, getting the last bit of speed out of the machine, etc. Simply and purely photography. Does that make a difference in how HDD storage should be configured?

--
Jeff

Jeff said:

Having recently made the jump to an external RAID 1 drive, I'd
enjoying hearing more about your thoughts and experience on
configuring RAID for photo use.

My goal was a to have a redundant hard drives for holding a working
copy of my photo library/archive, and then use a second external
drive (non-RAID) for backup. I went external so that I could use
the library with either a desktop or laptop machine, and be able to
easy transfer the library to a new machine at a later date.

Click to expand...

I think RAID 1 is overkill in your case. A single drive in a box, or a RAID 0 would serve just as well. If you have a backup copy of your library on an external drive, if your primary copy fails - all you have to do is plug in the backup drive. At most, you lose the images added since the last backup. Those images should still be on your flash cards.

In my opinion, it is safe for you to convert your RAID 1 into RAID 0, and benefit from the faster performance and added capacity.

Jeff said:

But there sure is a lot of jargon and conflicting advice out there
regarding RAID setups. Would it have been better to go with RAID 5?
If the primary goal is reliability and a backup, what's the best
way to allocate a fixed amount of money?

Click to expand...

I think you already got enough reliability from your backup scheme.

A hardware-based RAID5 will be fairly expensive. It will also lock you into a specific size and model of hard drives. (RAID 5 needs at least three drives). This may not be easy if you had the box for a couple of years.

If the RAID5 box itself dies (a bad fan, or a controller, or a cable connector), most likely your data will be inaccessable by any other means.

Software-based RAID5 has its own problems. It's not easy to set up and maintain, and it takes up CPU cycles.

Jeff said:

My sole interest is to support photography, so no need to worry
about databases, gaming, getting the last bit of speed out of the
machine, etc. Simply and purely photography. Does that make a
difference in how HDD storage should be configured?

Click to expand...

I think the extra storage and performance of RAID 0 would benefit you more than the 0% down-time guarantee of RAID 1.

Do you have an uninterrupted power supply? If you don't, you certainly don't need RAID 1.

Hope this helps!

Boris

Thanks! That's precisely the advice I was seeking.

My two disk Western Digital device can run either in RAID 0 as a fast, 1TB drive, or in RAID 1 as mirrored 500GB drive. I've done the later, with an additional 500GB drive for backup. (As much for insurance against my own screwups as hardware failure).

I'm hoping to do a PC upgrade sometime in coming months, and at that point will probably reconfigure as RAID 0 to take advantage of the additional speed and capacity.

Your advice regarding RAID 5 makes good sense, thanks adding some clarity.
--
Jeff

Jeff said:

Thanks! That's precisely the advice I was seeking.

Click to expand...

You're welcome!

I just thought of something though. In order to get the speed advantage of RAID 0, you need a fast connection. If your RAID box has an external eSATA connector, you're good. However, if it's USB 2.0 or Firewire, RAID 0 may not give you much benefits. You might want to download HD benchmarking software (like HD Tach) and run your own tests.

In that case, you could be better off switching the box to JBOD ("Just a Bunch Of Disks" - making one disk out of two, using the second disk when the first one is full).

Boris

I was wondering about that, too.

The connections to the box are USB 2.0, Firewire 400, and Firewire 800. Im currently using Firewire 400 with good luck. If I'm working with some files, I usually take them over to the local drive of whatever machine I'm working on, work with them there, and then put them back in the archive and catalog when done. I really want to preserve the integrity of the archive, so don't wish work directly in those folders anyways (to avoid accidental screwups).

I'd plan to get firewire 800 in any new machine, so that's a small upgrade path. Maybe eSATA would be the way to go later, unfortunately don't have that in this current device.

I know I'm being cautious having three physical copies of every file. (2 via RAID 1, and one on backup), but the two redundant copies are more or less automated and easy to keep sync'd. The photos include many of kids growing up and all the rest, so fail-proof is my highest priority.

--
Jeff

I have issues with RAID-0, if no other backup scheme is in place. Every time someone talks about going to a stripe set I think back to statistics class. One unit means rolling the dice once and hoping that you don't have a negative result. Spreading your data across 2 or 3 drives is all well and good if you want the speed advantage of striping and are willing to be diligent in back-ups, but if speed isn't a factor then you might as well split those drives up and use them individually. Having 2 drives is like rolling the dice twice and hoping that one, the other, or both don't come up snakeyes.

BorisK1 said:

I think RAID 1 is overkill in your case. A single drive in a box,
or a RAID 0 would serve just as well. If you have a backup copy of
your library on an external drive, if your primary copy fails - all
you have to do is plug in the backup drive. At most, you lose the
images added since the last backup. Those images should still be on
your flash cards.

Click to expand...

Note, a RAID0 system (more precisely an interleaved system, since there is no redundancy) is less secure than higher levels of RAID, since if either disk has hardware problems, it brings down the whole system.

And as always, a RAID system is not a backup system, in that it doesn't give you any backup in case you or the OS accidently delete files. All a RAID system does is prevent a hardware failure from taking you down.

No Text

Rob MacLennan said:

I have issues with RAID-0, if no other backup scheme is in place.
Every time someone talks about going to a stripe set I think back
to statistics class. One unit means rolling the dice once and
hoping that you don't have a negative result. Spreading your data
across 2 or 3 drives is all well and good if you want the speed
advantage of striping and are willing to be diligent in back-ups,
but if speed isn't a factor then you might as well split those
drives up and use them individually. Having 2 drives is like
rolling the dice twice and hoping that one, the other, or both
don't come up snakeyes.

Click to expand...

You're absolutely correct (except, you forgot to include the box itself - we're talking external RAID here. That's one more thing that can fail).

Still, I was replying to the poster who already has a good backup solution.

There's one wide-spread misconception though: None of the redundant RAID schemes (1, 5, 10, etc) are substitutes for proper backup. Their built-in redundancy guarantees uninterrupted operations, but it does not safeguard from losing images to file corruption, user error, viruses, etc.

Michael Meissner said:

Note, a RAID0 system (more precisely an interleaved system, since
there is no redundancy) is less secure than higher levels of RAID,
since if either disk has hardware problems, it brings down the
whole system.

Click to expand...

In general, this is correct. I was replying to a specific situation:

First, his RAID box is not a system drive, so if it goes down, the machine will still boot;

Second, he has another (non-RAID) drive with copies of all files from the RAID box. So if the RAID box goes down, he can simply plug in the backup drive, and he's back in business.

Michael Meissner said:

And as always, a RAID system is not a backup system, in that it
doesn't give you any backup in case you or the OS accidently delete
files. All a RAID system does is prevent a hardware failure from
taking you down.

Click to expand...

Agreed.

Boris

You're quite correct. The only thing that will protect you somewhat from an errant press of the key is a proper backup scheme. A good RAID implementation is useful for 'mission critical' data. If you make your living from your pictures and work to a deadline (ie. photojournalist), then I would definitely recommend RAID-5 as part of a data storage solution, with a spare drive or two available for immediate rebuilds. A large amount of space in a single drive or tape backup solution would be adequate backup in such a situation.

I see far too many otherwise intelligent clients trust irreplaceable data to a single storage medium. I've lost count of the number of professors whose course outlines, work for publication, or thesis documents I've rescued for them.

This has been helpful to me, thanks all. While I had some understanding of RAID after a little googling, it's great to have the advantage of real world experience.

Boris has a good read on my situation, these are photos (about 40,000 in 200GB) that represent a lot of family memories and photography learning. So they have high value to us, but we don't need 'hot-spares'. I might also use this system to backup the itunes library, but that's it, I want to otherwise keep these completely segregated from systems, etc. Just archives.

Based on your helpful advice, I think I'll stick with a working disk + RAID 1 + backup until I get a new system update in a few months, then consider going to a working disk + RAID 0 + backup if benchmarking shows any advantage to that move.

Thanks again,
Jeff

From what I understand about RAID5, it's not that easy to build and maintain. It also isn't what you'd call affordable or flexible. It's perfect for an online mission-critical database, but for a journalist - I doubt it.

I'm not a journalist myself, so maybe my impression is wrong. But I think a journalist is somebody who is on the road all the time, switches computers fairly often, and needs to be able to take his data with him. He may have to move the data from computer to computer, from laptop to desktop and back, etc.

Rob MacLennan said:

You're quite correct. The only thing that will protect you somewhat
from an errant press of the key is a proper backup scheme. A good
RAID implementation is useful for 'mission critical' data. If you
make your living from your pictures and work to a deadline (ie.
photojournalist), then I would definitely recommend RAID-5 as part
of a data storage solution, with a spare drive or two available for
immediate rebuilds. A large amount of space in a single drive or
tape backup solution would be adequate backup in such a situation.

Click to expand...

Different types of backup suit different needs. Some people (for example, software engineers like myself) need to be able to come back to a "snapshot" of a drive as it was on a specific date. I need all intermediate versions of all files in my source code.

An image library is different. There's usually no need to bother with versions - but ease and simplicity of recovery is very important.

What would you say of this backup strategy for an image library: Have a second copy of the library on an external drive, that's usually turned off. Every night (or every week, or whenever new images come in), synchronize that drive with the active library.

If the active library goes down - simply plug in the backup one.

In this scheme, the active library can be RAID0 for speed.

To be completely safe, I would also place the most important "keeper" images to a photo website, or to an internet backup service.

Does this sound logical?

Boris

For the video card I use the Diamond (now sold as ATI) X550 PCIE 256mb card.

It does all I'll ever need for video plus its driver software and control software make calibration a snap.
--
It seems Mark Twain had something to say about Oly's next Pro DSLR

'There is something fascinating about science. One gets such wholesale returns of conjecture out of such a trifling investment of fact'. Mark Twain.

Larry Lynch
Mystic, Ct.

I wouldn't advise RAID 0 unless you keep very current backups. HDDs are the most common part to fail in a computer, since they're also the only one with moving parts. In RAID 0 you're twice as exposed. I've got a drawer full of failed HDDs.

As great as RAID 5 is, or any RAID, it doesn't replace a backup. That's a mentra to sysadmins everywhere. The purpose of using RAID 5 in those workstations wasn't to keep data safe, but to have the highest possible disk performance without overexposing the machines to failure. To safeguard your pictures, you'll still need external backup. HDDs can be replaced, but your data can't.

If you want disk speed and a snappier system, RAID is the solution. Borisk1 is right in saying that software RAID 5 is a load of bull. (By "software" I mean the one found integrated on motherboards and cheap RAID cards, where the driver does all the heavy lifting. Linux people call this fakeraid.) For hardware RAID 5 you'll need to spend at least 400$US for the controler, a motherboard with two PCIe ports (one taken by the video card) and for best results, four hard drives.

--
Djof ~ 'Je me souviens'

I run AutoCAD, BuildersCAD and SoftPlan on my PC's. They are (believe it or not) more efficient to run than my photo editing and desktop publishing programs. I do much less "napping" between saves of any CAD files than editing, saving, cut/pasting photos or creating documents for print/web. Now, if I am creating a 3D rendering, yeah, that demands some memory.

But unless you are making computer-animated movies (silicon graphics) or designing 100 story skyscrapers, don't worry about the CAD. Worry about the ever-increasing sizes of RAW files and the needs of the various photo-editing programs we use!
--
Brian

BorisK1 said:

From what I understand about RAID5, it's not that easy to build and
maintain. It also isn't what you'd call affordable or flexible.

Click to expand...

Modern controllers make it really easy to build a RAID5 storage: just plug-in 3 or more hard disks, create the volume and format it with your favourite file system from the operative system.

Most of them have both a bios-like interface and a windows interface to manage the array (creation, health monitoring , rebuilding etc).

Moreover since a couple of years some NAS are marketed at reasonable prices as home appliances like Infrant ReadyNAS, Buffalo Terastation, Techus, Intel etc.

These provide RAID5 and several other network services (shares, streaming, dhcp, ftp etc.) out-of-the-box and, to some extent, also some propertary technologies to upgrade online the array (bigger disks) and to synch/backup to remote units.

BorisK1 said:

It's perfect for an online mission-critical database, but for a
journalist - I doubt it.

Click to expand...

RAID5, compared to other levels, is slower in writing because parity blocks must be maintaned. Although in a home scenarios it would be overkill, for mission critical databases RAID10 if often a better choice. But for home needs, I find RAID5 a very good compromise.

Cheers,
--
Cico - Oly E-1 and C-8080 - http://www.pbase.com/cico71

I was originally looking at AMD based on this review, but the TomsHardware heavily favors the Intel.

http://reviews.cnet.com/4520-10442_7-6389077-1.html

Is there a particular motherboard chipset that runs a C2D the best?