Raw is better how? 2

Joe Pa said:

With UDO storage you have two options WORM and Rewritable cartridges.
A 5 pack of 30GB Rewritable media is $357 MSRP, a 5 pack of 60GB WORM
is $350 MSRP. This works out to $2.38 per GB for ReWritable and
$1.16 per GB for WORM.

A jukebox capable of 8TB of WORM storage or 4TB of Rewritable
storage with 2 UDO drives, and 170GB of RAID Cache has an MSRP of
under $30,000 USD.

Click to expand...

Wow! That is not cheap storage! I thought that maybe these might provide an alternative to the tape backup (optical is more user friendly than tape) I am currently using (300/600GB SDLT) which is attached to a 6.2TB( 5 running raid 5) disk array.

The array uses also keeps one drive (out of 16) as a spare in case of a single drive failure (instant recovery), and reduntant power supplies. The tapes are used for backup (1 set onsite 1 set offsite) at about a cost of about $100(CDN) for a single 300/600 GB tape. Our IT department still trust tape over all other mdeia for longevity.

This services 4 computers full time plus a handfull of occasional guests.

3 years ago this cost less than $25000 CDN! With current arrays that are available you could probably double that capacity for the same money even if you don't like using drives bigger than 500GB.

Joe Pa said:

I've worked with this technology for many years and have never seen a
platter fail in a jukebox that is regularly maintained.

Click to expand...

Is this the same tech that has only been around since 2003 that I was reading about?

James

nanook s said:

What you are missing is that theoretically some people can hear the
frequencies higher than the 44kHz redbook CD sampling rate (22kHz
Nyquist). The CD its self is lossy. Likewise, the anti-aliasing
filter and Bayer filter toss away a ton of data in the capture
process. Compared to that, the additional loss introduced by JPEG
compression is miniscule. Likewise, the typical print toss away far
more info than JPEG compression process. The weakest link that
decides how strong the chain is. So long every other step tosses
away less than the print making process, there is no impact to the
final output.

Click to expand...

You are so close minded you refuse to see the obvious. Most cameras have Bayer and anti-alias filters, whatever data these filters discard is gone the moment you click the shutter button. So let's agree the data captured by the camera is a subset of what it actually saw due to these filters. The data is lost, what percent, we dont' know for sure, but whatever it is, is lost forever it's not recoverable if these filters exist on the camera.

If we agree RAW is actually a subset of all the data that the camera sees due to anti-alias filters and Bayer filters then you have to also agree that JPEG is a subsample of the RAW file as it is compressed with data loss from the original RAW file. This is the point everyone is making...there is more data in RAW than JPEG, as the JPEG data is a subset of the RAW data.

If you agree that RAW represents all of the data the camera saw after the anti-alias and Bayer filters then let's say the RAW data is "X", "Y" is the resultant JPEG data after compression, and Z is the discarded data from JPEG compression, it's reasonable to express the following Y(JPEG) = X(RAW) - Z(discarded data). Through that expression, it is clearly obvious that there is less data in a JPEG than RAW.

nanook s said:

There is another dimension involved in this issue. If the pursuit of
RAW limits one's business to older generation cameras with lower
spatial sampling rate and lower dynamic range, the RAW vs. JPG
difference can not nearly make up for the advantage brought on by
more expensive newer generation cameras, even if the latter is
shooting in JPG, so long as the exposure is reasonably correct. RAW
is not free lunch for a busy studio.

Click to expand...

I don't care about this stuff, the original question was "does RAW contain more information than JPEG?" Mathematically you cannot demonstrate when using the same camera that JPEG images contain the same amount of image data as RAW, as the nature of the algorithm is lossy.

The rest of these arguments you pose deal in practicality and work flow much like my choice to use lower quality MP3 audio instead of CD quality audio on my iPod to increase the number of songs I can store.

If you take so many pictures at a wedding you cannot process RAW then so be it. Given you take 2000+ pictures at a wedding I'd suggest you consider another concept "signal to noise" ratio. I highly doubt all 2000 of the photo's are selected by the B&G, and most would likely be redundant. By being more selective in your shots (like in the film days) you would improve your signal to noise ratio to the point where working with RAW images is possible for you.

I don't know where you get your information from, but you are very mis-informed about UDO storage and jukebox robotics. UDO jukeboxes go up to 19TB today, and the robotics if maintained hardly ever fail. JB robotics from 15 years ago were subject to failure but robotics have come a long way since then.

I'm seeing a pattern here which is if you decide to not utilize a technology it's not good for anyone. Maybe you should read up about UDO, and find out why every major corporation is using UDO and RAID to store their critical corporate and HR documents and not just RAID.

Tape is by far the cheapest archival storage, you find find any optical or magnetic storage that comes close to it in price. Tape is strictly meant for archival purposes though where as UDO is used for online and near online applications standalone or in conjunction with RAID.

UDO was introduced in 2003, and was replaced by UDO2, which has significantly increased the capacity of the media. The expectations are that the storage capacities will double again in the next two years which would significantly lower the cost per GB.

Can we drop the comments on the other person for a moment, and talk about the subject for a moment?

A (tiff or PSD) file converted from RAW (whether in camera or in a desktop) from 1DsII has 17 megpix (with rounding), each pixel has info in the three color dimensions. So that is 17*3 = 51meg supposedly independent pieces of color info. Except that's not what the camera captured. The camera sensor actually captured 8.5meg pieces of Green, 4.25meg pieces of Red, and 4.25meg pieces of Blue . . . for a total of 17meg pieces of data. So in other words, 2/3 of the data you save in a PSD or tiff file from RAW conversion is made up.

That's before sharpening, which introduces even more made-up data en masse.

Then we get to printing, which is typically about 5-6 stops in real life. All that flood of data will have to be crammed down a straw pipe. Is all that made up data being carried along in the work flow really necessary?

As for reducing the number of shots . . . well, for someone who has a professed love for preserving data points, don't you think it's just a wee bit ironic that you are suggesting capture less to begin with?

The problem is not lack of time for RAW conversion, but storage of RAW given current storage technology. I tend to be a meticulous data preserver at image/picture level, but not a meticulous data preserver at pixel level. Because I know that if a moment is not captured or preserved, it will be permanently lost . . . whereas so long as I capture enough data to make a high quality 12x18, there are ways to dress up images for the relatively rare larger prints.

Working along these lines, delivering high quality images in a timely and reliable fashion, with numerous backup plans, enable me to get the next generation of "film" (sensor) ahead of most of my competition . . . that delivers superior image with higher spatial sampling rate and dynamic range for the clients, far more than the difference between RAW vs. JPG on the previous generation of cameras. Running a business is about priorities. Do what benefits the clients the most, and they will appreciate that.

"I don't know where you get your information from, but you are very mis-informed about UDO storage and jukebox robotics."

hmm, you were the one providing info. Remember that 8T for $30k quote that you gave yesterday?

"JB robotics from 15 years ago were subject to failure but robotics have come a long way since then. "

Perhaps that's what the salesperson says. Do I really want to bet my farm on history not replaying itself?

"I'm seeing a pattern here which is if you decide to not utilize a technology it's not good for anyone."

You are imagining a pattern here. All I have been saying is that a suggested method is not good for me. As for anyone else? They can do whatever they wish. Just don't give grief about RAW being the only way to shoot for pros or UDO being the only way to storing large amount of data. There are more than one way to skin a cat.

You have not posted to this site in the last two months and in the last 6 days you have over 60 posts on this topic.

I started looking at some of your other posts. You're quite the debater. I think it is important for folks to have an outlet for frustration.

I agree with your attempt to counter the dogmatic approach to RAW shooting, it gives us balanced information.

But, at this point, do you think your advanceing your cause?

Within the hour this thread will be closed at 150. Can't say I'm sorry to see it go.

--
BrianH

timpa42 said:

Raw files are harder to recover then jpeg, I know that for a fact see
my other post, on how I know that for a fact.

Click to expand...

Well, you're probably using the wrong recovery tool. I know, for a fact, that they do work because I've used them to recover my own RAW files. And so have colleagues of mine. IN ADDITION, a little while back, I went to an Ed Pierce/PhotoVision seminar where Ed demonstrated image recovery of is RAW files by first deleting the card, then formating his card, then recovering all his RAW files using the software that he was demonstrating (don't remember which one he was demonstrating, but I use Photo Rescue). And there were about 200 people in the seminar room who witnessed this alongside me. So don't be saying that you can't recover a RAW file just like you can a JPEG file, because YOU CAN!!!

The only thing you need to do is make sure that your recovery software supports the RAW file you are recovering. For example, Photo Rescue "offers a constantly updated support for most camera's RAW file formats: CR2, RAW, RAF, CRW, NEF, ORF, MRW, etc." On top of that, "In some cases, we can even rebuild pictures that have suffered minor corruptions."

The "I lose cards" excuse is an issue of you losing cards, and it has more to do with carelessness than RAW or JPEG. Because if you "lose cards", you'll lose whatever is on them, and in your case, that would be JPEGs.

Listen, cards get lost, cameras with cards IN THEM get stolen, cards get corrupted. The whole notion of trying to fit more images onto any single card has its own data security risk. So again, this excuse holds no water.

Personally, I use a Lowepro Film Drop pouch which I've had since my film days, but that I still occassionally use for digital. Basically, it's a cleverly designed pouch that has an opening where you can stick your used film rolls into, and they don't come out. You can shake the pouch or turn the pouch upside down, and nothing will fall out. The way to get them out is by a side zipper. It was a great way to ensure you never lost a roll of film during a wedding. And I often use it for digital weddings, too, since the results are the same. You stick your memory card into the pouch, and it's virtually impossible for it to accidently fall out.

So again, your issue holds no water. You simply need better tools to make sure you don't lose ANY cards, whether they store JPEG or RAW files. The Lowepro Film Drop is a perfect example of great tool that you might look into:

http://www.lowepro.com/Products/Accessories/sliplock_cases/Film_Drop_AW_Patented.aspx

nanook s said:

Do you run around shooting weddings with primarily one camera? Using
multiple cards in one camera is not much backup because most
important group pictures are taken in quick sucession . . . chances
are that most of those will be on the same card if you use one camera
regardless how many cards you use for the rest of the day . . .
unless of course you use the previously mentioned 512meg cards

I usually shoot a wedding with 3 cameras; my second shooter usually
brings 2, and the lighting assistant usually has another one. That's
6 cameras to share the captures. The card security that I talked
about relates to the physical handling of cards. I keep all used
cards in my chest pocket inside my suit. It has capacity for about 4
cards . . . beyond which there is a real risk of falling out when I
bend over. 4 used cards in pocket, plus 6 cards in cameras . . .
that's a total of up to 10 cards from one wedding (obviously not all
of them are full). With 2-4 weddings going on in the studio on a
typical summer or fall Saturday, and another 1-2 on Sunday that's
25-60 cards for the downloader to juggle on Monday morning. You can
see why having to use too many cards would pose data security risk,
both for getting lost on the day, and for mixing up later. A 4gig
card really doesn't take that many 1DsII RAW files.

Click to expand...

I agree, debates like this can be quite tiring. I hadn't visited this site for quite a few week because I was really busy photographing. I had a little time between the shooting season and the holiday orders that are coming soon (the Thanks-giving orders were already shipped out), so I got a chance to do a bit of quick browsing earlier this week. I was quite shocked by some of the RAW dogma that floated around . . . some were making very disparaging remarks on other professionals based on very limited work experience or knowledge. It's as if some one who just learned to drive stick shift started making comments about all pedal shifts and computer controlled clutching belonged only to non-performance driving. Most F-1 drivers would take exception to that kind ignorance-based categorical statements.

Once again, please re-read my post. You are assuming way too much. I never lost a single memory card with image on it before downloading and backing up in my entire shooting life. Why? Because I always take significant steps to minimize the chance of that happening; some say drastic steps. Can I say I will never lose a card? Of course not. Nor can you. That's why I keep very strict policies regarding the handling of cards after they come out cameras. Limiting the number of cards going into the most secure location for storage (2 inch distance my heart) is one of the policies. You "occasionally" use that pouch? So you don't even have a consistent card handling policy. Do you want to explain to a bride how and why you lost her wedding pictures because you did not handle her particular wedding as carefully as you "occasionally" handle other weddings?

To guard against card corruption, I institute the following policies:

(1) Like I mentioned before (once again that phrase), a typical wedding shot by me has 6 cameras from our studio running. For the most important pictures, I capture it on at least two different cameras myself, and my second-shooter supply additional ones;

(2) I do not use card models that are entirely new on the market place. I prefer someone else try them out in the field first;

(3) I prohibit the use of microdrives;

(4) I do not use the fastest cards. The slower cards have much higher statistic re-write life span MTBF.

(5) There are very specific card handling and identification procedures that everyone at the studio is expected to follow.

nanook s said:

You "occasionally"
use that pouch? So you don't even have a consistent card handling
policy.

Click to expand...

My standard attire for weddings is a black cotten-blend shirt with black pants. The shirt has a breast pocket with a button closure. The breast pocket of these shirts easily accomodates an entire day's worth of memory cards. No loss of cards, ever. It's only when I have multiple shooters with an extremely high volume of cards in use that I employ the use of the Film Drop.

Obviously, you've never lost a card. And I've never lost a card. And any loss of a card would impact not only RAW images but also JPEG images. So to put this all on RAW is simply absurd. And given the (un)likeliness of loss of any cards by either of us, to use this as a primary reason not to sure RAW is, as any level-headed person can see, quite unjustified or weak at best. Every photographer has a system to ensure they don't lose cards, whether they shoot RAW or JPEG. To say that you don't lose cards because you shoot JPEG, or that you are more likely to lose images if you shot RAW is simply absurd because you can still lose a card if you shot JPEG (if you don't have a proper system) and you can still never lose a card if you shot RAW (if you do have a proper system). This will make sense to any rational person, but unfortunately, I don't expect it to make much sense to you.

In fact, if you did a survey of RAW-shooting professionals versus JPEG-shooting professionals, I am quite certain that you would not find any difference in accidental card losses between each camp. Furthermore, if RAW usage did create a greater data security risk, I am quite sure that National Geographic and Sports Illustrated would NOT be requesting that all their photographers shoot in RAW.

Once again you ignore the technical and mathematical details of my post to pursue a tangent on different camera's, etc.

You're obviously smart enough to know when you've lost an argument so you attempt to obfuscate the discussion to save face. You understand that RAW images contain more data than derived JPEG images as you yourself admit to using RAW when you need to "push the film", so I see no point in continuing this debate when you've already acknowledged that RAW offers advantages over JPEG due to the additional data it contains.

I'm not arguing that RAW is the only or best way to shoot, I'm arguing the technical merits of image quality comparing RAW to JPEG, and quality wise RAW IS superior to JPEG. How someone chooses to run their business or use their camera is a personal decision with alot of trade-offs.

As for the number of pictures you take during a wedding, i could care less. I was merely pointing out that 2000 pictures was alot for a wedding given the size of most wedding albums. You could probably lower the total number of shots, process in RAW and still have more than enough to fill a wedding album.

Once again, the purpose of that side excursion is??

Out of that TIFF file, if it's shot by a typical Bayer DSLR, at least 2/3 the data is made up, not captured by the sensor. Magazines as private business can institute whatever policy they wish. They probably do not want a flood of submission from cheap digicams from uncle Bob regardless what format it's in. The weeding out helps reduce the work load of the editors, who have to wade through all the submissions. Requiring RAW also accomplish two other objectives for the magazine editor:

(1) To ensure the image is not a composed fake picture; publishing that would be a scandal for National Geographic;

(2) The editor only gives few seconds to a picture before it goes into the intial selection. Because the editor has no idea what camera or work flow the photographer uses, requiring RAW helps ensuring a minimum set of standards that would be publishable when enlarged to full spread. JPG submissions could be heavily curved by the photographer due to bad initial exposure, yet not visually obvioius when the editor is looking over thumbnails.

The cold fact remains that a JPG file from today's leading DSLR's provide far more information than can be possibly printed on a magazine spread. If the magazine takes no submission at all from strangers, and control the entire capture to print process in house . . . there wouldn't be need for a submission guide line requiring RAW. In case it's not obvious, a wedding photography studio does indeed control the entire capture to print process.

I quoted an 8TB UDO jukebox, that doesn't mean 19TB isn't available, how did you reach such a conclusion???

As for jukebox reliability, I'll guarantee you the robotics in a jukebox outlast your RAID arrays, and have less failure in the same given time. I didn't get this information from a sales person, I experienced it first hand.

I never said UDO is the best storage solution for everyone, you were bragging about your RAID configuration and all the hoops you go through to keep them in sycn, I just suggested an alternative that would eliminate all the unnecessary work you do to maintain them.

nanook s said:

Once again, the purpose of that side excursion is??

Out of that TIFF file, if it's shot by a typical Bayer DSLR, at least
2/3 the data is made up, not captured by the sensor. Magazines as
private business can institute whatever policy they wish. They
probably do not want a flood of submission from cheap digicams from
uncle Bob regardless what format it's in. The weeding out helps
reduce the work load of the editors, who have to wade through all the
submissions. Requiring RAW also accomplish two other objectives for
the magazine editor:

(1) To ensure the image is not a composed fake picture; publishing
that would be a scandal for National Geographic;

(2) The editor only gives few seconds to a picture before it goes
into the intial selection. Because the editor has no idea what
camera or work flow the photographer uses, requiring RAW helps
ensuring a minimum set of standards that would be publishable when
enlarged to full spread. JPG submissions could be heavily curved by
the photographer due to bad initial exposure, yet not visually
obvioius when the editor is looking over thumbnails.

The cold fact remains that a JPG file from today's leading DSLR's
provide far more information than can be possibly printed on a
magazine spread. If the magazine takes no submission at all from
strangers, and control the entire capture to print process in house .
. . there wouldn't be need for a submission guide line requiring RAW.
In case it's not obvious, a wedding photography studio does indeed
control the entire capture to print process.

Click to expand...

You still don't understand RAW, do you? I HIGHLY recommend you pick up a good book on RAW, or read any one of the countless articles about RAW that are in the web. Yes, indeed, whole books have been written about RAW. Many of the greatest minds in digital photography have written excellent books and articles on RAW. Look up Bruce Fraser, Martin Evening, Kevin Ames, Jeff Schewe, etc. It's clear that speaking to someone as ill-informed about RAW as you are is simply an excercise in futiility.

And maybe after you've done some reading, you can argue with Bruce Fraser, Jeff Schewe, David Blatner, Martin Evening, Rob Sheppard, and many other extremely experienced digital photographers about how pointless and needless RAW is for even the most experienced photographers, or that it's only something to be used for rare occassions or that it's only for inexperienced photographers who have weak photographic skills or that it's mainly for poor photographers who need a "crutch" because they are not confident in their metering skills. Go for it! I'm sure you'll really teach them something (or not)!

Here are my positions:

1. RAW contains more data than a JPG file from the same camera. RAW and JPG from different cameras however are not comparable. Not all photographers work with the same camera in the world. Different workflows affect the ability to afford different equipment.

2. When one is looking at RAW file on display, or TIFF or PSD derived from it, at least 2/3 the information is artificially made up, not captured.

3. Even JPG files contain far more information than can be printed out, for overwhelming majority of print orders. Since RAW conversion too is inventing more than 2/3 the information, it's no great sin to use fractal interpolation for the rare humongous prints . . . just like "highlight/shadow recover" tools in "raw converters" do already anyway.

4. Second to the limitation imposed by printing, the next biggest information loss in the capture-to-print-making process is the spatial sampling frequency, and the imposition of physical low-pass filters. They impose far more information loss than the JPG software filter. So the most important decision a digital photographer can make regarding image quality is the acquisition of the greatest camera sensor with high spatial frequency and high dynamic range . . . assuming of course the lenses are perfect (lenses are a separate topic).

5. If maintaining a RAW workflow over JPG, which is more than sufficient for print making, gets in the way of affording the best camera/sensor, focusing on RAW workflow becomes a liability. The clients would be far better served if the photographer can get exposure right, and save the time and resources to buy better cameras . . . and ensure that all the pictures that are captured are reliably backed up for the long term.

I'm glad you are backing off from earlier contention that I should shoot less.

So, are you giving me a 19TB UDO system for free? You quoted 8TB UDO for $30,000, so that's the set of numbers that I used for comparison. I have an annual equipment budget of $25-30k, out of a studio revenue about 10 times that. That's sufficient to keep multiple top flight cameras like 1DsII upgraded every 18-24 months. Somehow I don't think it's a wise investment to put the resources in a system that will in all likelihood be surpassed by RAID system that cost 90% less. Not to mention RAID gives much much quicker access and update speed.

See, in the real world, running a business is about allocating the resources that clients give us to the most efficient use . . . so the clients can get the best bang for their buck. Somehow, I have a feeling that UDO as it currently stands is similar to the all-RAW theory: assuming all other objectives and constraints do not exist, and nothing else matters. Perhaps someday when I get a big government contract, I may try that

What's best for you and your budget doesn't make it best for everyone with different budgets. Someone less successful than you might consider your 3 servers with RAID arrays and the synching process you go through to be too excessive and might instead use external drives or DVD backups.

I offered an alternative solution that would provide you with better archival characteristics and reliability.

With the exception of Point 5, we have finally reached an agreement. I don't subscribe to the latest and greatest camera yields the best results, but it's off topic and not worth discussing. Despite some frustration it was interesting having the discussion with you.