So... I finally come to a decision to go with the OM-D... Is there any big hand users out there?

Started Mar 13, 2013 | Questions thread
texinwien
Senior MemberPosts: 2,650Gear list
Like?
Re: Issue 2: How to fairly compare output image IQ of two similar DSCs
In reply to Jack Hogan, Mar 16, 2013

Jack Hogan wrote:

I never addressed Issue 2 here, but here goes a summary of the posts in the other thread. Given Issue 1, the fact that in-camera ISOs are mostly not directly comparable, how does one then go about comparing two cameras fairly in the same scene and with the same photographic intent (blur, dof, and retained shadows and highlights)? The answer is pretty straight forward.

Place both cameras to look at the same scene, with the same lens, the same Exposure (i.e. shutter speed and f/number) and to record the same tonal range (highlights and shadows). To capture the same tonal range you could simply read the ISO difference in stops off of the DxO graphs and dial in-camera ISOs with that difference into the two cameras. That's how I'd do it, because I trust DxO for things like this. Or you could figure it out yourself as follows:

In Manual mode, set the same shutter speed and f/number in both cameras and place them in the scene. Then take a capture of a gray card with both at the same estimated nominal ISO for the desired tonal range, say 1600. Correct the ISO of one of the two while keeping Exposure fixed, iteratively, until the gray card appears in the same spot of the Raw histogram of both*, say 10% of full scale. This way both cameras will be recording the same range of incoming tones, from shadows to highlights. Having set them up to respond equally to the incoming light, you can now take a capture of the scene with both DSCs, process them with the same Raw converter parameters while ensuring that the brightness and size of the output images are more or less the same and judge their relative IQ - SNR in this discussion - fair and square.

If you were to go through this exercise with the EM5 and the G3, you'd find that you'd end up with the EM5 at in-camera ISO 3200 and the G3 at 1600 - at the same Exposure and very similar SNR (EM5 slightly better as per DxO graph).

That is exactly right, and you'd also end up with an E-M5 3200 ISO photo that got one full EV of exposure more than the E-M5 3200 ISO photo you tried to compare with DPReview's G3 ISO 1600 photo.

In other words, I agree with your test setup, but your proof didn't. As a matter of fact, I often ignore the camera ISO and expose roughly by DxO's measures, which means, in effect, that I dial in one extra EV of exposure compensation than my E-M5's ISO Setting would indicate.

And, as you might imagine, RAW files taken at ISO Setting 200 on my E-M5 and exposed based on saturation sensitivity of ~107 look even better than RAW files taken at ISO Setting 200 on my E-M5 and exposed based on the ISO Setting. Which is clear, of course, since the saturation sensitivity files get a full EV (or more if I really push) than the ISO Setting files.

And perhaps you'll realize here, if you think it through - if I expose based on the ISO setting, and the ISO setting leads to 1/2 the exposure that the saturation sensitivity would, I'm UNDEREXPOSING (on the E-M5) by one full EV (in comparison to our 'ideal' exposure, based on saturation sensitivity).

Lower exposure = more noise, right Jack? Whereas you were making the opposite claim - that this underexposure could be used by a devious camera company to fake lower noise for a camera. Ouch, that's gotta sting!?

For a practical example (the very one that invalidated your G3 vs. E-M5 'proof'), the DPReview E-M5 ISO 1600 RAW files were underexposed by one full EV from the ideal, whereas the G3 files were exposed very near to the ideal.

What you have claimed could be used to Olympus' advantage, as some way of making the camera appear to have less noise than it should, actually results in the camera producing more noise than it should (or could, if exposed ideally). That means the performance you're seeing in DPReview's tests is worse than camera's optimum best. You're looking at the worst, and it only gets better from here Jack. Do I hear you putting in an order at Amazon.com for an E-M5 over there?

And, as DxO and DPReview and everyone else who has a handle on the difference between saturation sensitivity, ISO settings and why the ISO standard is written with the conscious intention of allowing this (and other) variance(s) - leaving it up to the manufacturer - the reason Olympus would choose to expose a full EV lower than the ideal is to protect highlights at the expense of additional shadow noise.

DxOMark - Measurements - ISO Sensitivity

As tests show, the ISO settings [aka Exposure Indices or EIs] reported by camera manufacturers can differ significantly from measured ISO [a type of Saturation-based Sensitivity measure] in RAW. This difference stems from design choices, in particular the choice to keep some “headroom” to avoid saturation in the higher exposures to make it possible to recover from blown highlights.

DxOMark: RAW ISO measures are inferior to manufacturer ISOs: is this a problem?

In fact, it is precisely the JPEG ISO value [aka Exposure Index or EI] that all the manufacturers publish. ... This is absolutely legitimate: the ISO standard allows manufacturers to use this JPEG value. They are not cheating.

...

Moreover, underexposing the RAW file allows manufacturers to use their own complex algorithms to obtain a better output for the highlights while retaining good medium tones.


Note: I am posting these quotes to show that, not only does DxO understand and accept that the variances between saturation sensitivity and ISO Setting are legitimate, they also make one half of the point I made above - camera manufacturers usually have ONE REASON for introducing such a variance, and that is to protect the highlights.

And as I'm sure you know, the wider the variation between ISO Setting and saturation sensitivity, the more 'headroom' or highlight protection ability a camera will have (in comparison to the 'ideal' exposure based purely on saturation sensitivity) AND the more shadow noise the camera will have (in comparison to the same ideal).

Are you with us Jack? Are you seeing why your arguments are wrong-headed? I get the impression you were already starting to turn toward the light, since you've abandoned your invalid G3 vs E-M5 comparison, ostensibly realizing that it was fatally flawed, and with it, a crucial part of your own argument.

The further thing from my mind is suggesting to shoot evey scene in the field like this. But I am suggesting that if you wanted to compare the two cameras fairly that's one way you could do it.

That'd be awesome! As I have stated in more than one reply to you, that would be a valid addition to the tests based on camera ISO setting that most testing sites already run, as long as they maintain equivalent exposures, 'cause, woops, forgetting to do that will just mess everything right up, am I right? Jack? Am I right, or am I wrong?

I am right.

On the other hand, comparing two cameras at the same ss, f/n and in-camera ISO setting is more often than not misleading because of Issue 1.

More often than not? Remember, you gave us a flawed example with a G3 and an E-M5 that you thought offered proof of your misguided conclusions. I'm going to have to ask you to offer support for this claim (again).

So back to us. Where are my misconceptions and erroneous conclusions here?

See above.

Jack
* RawDigger is a great free program to obtain a Raw histogram from.

I downloaded it 10 months ago, just after I purchased my E-M5. I have used it regularly with files from my E-M5, as well as with RAW files from DPReview, imaging-resource.com and other test sites. I have the latest version available for my operating system. Same with RAW Therapee - also great if you want to geek out a little on this stuff.

As a matter of fact, I used Raw Digger to record black frame read noise measurements from my E-M5 in May of 2012. Measurements that Anders W, another forum member, used to estimate the camera's dynamic range. His estimates were, by the way, quite close to the measures provided by DxO some months later.

A few of us even suspected that the variance between ISO setting and saturation sensitivity on the E-M5 was between 2/3EV and 1EV, long before DxO came out with its report, and a number of us were already experimenting with exposing based on suspected saturation sensitivity long before DxO released it's (rather delayed) measurements of the E-M5.

So yeah, dunno Jack. I'm with you in a lot of ways here, and I understand where you're coming from, but you're making one or two small errors in logic that are throwing off the entire rest of your argument.

Sucks how that works - you can know the inner workings of the entire space and be 100% correct on 99.9% of your reasoning, but that 0.1% of incorrectness can cause your entire case to be invalid.

tex

 texinwien's gear list:texinwien's gear list
Canon EOS 300D Olympus OM-D E-M5 Canon EF-S 18-55mm f/3.5-5.6 Panasonic Lumix G 20mm F1.7 ASPH Panasonic Lumix G Vario 100-300mm F4-5.6 OIS +14 more
Reply   Reply with quote   Complain
Post (hide subjects)Posted by
(unknown member)
+1New
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark post MMy threads
Color scheme? Blue / Yellow