Exposure basics, lesson two point one (& ISO)

Started Mar 19, 2013 | Discussions thread
Anders W
Forum ProPosts: 18,755Gear list
Like?
Re: It's about useful comparisons - not which is better
In reply to Jack Hogan, Mar 20, 2013

Jack Hogan WQrote: OK, but in fact the G3 was the one properly set up for the assignement. So the EM5 recorded 1 additional stop of highlights that were not called for at the expense of 1 stop of shadows that instead were called for (ok 2/3 of a stop to be precise). You are fired

AndersW wrote: If you think differently, then please answer the following question: If exposure is kept constant, as it is in the scenario you outlined, the amount of light hitting the sensor is exactly the same regardless of whether the camera is set to ISO 1600 or ISO 3200. So on what grounds would shadow noise be worse in the first case than in the second?

I see your point, bad example then. The point is not favoring one or the other in a comparison in order to have useful apples-to-apples data.

And so with the other questions. AndersW, if you've followed my posts you know that they are only about two issues: that manufacturers label in-camera ISOs inconsistently and that therefore it is often misleading to compare two cameras' performance at the same in-camera ISO aotbe. I did not bring up the EM5-G3 example, ultimitsu did in a thread titled 'DxOMark's measured ISOs vs. manufacturer ISOs', four threads ago. I merely chimed in with my opinion and for some reason texinwein and mjancor have been chasing me ever since.

I don't care about the EM5 vs the G3 (I am a DSLR guy), what I care about is fair comparisons.

I realize that. But my point is that you have in some important ways misunderstood what you refer to as an inconsistency in ISO labeling and therefore also misunderstood in some important ways what it takes to make a comparison fair. My response addressed one of these misunderstandings.

This was the example that brought this discussion to this point, on the effect that inconsistent ISO labelling across cameras and manufacturers can have on a naive evaluation of the SNR performance of two cameras :

Noise performance when both cameras are set to the same in-camera ISO = 3200

Noise performance when both camera's ISO dials are set so that they will produce the same mean raw value for the same signal*

My original opinion , which I continue to hold, is that it appears to me that the more useful of the two is the bottom one, but that most people would not know it. In fact my sole mentioning it four threads ago has resulted in flaming that is still ongoing.

Cheers,
Jack

*From DPR's comparometer and trusting DxO, which I generally do. In fact the EM5 is penalized by 1/3 of a stop less light, but you get my point.

OK. So let's skip your example with the sports game and instead focus on the one you present here (and in the post you label "my original opinion").

What you are saying, judging by the caption of the second image, is that when we compare cameras, we should do so at the (potentially different) camera ISOs where they produce the same mean raw value for the same signal.

By the same signal, I suppose you mean the same exposure (same amount of light per area unit on the sensor). If not, let me know what you mean.

Provided that I have understood your use of the word "signal" right, I have nothing against the criterion as such (which is effectively the one used by DxOMark in their comparisons)

Agreed. Exposure as a certain number of photons hitting the sensor.

although there are others that I also find acceptable.

But what makes you think that the signal, as I defined it above, is the same (or the same within one 1/3 EV) when comparing the DPR studio scene sample for the E-M5 at a camera ISO of 3200 with that of the G3 at a camera ISO of 1600.

The EM5 at in-camera ISO 3200 is set at f/6.3, 1/800, while the G3 at in-camera ISO 1600 is set at f/6.3, 1/640, a difference of 1/3 of a stop. Both cameras appear to use the same lens. So the question of lighting comes in. Is it relatively constant? If it is then, there is a 1/3 of a stop difference in exposure. Since both captures look equally bright, there is an apparent 0.7 stop difference in the relative ISO labelling.

I know for a fact that the signal is not the same in that comparison and am also in a position to prove that claim.

I additionally know, and can prove, that the DPR studio scene samples offer no set of camera ISOs such as to meet your preferred criterion. However, they do offer one set of camera ISOs such as to meet another criterion that I find acceptable, namely one in which the cameras are given the same signal (as I defined the word signal above) although the mean raw values are not the same. With respect to the E-M5 and the G3, the DPR comparison that comes closest to meeting that criterion is the one where the cameras are set to the same ISO, e.g., ISO 3200.

Going through the same exercise as above but assuming that DPR fiddles with the lights so that the same number of photons hit each sensor: The EM5 is at f/6.3, 1/800 and the G3 at f/6.3, 1/1300 at in-camera ISO 3200. DPR would have had to change lighting by 0.7 of a stop between the two tests in order to make sure that their sensors were equally exposed. And given that both images seem to have approximately the same brightness that would mean a 0.7 stop difference in relative ISO labelling.

Am I missing something?

DPR makes no attempt to keep the light level in the studio used for their studio scene samples constant from one camera review to another. It follows that nothing about the effective exposure (amount of light per sensor area unit) can be inferred from the f-stop and shutter speed used. As a rule, differences with regard to shutter speed, as in the case of the E-M5 and the G3, are simply a way to compensate for different studio light levels.

How you can check, without taking DPR's word for it, whether two cameras actually got the same signal (same amount of light per sensor area unit) at the same camera ISO setting is explained here:

http://forums.dpreview.com/forums/post/51123359

As you can see, the criterion of equal signal (same amount of light per sensor area unit) at the same camera ISO setting (I took ISO 200 as an example) was met for the E-M5 and the G3 within an error margin of 1/8 EV. Given that such is the case, what is wrong with comparing the sample images at the same camera ISO setting?

It follows logically from the above, that if we compare the studio scene samples with the E-M5 set to ISO 3200 and the G3 to ISO 1600, as you propose, the G3 has received nearly twice the signal of the E-M5. So that comparison certainly does not meet your "same signal" requirement. Given that such is the case, what would be the reason to prefer it?

 Anders W's gear list:Anders W's gear list
Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Olympus E-M1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS Panasonic Lumix G Vario 7-14mm F4 ASPH +21 more
Reply   Reply with quote   Complain
Post (hide subjects)Posted by
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark post MMy threads
Color scheme? Blue / Yellow