Exposure basics, lesson two point one (& ISO)

Started Mar 19, 2013 | Discussions
bobn2
bobn2 Forum Pro • Posts: 60,885
Re: It's about useful comparisons - not which is better
1

I additionally know, and can prove, that the DPR studio scene samples offer no set of camera ISOs such as to meet your preferred criterion. However, they do offer one set of camera ISOs such as to meet another criterion that I find acceptable, namely one in which the cameras are given the same signal (as I defined the word signal above) although the mean raw values are not the same.

I don't think that they do do that. DPR exposes as I understand for a fix point in the sRGB output - that is that 18% (though they never specify 18%) on their step wedge gives the appropriate value in the JPEG file. You have to relate that back to the effective (meter ignored) ISO value that they discovered (and do not always report) in their ISO 'sensitivity' test.

In general you are right that you have to check the outcome of the DPR ISO accuracy test reported in the "noise and noise reduction" section of each review. If a camera if off by a certain amount in that test, DPR procedures imply that the signal (exposure) will be off by the same amount when the studio scene samples are shot.

Speaking specifically about the two cameras at issue here, the E-M5 and the G3, the latter passed the ISO accuracy test without discrepany whereas the E-M5 was reported to be 1/3 EV off in the direction of "too dark". Consequently, one would expect the E-M5 to have enjoyed the extra benefit of 1/3 EV more exposure when the studio scene samples were shot. However, I have now, for one reason or another, taken the trouble to check for the actual presence of that extra 1/3 EV in comparison with three other cameras (GH2, D800, GH3), the first two of which known to pass the DPR ISO accuracy test without any discrepancy and the third not being reported yet, without finding any trace of it. You find the latest such comparison, which include a description of the methods used, here:

http://forums.dpreview.com/forums/post/51023463

Yes, I saw that. It puts an element of doubt there, doesn't it? If DPReview do as they say, the E-M5 should have had 1/3 more exposure, yet you found that it didn't. So either they are not doing as they say, or there is sufficient error in your method that 1/3rd of a stop is not registered. I have to say, I have more confidence in you than the other party.

-- hide signature --

Bob

Anders W Forum Pro • Posts: 21,468
Re: It's about useful comparisons - not which is better

I additionally know, and can prove, that the DPR studio scene samples offer no set of camera ISOs such as to meet your preferred criterion. However, they do offer one set of camera ISOs such as to meet another criterion that I find acceptable, namely one in which the cameras are given the same signal (as I defined the word signal above) although the mean raw values are not the same.

I don't think that they do do that. DPR exposes as I understand for a fix point in the sRGB output - that is that 18% (though they never specify 18%) on their step wedge gives the appropriate value in the JPEG file. You have to relate that back to the effective (meter ignored) ISO value that they discovered (and do not always report) in their ISO 'sensitivity' test.

Those ISO values are generally the same (within 1/6th of a stop) though, aren't they. And they do report if a camera is outside that, such as the OM-D at 1/3 of a stop.

Correct! With regard to the deviation of 1/3 EV for the E-M5, see here:

http://forums.dpreview.com/forums/post/51122252

 Anders W's gear list:Anders W's gear list
Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Olympus E-M1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS Panasonic Lumix G Vario 7-14mm F4 ASPH +28 more
Detail Man
Detail Man Forum Pro • Posts: 17,026
Re: It's about useful comparisons - not which is better

I additionally know, and can prove, that the DPR studio scene samples offer no set of camera ISOs such as to meet your preferred criterion. However, they do offer one set of camera ISOs such as to meet another criterion that I find acceptable, namely one in which the cameras are given the same signal (as I defined the word signal above) although the mean raw values are not the same.

I don't think that they do do that. DPR exposes as I understand for a fix point in the sRGB output - that is that 18% (though they never specify 18%) on their step wedge gives the appropriate value in the JPEG file. You have to relate that back to the effective (meter ignored) ISO value that they discovered (and do not always report) in their ISO 'sensitivity' test.

Furthermore, the viewer of the DPR SCT RAW thumbnails is seeing Adobe ACR's RAW-level post-scaling applied without regard for individual deviations which may exist between cameras at either the same, or differing, ISO settings, as well as the possibility existing that DPR "tweaked the brightness" - which, in the case of the G3 test-shots processed using ACR 6.x, means the ACR 6.x "Brightness" control which is in fact a Gamma control (not a RAW-scaling control), and thus (when adjusted from the default value of 50) alters the apparent SNR of the image. Food for thought ?

See Andy Westlake's statement here: http://forums.dpreview.com/forums/post/40965292

Whether two cameras got the same exposure at the same camera ISO is nothing we have to speculate about DM. We can just check it. See here:

http://forums.dpreview.com/forums/post/51122252

Yes, I am familiar with that bit. My post was about other things that may affect the DPR SCT RAW thumbnails as they are presented, and at what display luminance levels image-noise may appear. I recognize that this is an issue restricted to the practice of casual reliance upon those thumbnails - as one is free (with the downloadable RAWs) to eschew Adobe's potential proprietary idiosynchracies, and use RAW processors that are not silently messing around in numerous (and largely unknown to users) respects with the RAW conversion and processing. Only "Adobe-ites" seem satisfied with that.

Anders W Forum Pro • Posts: 21,468
Re: It's about useful comparisons - not which is better

I additionally know, and can prove, that the DPR studio scene samples offer no set of camera ISOs such as to meet your preferred criterion. However, they do offer one set of camera ISOs such as to meet another criterion that I find acceptable, namely one in which the cameras are given the same signal (as I defined the word signal above) although the mean raw values are not the same.

I don't think that they do do that. DPR exposes as I understand for a fix point in the sRGB output - that is that 18% (though they never specify 18%) on their step wedge gives the appropriate value in the JPEG file. You have to relate that back to the effective (meter ignored) ISO value that they discovered (and do not always report) in their ISO 'sensitivity' test.

In general you are right that you have to check the outcome of the DPR ISO accuracy test reported in the "noise and noise reduction" section of each review. If a camera if off by a certain amount in that test, DPR procedures imply that the signal (exposure) will be off by the same amount when the studio scene samples are shot.

Speaking specifically about the two cameras at issue here, the E-M5 and the G3, the latter passed the ISO accuracy test without discrepany whereas the E-M5 was reported to be 1/3 EV off in the direction of "too dark". Consequently, one would expect the E-M5 to have enjoyed the extra benefit of 1/3 EV more exposure when the studio scene samples were shot. However, I have now, for one reason or another, taken the trouble to check for the actual presence of that extra 1/3 EV in comparison with three other cameras (GH2, D800, GH3), the first two of which known to pass the DPR ISO accuracy test without any discrepancy and the third not being reported yet, without finding any trace of it. You find the latest such comparison, which include a description of the methods used, here:

http://forums.dpreview.com/forums/post/51023463

Yes, I saw that. It puts an element of doubt there, doesn't it? If DPReview do as they say, the E-M5 should have had 1/3 more exposure, yet you found that it didn't. So either they are not doing as they say, or there is sufficient error in your method that 1/3rd of a stop is not registered. I have to say, I have more confidence in you than the other party.

Yes, I agree that this "missing" 1/3 EV adds an element of doubt with regard to the accuracy of DPR procedures, although I wouldn't call it a major one.

A main point here, as I see it, is that I haven't yet encountered any case where the comparisons provided by the DPR studio scene samples are seriously off in the sense that one camera has been given significantly more exposure than another at the same camera ISO setting.

Another main point is that we don't have to speculate about the matter. We can just check the RAWs by means of RawDigger as I did and then relate that information to the DxO measured ISOs in the manner I did in the post I linked to. By that means, we can know whether two cameras got the same exposure at the same camera ISO regardless of what DPR says about the matter.

The method I use rests on the assumption that we can trust the DxO "measured ISOs". But in the end, we don't have to do that either. If we want, we can just test the ADU response of two cameras to the same exposure at the same camera ISO ourselves. We are not really in the hands of other parties here.

 Anders W's gear list:Anders W's gear list
Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Olympus E-M1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS Panasonic Lumix G Vario 7-14mm F4 ASPH +28 more
Anders W Forum Pro • Posts: 21,468
Re: It's about useful comparisons - not which is better

I additionally know, and can prove, that the DPR studio scene samples offer no set of camera ISOs such as to meet your preferred criterion. However, they do offer one set of camera ISOs such as to meet another criterion that I find acceptable, namely one in which the cameras are given the same signal (as I defined the word signal above) although the mean raw values are not the same.

I don't think that they do do that. DPR exposes as I understand for a fix point in the sRGB output - that is that 18% (though they never specify 18%) on their step wedge gives the appropriate value in the JPEG file. You have to relate that back to the effective (meter ignored) ISO value that they discovered (and do not always report) in their ISO 'sensitivity' test.

Furthermore, the viewer of the DPR SCT RAW thumbnails is seeing Adobe ACR's RAW-level post-scaling applied without regard for individual deviations which may exist between cameras at either the same, or differing, ISO settings, as well as the possibility existing that DPR "tweaked the brightness" - which, in the case of the G3 test-shots processed using ACR 6.x, means the ACR 6.x "Brightness" control which is in fact a Gamma control (not a RAW-scaling control), and thus (when adjusted from the default value of 50) alters the apparent SNR of the image. Food for thought ?

See Andy Westlake's statement here: http://forums.dpreview.com/forums/post/40965292

Whether two cameras got the same exposure at the same camera ISO is nothing we have to speculate about DM. We can just check it. See here:

http://forums.dpreview.com/forums/post/51122252

Yes, I am familiar with that bit. My post was about other things that may affect the DPR SCT RAW thumbnails as they are presented, and at what display luminance levels image-noise may appear. I recognize that this is an issue restricted to the practice of casual reliance upon those thumbnails - as one is free (with the downloadable RAWs) to eschew Adobe's potential proprietary idiosynchracies, and use RAW processors that are not silently messing around in numerous (and largely unknown to users) respects with the RAW conversion and processing. Only "Adobe-ites" seem satisfied with that.

OK. I see your point. But as you say, one can always download the RAWs and play with them oneself. What you would want to know in that case is simply whether the exposure was the same and we can check that by means of the methods I have used.

 Anders W's gear list:Anders W's gear list
Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Olympus E-M1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS Panasonic Lumix G Vario 7-14mm F4 ASPH +28 more
Martin.au
Martin.au Forum Pro • Posts: 13,248
Re: It's about useful comparisons - not which is better

Don't forget, with error bars of 1/6th, the gap betwetwo the OM-D and the G3 could be anything from 0 to 2/3s of a stop. Given the results so far I'd be guessing it may be smaller than 1/3.

 Martin.au's gear list:Martin.au's gear list
Olympus OM-D E-M5 Olympus E-M1 II Olympus M.Zuiko Digital ED 9-18mm F4.0-5.6 Panasonic Lumix G Fisheye 8mm F3.5 Olympus M.Zuiko Digital ED 12-50mm 1:3.5-6.3 EZ +7 more
Anders W Forum Pro • Posts: 21,468
Re: It's about useful comparisons - not which is better

Don't forget, with error bars of 1/6th, the gap betwetwo the OM-D and the G3 could be anything from 0 to 2/3s of a stop. Given the results so far I'd be guessing it may be smaller than 1/3.

The DPR RAW comparison widget doesn't work at the moment. Otherwise I could check. But unless there is something fishy about the G3 shots, the difference in exposure between that camera and the E-M5 should be very close to zero at the same camera ISO setting. As I pointed out in earlier posts, I have already checked the E-M5 against three other cameras (GH2, D800, GH3) and in all cases found the exposure at the same camera ISO to be practically the same.

 Anders W's gear list:Anders W's gear list
Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Olympus E-M1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS Panasonic Lumix G Vario 7-14mm F4 ASPH +28 more
OP texinwien Veteran Member • Posts: 3,326
Clarification

texinwien wrote:

what we still disagree on is what the real-world effects of inconsistent ISO labeling might be. Obviously, the example you've been parroting for a couple of days as supportive of your contention that a camera manufacturer could use 'inconsistent ISO labeling' in a way that 'results in them looking better than the competition' is false, as you've (finally) admitted.

Could it be that the contention you were trying to support with that example was also false? Yes Jack, it certainly could. As a matter of fact, it most certainly is.

I would like to make it clear that I have not seen Mr. Hogan offer a single explanation of the mechanism by which a manufacturer could use 'inconsistent ISO labeling' in a way that 'results in them looking better than the competition'.

It seems utterly clear to me (based on things Mr. Hogan has said, and things he has not said) that the labeling, itself, is the mechanism. Mr. Hogan seems to think that 'inconsistent ISO labeling' (specifically toward more headroom / highlight protection) is all it takes for a camera manufacturer to make its cameras produce results that are better than those of the competition. Quaint, but bunk

I think that's clear to most of us who are involved in this particular discussion, so far. One last person to convince, and we can move on to subjects that are actually interesting, rather than rehashing this same old discussion.

tex

 texinwien's gear list:texinwien's gear list
Panasonic Lumix DMC-GM5 Olympus E-M5 II Olympus 12-40mm F2.8 OnePlus One Canon EOS 300D +20 more
Anders W Forum Pro • Posts: 21,468
Re: It's about useful comparisons - not which is better
1

Don't forget, with error bars of 1/6th, the gap betwetwo the OM-D and the G3 could be anything from 0 to 2/3s of a stop. Given the results so far I'd be guessing it may be smaller than 1/3.

The DPR RAW comparison widget is working now so I ran the check for equal exposure of the E-M5 versus the G3 studio scene samples with both cameras set to ISO 200.

At that camera ISO, the G3 has a DxO "measured ISO" of 187 versus 107 for the E-M5. If the two cameras have received the same amount of light on the sensor, the ratio between the "measured ISOs" should be the same as the ratio between the saturation levels of the RAWs. The saturation levels can be measured (by downloading the RAW files and checking them out via RawDigger) as the green channel average (since the green channel is the strongest) divided by the clipping point. The G3 saturation level thus measured is 0.155 and that of the E-M5 0.097.

The ratio between the "measured ISOs" is 187/107 = 1.75 and the ratio between the saturation levels 0.155/0.097 = 1.60. It follows that the G3 received slightly less light on the sensor than the E-M5 but that the discrepancy is small (about 1/8 EV).

 Anders W's gear list:Anders W's gear list
Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Olympus E-M1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS Panasonic Lumix G Vario 7-14mm F4 ASPH +28 more
Jack Hogan Veteran Member • Posts: 6,982
Re: It's about useful comparisons - not which is better

Jack Hogan WQrote: OK, but in fact the G3 was the one properly set up for the assignement. So the EM5 recorded 1 additional stop of highlights that were not called for at the expense of 1 stop of shadows that instead were called for (ok 2/3 of a stop to be precise). You are fired

AndersW wrote: If you think differently, then please answer the following question: If exposure is kept constant, as it is in the scenario you outlined, the amount of light hitting the sensor is exactly the same regardless of whether the camera is set to ISO 1600 or ISO 3200. So on what grounds would shadow noise be worse in the first case than in the second?

I see your point, bad example then. The point is not favoring one or the other in a comparison in order to have useful apples-to-apples data.

And so with the other questions. AndersW, if you've followed my posts you know that they are only about two issues: that manufacturers label in-camera ISOs inconsistently and that therefore it is often misleading to compare two cameras' performance at the same in-camera ISO aotbe. I did not bring up the EM5-G3 example, ultimitsu did in a thread titled 'DxOMark's measured ISOs vs. manufacturer ISOs', four threads ago. I merely chimed in with my opinion and for some reason texinwein and mjancor have been chasing me ever since.

I don't care about the EM5 vs the G3 (I am a DSLR guy), what I care about is fair comparisons.

I realize that. But my point is that you have in some important ways misunderstood what you refer to as an inconsistency in ISO labeling and therefore also misunderstood in some important ways what it takes to make a comparison fair. My response addressed one of these misunderstandings.

This was the example that brought this discussion to this point, on the effect that inconsistent ISO labelling across cameras and manufacturers can have on a naive evaluation of the SNR performance of two cameras :

Noise performance when both cameras are set to the same in-camera ISO = 3200

Noise performance when both camera's ISO dials are set so that they will produce the same mean raw value for the same signal*

My original opinion , which I continue to hold, is that it appears to me that the more useful of the two is the bottom one, but that most people would not know it. In fact my sole mentioning it four threads ago has resulted in flaming that is still ongoing.

Cheers,
Jack

*From DPR's comparometer and trusting DxO, which I generally do. In fact the EM5 is penalized by 1/3 of a stop less light, but you get my point.

OK. So let's skip your example with the sports game and instead focus on the one you present here (and in the post you label "my original opinion").

What you are saying, judging by the caption of the second image, is that when we compare cameras, we should do so at the (potentially different) camera ISOs where they produce the same mean raw value for the same signal.

By the same signal, I suppose you mean the same exposure (same amount of light per area unit on the sensor). If not, let me know what you mean.

Provided that I have understood your use of the word "signal" right, I have nothing against the criterion as such (which is effectively the one used by DxOMark in their comparisons)

Agreed.  Exposure as a certain number of photons hitting the sensor.

although there are others that I also find acceptable.

But what makes you think that the signal, as I defined it above, is the same (or the same within one 1/3 EV) when comparing the DPR studio scene sample for the E-M5 at a camera ISO of 3200 with that of the G3 at a camera ISO of 1600.

The EM5 at in-camera ISO 3200 is set at f/6.3, 1/800, while the G3 at in-camera ISO 1600 is set at f/6.3, 1/640, a difference of 1/3 of a stop.  Both cameras appear to use the same lens.  So the question of lighting comes in.  Is it relatively constant?  If it is then, there is a 1/3 of a stop difference in exposure.  Since both captures look equally bright, there is an apparent 0.7 stop difference in the relative ISO labelling.

I know for a fact that the signal is not the same in that comparison and am also in a position to prove that claim.

I additionally know, and can prove, that the DPR studio scene samples offer no set of camera ISOs such as to meet your preferred criterion. However, they do offer one set of camera ISOs such as to meet another criterion that I find acceptable, namely one in which the cameras are given the same signal (as I defined the word signal above) although the mean raw values are not the same. With respect to the E-M5 and the G3, the DPR comparison that comes closest to meeting that criterion is the one where the cameras are set to the same ISO, e.g., ISO 3200.

Going through the same exercise as above but assuming that DPR fiddles with the lights so that the same number of photons hit each sensor: The EM5 is at f/6.3, 1/800 and the G3 at f/6.3, 1/1300 at in-camera ISO 3200. DPR would have had to change lighting by 0.7 of a stop between the two tests in order to make sure that their sensors were equally exposed.   And given that both images seem to have approximately the same brightness that would mean a 0.7 stop difference in relative ISO labelling.

Am I missing something?

OP texinwien Veteran Member • Posts: 3,326
Re: It's about useful comparisons - not which is better

Jack Hogan wrote:

Am I missing something?

Indeed, you are, and it's already been discussed, in detail, right here. Raw Digger - you've heard of it, right?

We're getting close. Epiphany is around the corner.

tex

 texinwien's gear list:texinwien's gear list
Panasonic Lumix DMC-GM5 Olympus E-M5 II Olympus 12-40mm F2.8 OnePlus One Canon EOS 300D +20 more
Anders W Forum Pro • Posts: 21,468
Re: It's about useful comparisons - not which is better
2

Jack Hogan WQrote: OK, but in fact the G3 was the one properly set up for the assignement. So the EM5 recorded 1 additional stop of highlights that were not called for at the expense of 1 stop of shadows that instead were called for (ok 2/3 of a stop to be precise). You are fired

AndersW wrote: If you think differently, then please answer the following question: If exposure is kept constant, as it is in the scenario you outlined, the amount of light hitting the sensor is exactly the same regardless of whether the camera is set to ISO 1600 or ISO 3200. So on what grounds would shadow noise be worse in the first case than in the second?

I see your point, bad example then. The point is not favoring one or the other in a comparison in order to have useful apples-to-apples data.

And so with the other questions. AndersW, if you've followed my posts you know that they are only about two issues: that manufacturers label in-camera ISOs inconsistently and that therefore it is often misleading to compare two cameras' performance at the same in-camera ISO aotbe. I did not bring up the EM5-G3 example, ultimitsu did in a thread titled 'DxOMark's measured ISOs vs. manufacturer ISOs', four threads ago. I merely chimed in with my opinion and for some reason texinwein and mjancor have been chasing me ever since.

I don't care about the EM5 vs the G3 (I am a DSLR guy), what I care about is fair comparisons.

I realize that. But my point is that you have in some important ways misunderstood what you refer to as an inconsistency in ISO labeling and therefore also misunderstood in some important ways what it takes to make a comparison fair. My response addressed one of these misunderstandings.

This was the example that brought this discussion to this point, on the effect that inconsistent ISO labelling across cameras and manufacturers can have on a naive evaluation of the SNR performance of two cameras :

Noise performance when both cameras are set to the same in-camera ISO = 3200

Noise performance when both camera's ISO dials are set so that they will produce the same mean raw value for the same signal*

My original opinion , which I continue to hold, is that it appears to me that the more useful of the two is the bottom one, but that most people would not know it. In fact my sole mentioning it four threads ago has resulted in flaming that is still ongoing.

Cheers,
Jack

*From DPR's comparometer and trusting DxO, which I generally do. In fact the EM5 is penalized by 1/3 of a stop less light, but you get my point.

OK. So let's skip your example with the sports game and instead focus on the one you present here (and in the post you label "my original opinion").

What you are saying, judging by the caption of the second image, is that when we compare cameras, we should do so at the (potentially different) camera ISOs where they produce the same mean raw value for the same signal.

By the same signal, I suppose you mean the same exposure (same amount of light per area unit on the sensor). If not, let me know what you mean.

Provided that I have understood your use of the word "signal" right, I have nothing against the criterion as such (which is effectively the one used by DxOMark in their comparisons)

Agreed. Exposure as a certain number of photons hitting the sensor.

although there are others that I also find acceptable.

But what makes you think that the signal, as I defined it above, is the same (or the same within one 1/3 EV) when comparing the DPR studio scene sample for the E-M5 at a camera ISO of 3200 with that of the G3 at a camera ISO of 1600.

The EM5 at in-camera ISO 3200 is set at f/6.3, 1/800, while the G3 at in-camera ISO 1600 is set at f/6.3, 1/640, a difference of 1/3 of a stop. Both cameras appear to use the same lens. So the question of lighting comes in. Is it relatively constant? If it is then, there is a 1/3 of a stop difference in exposure. Since both captures look equally bright, there is an apparent 0.7 stop difference in the relative ISO labelling.

I know for a fact that the signal is not the same in that comparison and am also in a position to prove that claim.

I additionally know, and can prove, that the DPR studio scene samples offer no set of camera ISOs such as to meet your preferred criterion. However, they do offer one set of camera ISOs such as to meet another criterion that I find acceptable, namely one in which the cameras are given the same signal (as I defined the word signal above) although the mean raw values are not the same. With respect to the E-M5 and the G3, the DPR comparison that comes closest to meeting that criterion is the one where the cameras are set to the same ISO, e.g., ISO 3200.

Going through the same exercise as above but assuming that DPR fiddles with the lights so that the same number of photons hit each sensor: The EM5 is at f/6.3, 1/800 and the G3 at f/6.3, 1/1300 at in-camera ISO 3200. DPR would have had to change lighting by 0.7 of a stop between the two tests in order to make sure that their sensors were equally exposed. And given that both images seem to have approximately the same brightness that would mean a 0.7 stop difference in relative ISO labelling.

Am I missing something?

DPR makes no attempt to keep the light level in the studio used for their studio scene samples constant from one camera review to another. It follows that nothing about the effective exposure (amount of light per sensor area unit) can be inferred from the f-stop and shutter speed used. As a rule, differences with regard to shutter speed, as in the case of the E-M5 and the G3, are simply a way to compensate for different studio light levels.

How you can check, without taking DPR's word for it, whether two cameras actually got the same signal (same amount of light per sensor area unit) at the same camera ISO setting is explained here:

http://forums.dpreview.com/forums/post/51123359

As you can see, the criterion of equal signal (same amount of light per sensor area unit) at the same camera ISO setting (I took ISO 200 as an example) was met for the E-M5 and the G3 within an error margin of 1/8 EV. Given that such is the case, what is wrong with comparing the sample images at the same camera ISO setting?

It follows logically from the above, that if we compare the studio scene samples with the E-M5 set to ISO 3200 and the G3 to ISO 1600, as you propose, the G3 has received nearly twice the signal of the E-M5. So that comparison certainly does not meet your "same signal" requirement. Given that such is the case, what would be the reason to prefer it?

 Anders W's gear list:Anders W's gear list
Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Olympus E-M1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS Panasonic Lumix G Vario 7-14mm F4 ASPH +28 more
OP texinwien Veteran Member • Posts: 3,326
Of course, i already told you this

It's a shame, Jack, that you didn't listen to me a few days ago, when I explained this all to you, politely . No, you were just a ball of hubris-fuelled ad hominem. and appeals to (false) authority. I tried, Jack.

The 'deeply flawed' illustration to which I refer in the above-linked post is exactly the one you've provided again in this post, and I told you exactly why it was flawed 4 days ago. 4 days you've been stuck on this and similar invalid examples, refusing to see the light.

Oh well

tex

 texinwien's gear list:texinwien's gear list
Panasonic Lumix DMC-GM5 Olympus E-M5 II Olympus 12-40mm F2.8 OnePlus One Canon EOS 300D +20 more
Detail Man
Detail Man Forum Pro • Posts: 17,026
Re: Of course, i already told you this

... you were just a ball of hubris-fuelled ad hominem. and appeals to (false) authority. I tried, Jack.

I myself see Jack as an intelligent, knowledgeable, and a reasonable fellow to communicate with. There are times when I do find that I may see some technical things somewhat differently than him, but have never known him to in my view be anything of the sort of what you describe above.

None of us are always and invariably immaculate in our reasoning:

http://forums.dpreview.com/forums/post/41980728

Even when we do learn and progress we do not become invincible authorities.

Vigorous technical debate is one thing. I rarely make comments surrounding tone of communication of others, but find such distinct, ongoing personalizations both unfortunate and in very bad taste.

OP texinwien Veteran Member • Posts: 3,326
Re: Of course, i already told you this

... you were just a ball of hubris-fuelled ad hominem. and appeals to (false) authority. I tried, Jack.

I myself see Jack as an intelligent, knowledgeable, and a reasonable fellow to communicate with. There are times when I do find that I may see some technical things somewhat differently than him, but have never known him to in my view be anything of the sort of what you describe above.

None of us are always and invariably immaculate in our reasoning:

http://forums.dpreview.com/forums/post/41980728

Even when we do learn and progress we do not become invincible authorities.

Vigorous technical debate is one thing. I rarely make comments surrounding tone of communication of others, but find such distinct, ongoing personalizations both unfortunate and in very bad taste.

In my very first posts to Mr. Hogan, I was both polite and clear that I did not think less of him for making the mistakes in thinking that he had made. I explained that I understood that the problem was complex, that it was easy to make a simple error in logic, and that I had also been mistaken about these very topics some few months ago. I told him I thought he was 100% right in 99.9% of the argument, but that the last 0.1% was ruining his conclusion.

I approached jack because he struck me as being much more intelligent than the average schlub and told him so, up front. Had he not replied with full on personal attacks and implications that I would be unable to comprehend the explanations of his correctness, and, on those grounds, refused to even justify his positions, well, I'm no saint, and may not respond to this kind of hubris too well, especially when I start out trying to be helpful.

So, there you have it. He's wrong. I would have gladly continued the discussion on a purely friendly level, but if he wishes to make it personal, well, I'm going to take a little pleasure in watching him get his well-deserved comeuppance. Not as zen as I'd like. Perhaps I'll have to do another round as a snail.

tex

 texinwien's gear list:texinwien's gear list
Panasonic Lumix DMC-GM5 Olympus E-M5 II Olympus 12-40mm F2.8 OnePlus One Canon EOS 300D +20 more
Detail Man
Detail Man Forum Pro • Posts: 17,026
Re: It's about useful comparisons - not which is better
1

I additionally know, and can prove, that the DPR studio scene samples offer no set of camera ISOs such as to meet your preferred criterion. However, they do offer one set of camera ISOs such as to meet another criterion that I find acceptable, namely one in which the cameras are given the same signal (as I defined the word signal above) although the mean raw values are not the same.

I don't think that they do do that. DPR exposes as I understand for a fix point in the sRGB output - that is that 18% (though they never specify 18%) on their step wedge gives the appropriate value in the JPEG file. You have to relate that back to the effective (meter ignored) ISO value that they discovered (and do not always report) in their ISO 'sensitivity' test.

Furthermore, the viewer of the DPR SCT RAW thumbnails is seeing Adobe ACR's RAW-level post-scaling applied without regard for individual deviations which may exist between cameras at either the same, or differing, ISO settings, as well as the possibility existing that DPR "tweaked the brightness" - which, in the case of the G3 test-shots processed using ACR 6.x, means the ACR 6.x "Brightness" control which is in fact a Gamma control (not a RAW-scaling control), and thus (when adjusted from the default value of 50) alters the apparent SNR of the image. Food for thought ?

See Andy Westlake's statement here: http://forums.dpreview.com/forums/post/40965292

Whether two cameras got the same exposure at the same camera ISO is nothing we have to speculate about DM. We can just check it. See here:

http://forums.dpreview.com/forums/post/51122252

Yes, I am familiar with that bit. My post was about other things that may affect the DPR SCT RAW thumbnails as they are presented, and at what display luminance levels image-noise may appear. I recognize that this is an issue restricted to the practice of casual reliance upon those thumbnails - as one is free (with the downloadable RAWs) to eschew Adobe's potential proprietary idiosynchracies, and use RAW processors that are not silently messing around in numerous (and largely unknown to users) respects with the RAW conversion and processing. Only "Adobe-ites" seem satisfied with that.

OK. I see your point. But as you say, one can always download the RAWs and play with them oneself. What you would want to know in that case is simply whether the exposure was the same and we can check that by means of the methods I have used.

I agree with what you have been saying. Your method of using the ratio of DxOMark Saturation ISOs combined with RawDigger statistics represents an accurate way to determine the actual exposure.

It seemed to me that the appearance of DPR SCT RAW thumbnails (not downloaded RAWs) is (at least a partial) factor in the making of value judgment by (at least some) readers of this thread.

Beyond issues of the exposure used in the test-shots involved, it seems to me that the apparent SNR (when viewing such DPR thumbnails) is an important factor. In that respect, the possibility that use of the ACR 6.x "Brightness" (which alters gamma-correction, as opposed to linearly scaling the output image-data) with respect to the G3 test shots represents a legitimate concern.

The "Brightness" control no longer existing in ACR 7.x, it is unclear what ACR control may now occasionally be "tweaked", and unclear whether the underlying mechanisms are understood by those making the adjustments, or by those viewing and relying upon such thumbnail views.

Please note that my raising of this issue related to the apparent SNR of these particular thumbnails is not intended to impeach your arguments in a major way. Instead, it seems to me to be a valid general concern which may possibly have some impact that it seems should rightly be noted.

Detail Man
Detail Man Forum Pro • Posts: 17,026
Re: Of course, i already told you this

... you were just a ball of hubris-fuelled ad hominem. and appeals to (false) authority. I tried, Jack.

I myself see Jack as an intelligent, knowledgeable, and a reasonable fellow to communicate with. There are times when I do find that I may see some technical things somewhat differently than him, but have never known him to in my view be anything of the sort of what you describe above.

None of us are always and invariably immaculate in our reasoning:

http://forums.dpreview.com/forums/post/41980728

Even when we do learn and progress we do not become invincible authorities.

Vigorous technical debate is one thing. I rarely make comments surrounding tone of communication of others, but find such distinct, ongoing personalizations both unfortunate and in very bad taste.

In my very first posts to Mr. Hogan, I was both polite and clear that I did not think less of him for making the mistakes in thinking that he had made. I explained that I understood that the problem was complex, that it was easy to make a simple error in logic, and that I had also been mistaken about these very topics some few months ago. I told him I thought he was 100% right in 99.9% of the argument, but that the last 0.1% was ruining his conclusion.

I approached jack because he struck me as being much more intelligent than the average schlub and told him so, up front. Had he not replied with full on personal attacks and implications that I would be unable to comprehend the explanations of his correctness, and, on those grounds, refused to even justify his positions, well, I'm no saint, and may not respond to this kind of hubris too well, especially when I start out trying to be helpful.

So, there you have it. He's wrong. I would have gladly continued the discussion on a purely friendly level, but if he wishes to make it personal, well, I'm going to take a little pleasure in watching him get his well-deserved comeuppance. Not as zen as I'd like. Perhaps I'll have to do another round as a snail.

Jack sometimes has a bit of a flourish about him, but in my communciations with him I have found him to be overall a rather gentle and reserved fellow who is not at all prone to making personal and mean spirited attacks. Have been following your recent interchanges. While I confess I have not been immersing myself deeply in the technical details of the various averments, I have read the surrounding text, and am familiar with the tone of communications.

Since you are making such charges, could you please provide specific quoted examples of statements made by Jack that you feel rise to the level of "ad hominem" attacks upon you as a person? Thanks.

OP texinwien Veteran Member • Posts: 3,326
Re: Of course, i already told you this

... you were just a ball of hubris-fuelled ad hominem. and appeals to (false) authority. I tried, Jack.

I myself see Jack as an intelligent, knowledgeable, and a reasonable fellow to communicate with. There are times when I do find that I may see some technical things somewhat differently than him, but have never known him to in my view be anything of the sort of what you describe above.

None of us are always and invariably immaculate in our reasoning:

http://forums.dpreview.com/forums/post/41980728

Even when we do learn and progress we do not become invincible authorities.

Vigorous technical debate is one thing. I rarely make comments surrounding tone of communication of others, but find such distinct, ongoing personalizations both unfortunate and in very bad taste.

In my very first posts to Mr. Hogan, I was both polite and clear that I did not think less of him for making the mistakes in thinking that he had made. I explained that I understood that the problem was complex, that it was easy to make a simple error in logic, and that I had also been mistaken about these very topics some few months ago. I told him I thought he was 100% right in 99.9% of the argument, but that the last 0.1% was ruining his conclusion.

I approached jack because he struck me as being much more intelligent than the average schlub and told him so, up front. Had he not replied with full on personal attacks and implications that I would be unable to comprehend the explanations of his correctness, and, on those grounds, refused to even justify his positions, well, I'm no saint, and may not respond to this kind of hubris too well, especially when I start out trying to be helpful.

So, there you have it. He's wrong. I would have gladly continued the discussion on a purely friendly level, but if he wishes to make it personal, well, I'm going to take a little pleasure in watching him get his well-deserved comeuppance. Not as zen as I'd like. Perhaps I'll have to do another round as a snail.

Jack sometimes has a bit of a flourish about him, but in my communciations with him I have found him to be overall a rather gentle and reserved fellow who is not at all prone to making personal and mean spirited attacks. Have been following your recent interchanges. While I confess I have not been immersing myself deeply in the technical details of the various averments, I have read the surrounding text, and am familiar with the tone of communications.

Since you are making such charges, could you please provide specific quoted examples of statements made by Jack that you feel rise to the level of "ad hominem" attacks upon you as a person? Thanks.

Read my post to him, linked above. Totally focused on the issue. Totally polite, although he'd already made snide comments elsewhere, suggesting that I had run out of arguments and was trying to 'shoot the messenger' - no idea why.

Then read his reply to the comment of mine linked above. Sorry, but I was being nothing other than polite and patient, and was sincerely hoping to help him post his mental block.

Ah, when I know I'm right, and when I'm trying to politely explain what is right to someone who has made an error in his logic, and his response is that he thinks I wouldn't be able to understand the explanation anyway (accompanied by a BS 'joke' defense of his wrongheaded ideas), yeah, not yet zen enough to just turn the other cheek yet. Working on it, but still have a ways to go.

tex

 texinwien's gear list:texinwien's gear list
Panasonic Lumix DMC-GM5 Olympus E-M5 II Olympus 12-40mm F2.8 OnePlus One Canon EOS 300D +20 more
Jack Hogan Veteran Member • Posts: 6,982
Re: It's about useful comparisons - not which is better

How you can check, without taking DPR's word for it, whether two cameras actually got the same signal (same amount of light per sensor area unit) at the same camera ISO setting is explained here:

http://forums.dpreview.com/forums/post/51123359

As you can see, the criterion of equal signal (same amount of light per sensor area unit) at the same camera ISO setting (I took ISO 200 as an example) was met for the E-M5 and the G3 within an error margin of 1/8 EV. Given that such is the case, what is wrong with comparing the sample images at the same camera ISO setting?

I am trying to follow, but the inconsistencies are starting to pile up In your link above it appears that you have confirmed DxO's relative saturation ISOs within 1/8th of a stop.   Please read on for the answer to your question.

It follows logically from the above, that if we compare the studio scene samples with the E-M5 set to ISO 3200 and the G3 to ISO 1600, as you propose, the G3 has received nearly twice the signal of the E-M5. So that comparison certainly does not meet your "same signal" requirement. Given that such is the case, what would be the reason to prefer it?

Perhaps the reason to prefer it is that your assumption that the G3 received twice the signal appears incorrect.  Warning, this is about to get very tedious for 95% of readers.  The other 5% may actually enjoy it

In the previous paragaph you have confirmed the relative saturation ISOs, but that does not mean that you can use mean raw values directly to estimate the signal (alias exposure, alias the number of photons that hit the sensors).  To do that you need good ol' SNR, around a mean of such a level that the vast majority of the N is shot noise.  At ISO 3200 and 1600 respectively for our two little cameras that means at around 25-50% of full scale.  We find just such a compliant little area in DPR's RAW captures under the letter A of the Kodak Gray Scale.  We find that the EM5 at in-camera ISO 3200 has an SNR of 19.39 and the G3 at ISO 1600 has an SNR of 21.04.  Squaring these values will give us the number of electrons output by each sensor.  That is 376 and 443.  Taking into consideration the G3's 11% lower Absolute Quantum Efficiency we can calculate that for the two captures in question the G3 received 0.41 stops more photons,/exposure/signal than the EM5.

So in DPR's test, the one I referred to with the images earlier, the G3 at in-camera ISO 1600 received 0.41 stops more light than the EM5 at ISO 3200, not twice the signal or 1 stop as you suggest.  This is fully explained by its 0.32 stop slower shutter speed (1/640 vs 1/800 and possible inaccuracies in the system), as mentioned in my post with the pictures. Since the lens appears to be the same, both cameras were illuminated by the exact same light, within +0.1 of a dB, good job DPR.

All this to say that when these two captures were taken the cameras were receiving virtually the same light, meeting the "same signal, exposure, number of incident photons requirement", within the allowed 1/3 of a stop difference fully accounted for by the different shutter speeds.

So both cameras, the EM5 at ISO3200 and the G3 at ISO1600, were illuminated by the same light, were subject to the same exposure, and would reach saturation at the same absolute luminous exposure, with the usual 1/3 of a stop proviso.  A very good match indeed, and imho the most useful of the ones available for the G3 to compare SNR performance with the EM5 at ISO 3200.

Cheers,
Jack

Detail Man
Detail Man Forum Pro • Posts: 17,026
Re: Of course, i already told you this

... you were just a ball of hubris-fuelled ad hominem. and appeals to (false) authority. I tried, Jack.

I myself see Jack as an intelligent, knowledgeable, and a reasonable fellow to communicate with. There are times when I do find that I may see some technical things somewhat differently than him, but have never known him to in my view be anything of the sort of what you describe above.

None of us are always and invariably immaculate in our reasoning:

http://forums.dpreview.com/forums/post/41980728

Even when we do learn and progress we do not become invincible authorities.

Vigorous technical debate is one thing. I rarely make comments surrounding tone of communication of others, but find such distinct, ongoing personalizations both unfortunate and in very bad taste.

In my very first posts to Mr. Hogan, I was both polite and clear that I did not think less of him for making the mistakes in thinking that he had made. I explained that I understood that the problem was complex, that it was easy to make a simple error in logic, and that I had also been mistaken about these very topics some few months ago. I told him I thought he was 100% right in 99.9% of the argument, but that the last 0.1% was ruining his conclusion.

I approached jack because he struck me as being much more intelligent than the average schlub and told him so, up front. Had he not replied with full on personal attacks and implications that I would be unable to comprehend the explanations of his correctness, and, on those grounds, refused to even justify his positions, well, I'm no saint, and may not respond to this kind of hubris too well, especially when I start out trying to be helpful.

So, there you have it. He's wrong. I would have gladly continued the discussion on a purely friendly level, but if he wishes to make it personal, well, I'm going to take a little pleasure in watching him get his well-deserved comeuppance. Not as zen as I'd like. Perhaps I'll have to do another round as a snail.

Jack sometimes has a bit of a flourish about him, but in my communciations with him I have found him to be overall a rather gentle and reserved fellow who is not at all prone to making personal and mean spirited attacks. Have been following your recent interchanges. While I confess I have not been immersing myself deeply in the technical details of the various averments, I have read the surrounding text, and am familiar with the tone of communications.

Since you are making such charges, could you please provide specific quoted examples of statements made by Jack that you feel rise to the level of "ad hominem" attacks upon you as a person? Thanks.

Read my post to him, linked above. Totally focused on the issue. Totally polite, although he'd already made snide comments elsewhere, suggesting that I had run out of arguments and was trying to 'shoot the messenger' - no idea why.

Then read his reply to the comment of mine linked above. Sorry, but I was being nothing other than polite and patient, and was sincerely hoping to help him post his mental block.

Ah, when I know I'm right, and when I'm trying to politely explain what is right to someone who has made an error in his logic, and his response is that he thinks I wouldn't be able to understand the explanation anyway (accompanied by a BS 'joke' defense of his wrongheaded ideas), yeah, not yet zen enough to just turn the other cheek yet. Working on it, but still have a ways to go.

I did not ask for your personal impressions surrounding statements made by Jack (which I think that you have made evident), or for your own personal impressions of the tone of your statements.

I did ask you to cite:

... specific quoted examples of statements made by Jack that you feel rise to the level of "ad hominem" attacks upon you as a person.

Keyboard shortcuts:
FForum MMy threads