DxOMark's measured ISOs vs. manufacturer ISOs

Started Mar 13, 2013 | Discussions thread
texinwien Veteran Member • Posts: 3,326
Re: No Sarcasm - The Summary

Jack Hogan wrote:

texinwien wrote: It's all there, I believe. You may want to skip my pontifications and go straight to the sources I link to and quote.

Ok tex, I've been through your links with an open mind. If we take away the conspiracy theory and focus on the issues, I think it boils down to this:

Camera manufacturers can define ISO (S) as they please thanks to a lax standard.

'Lax' is not the word I would use, as it carries with it a value judgment that is not universally accepted.

Because of this fact it is not possible to measure the performance of camera A at a set in-camera ISO (let's call it Subjective S) and expect to meaningfully compare its results to those from Camera B at the same ISO setting.

Please provide a concise example in support of this statement. To be clear, I am challenging the validity of this statement and asking you to provide proof for it.

DPR is aware of this limitation and attempts to make up for it by taking its measurements in striclty controlled circumstances, which helps.

I'm not certain that DPR would agree with you about the existence or nature of the limitation you mention.

They use in-camera ISOs at face value, though, limiting the value of their analysis to the specific camera at hand.

This limit is in dispute.

Comparisons, such as studio scene IQ comparisons, are not directly comparable as shown in an earlier post in this thread.

The illustration in the earlier post was deeply flawed, as I have shown elsewhere in detail. In it, you attempted to compare images from two cameras based on those cameras' saturation sensitivity measures, forgetting that the images were exposed based on the cameras' ISO settings, meaning that one image received almost a full stop of exposure more than the other. Fatal error.

Furthermore, it seems you have not understood DPReview's test methods (although I provided you with links to official, detailed explanations of those methods), the first part of which is to test each camera's adherence to the SOS method of determining exposure index, which measure they use as the basis for all ISO test scene exposures. Either you did not read this, did not internalize it, or did not recognize its significance.

They are quite meaningless when a manufacturer has deviated substantially from current practice in defining its Subjective ISO.

They are not, as DxO and DPReview have both specifically stated multiple times.

So how do we solve this issue of manufacturers each defining in-camera ISOs subjectively?

It is in dispute whether there is any issue to solve.

Develop a sensitivity variable (let's call it Objective S) which can be used as a base to compare cameras. What do you know, it already exists and DxO measures it for us: Ssat (or DxO ISO).

Correct - and a photographer may perform valid tests based on ISO settings or on saturation sensitivity measures, as long as he remembers that no such tests will be valid unless all of the tested cameras received equal levels of exposure at equal ISO settings or equal saturation sensitivity measures.

So now that we have an Objective ISOif we want to know how two cameras will perform in the same situation apples-to-apples all we have to do is set their respective in-camera Subjective ISOs so that they correspond to the same Objective ISO. For example, for the EM5 and G3 Subjective S's of 3200 and 1600 respectively, which correspond to the same Objective S of about 1485. Easy.

Very, but you forgot to ensure that both cameras got equal levels of exposure, by, for instance,  dialing in +1 EV of exposure compensation for the E-M5. Which, as I have stated elsewhere, would lead to a similar shutter speed for both cameras, assuming matching apertures and scene illuminance.

So easy that makes me wonder why DPR is not doing it, or at least explaining the implications of not doing it to its subjects

I agree - it would be interesting to see these tests, in addition to those based on camera ISO settings, as long as the testers didn't forget to control carefully to ensure that all cameras received consistent exposure levels, which very important step you forgot

tex

 texinwien's gear list:texinwien's gear list
Panasonic Lumix DMC-GM5 Olympus E-M5 II Olympus 12-40mm F2.8 OnePlus One Canon EOS 300D +20 more
Post (hide subjects) Posted by
tko
tko
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark MMy threads
Color scheme? Blue / Yellow