Photographic Dynamic Range (PDR) is a More Useful Measure than the DxOMark Landscape Score

Started Apr 17, 2015 | Discussions
bclaff Veteran Member • Posts: 8,710
Photographic Dynamic Range (PDR) is a More Useful Measure than the DxOMark Landscape Score
10

I had been performing Photographic Dynamic Range (PDR) measurements for some time when DxOMark arrived on the scene with their Landscape Dynamic Range Score.
The methodologies are simlilar, and although the scores are on different scales, they correlated well. I preferred my PDR to the DxOMark Landscape Score but there was no compelling reason in the past to argue that one was preferred over the other.

However, with improved technology at the pixel level and higher pixel densities differences between the results are becoming more apparent.

The DxOMark criteria has the effect of pushing their result into the portion of the Photon Transfer Curve (PTC) which is near or below SNR = 1 (read noise).
In this region differences in the shape/slope of the foot of the PTC are lost. This results in DxOMark Landscape Scores that sometimes don't really fit. A prime recent example would be the DxOMark ranking the D7200 (a DX/APS-C sensor) ahead of numerous FX/Full Frame sensors such as the D750, D800, D800E, D600 and Sony A7R. (See below)

The PDR criteria operates in the PTC region well above SNR=1 (at least until we get 384Mp sensors!).
In this region the subtle interplay of read noise, other noise, small amounts of photon noise, slope due to Full Well Capacity (FWC), etc. is captured.
PDR has scaled better with shifting technology.

So now I do think that PDR is a better measure than the DxOMark Landscape Score.

As it happens, I can take DxOMark Signal to Noise Ratio (SNR) data and apply my PDR criteria to produce the dynamic range curves that DxOMark would get if they had applied my criteria rather than theirs. I never published these before, but now I think they have value.

I invite you to peruse the new interactive chart at my site: DxOMark Photographic Dynamic Range Chart

So we can have a concrete discussion about this matter consdier the following charts:

D7100 PTC dark corner

D7200 PTC dark corner

D610 PTC dark corner

D750 PTC dark corner

This data was all collected at the same time using the same technique. These are all 24Mp cameras so they normalize identically.
The axes are linear and cover the deep lower left corner of the PTC with signal from 0 to 14 and SNR from 0 to 6.

Note that the cameras with larger pixels and higher Full Well Capacities (FWC) have SNR values that improve more quickly as signal increases.
The blue diamond is the DxOMark unnormalized dynamic range and the red diamond is the DxOMark Landscape (normalized) Dynamic Range Score.
The green diamon is my Photographic Dynamic Range (PDR) measure.

The positions of the red diamonds are dictated exclusively by read noise.
The green diamonds are influenced by FWC (and to a lesser extent other factors).

From the charts you can see how great D7200 read noise causes DxOMark to rank the D7200 as better than the D610 and D750 (among others).
But it's also clear that the D610 and D750 have better SNR values and therefore better Image Quality (IQ) than the D7200 (and D7100).

To me it's very clear that PDR ranks the cameras more reasonably than DxOMark Landscape Score.

Respectfully,

-- hide signature --
Nikon D600 Nikon D610 Nikon D7100 Nikon D7200 Nikon D750 Nikon D800 Nikon D800E
If you believe there are incorrect tags, please send us this post using our feedback form.
Detail Man
Detail Man Forum Pro • Posts: 17,073
Re: Photographic Dynamic Range (PDR) is a More Useful Measure than the DxOMark Landscape Score

bclaff wrote:

... The PDR criteria operates in the PTC region well above SNR=1 (at least until we get 384Mp sensors!).

In this region the subtle interplay of read noise, other noise, small amounts of photon noise, slope due to Full Well Capacity (FWC), etc. is captured.

It seems that (especially thses days, with higher performance image-sensors), Readout/Dark noise component are effectively "numerically swamped" by Photon Shot Noise. Rather "subtle", indeed.

PDR has scaled better with shifting technology.

Would not methods such as the "PDR" method (using much higher SNRs) all seem to "scale better" ?

So now I do think that PDR is a better measure than the DxOMark Landscape Score.

OP bclaff Veteran Member • Posts: 8,710
Re: Photographic Dynamic Range (PDR) is a More Useful Measure than the DxOMark Landscape Score

Detail Man wrote:

bclaff wrote:

PDR has scaled better with shifting technology.

Would not methods such as the "PDR" method (using much higher SNRs) all seem to "scale better" ?

I think any similar method with an SNR criteria "reasonably" above 1 would be well behaved.
In the cases presented above the criteria is SNR = 4; I'm not sure I'd classify that as "much higher".

Regards,

-- hide signature --
Detail Man
Detail Man Forum Pro • Posts: 17,073
Re: Photographic Dynamic Range (PDR) is a More Useful Measure than the DxOMark Landscape Score

bclaff wrote:

Detail Man wrote:

bclaff wrote:

PDR has scaled better with shifting technology.

Would not methods such as the "PDR" method (using much higher SNRs) all seem to "scale better" ?

I think any similar method with an SNR criteria "reasonably" above 1 would be well behaved.
In the cases presented above the criteria is SNR = 4; I'm not sure I'd classify that as "much higher".

Nor would I.

However, it would surprise me if at SNR=26 dB (a linear factor of 20), sensor Readout/Dark noise components (these days) were not quite effectively "numerically swamped" by Photon Shot Noise.

OP bclaff Veteran Member • Posts: 8,710
Re: Photographic Dynamic Range (PDR) is a More Useful Measure than the DxOMark Landscape Score

Detail Man wrote:

bclaff wrote:

Detail Man wrote:

bclaff wrote:

PDR has scaled better with shifting technology.

Would not methods such as the "PDR" method (using much higher SNRs) all seem to "scale better" ?

I think any similar method with an SNR criteria "reasonably" above 1 would be well behaved.
In the cases presented above the criteria is SNR = 4; I'm not sure I'd classify that as "much higher".

Nor would I.

However, it would surprise me if at SNR=26 dB (a linear factor of 20), sensor Readout/Dark noise components (these days) were not quite effectively "numerically swamped" by Photon Shot Noise.

I never use dB because I'm thinking photography not engineering; but in those terms my SNR = 4 is 12 dB

Regards,

-- hide signature --
Detail Man
Detail Man Forum Pro • Posts: 17,073
Re: Photographic Dynamic Range (PDR) is a More Useful Measure than the DxOMark Landscape Score

bclaff wrote:

Detail Man wrote:

bclaff wrote:

Detail Man wrote:

bclaff wrote:

PDR has scaled better with shifting technology.

Would not methods such as the "PDR" method (using much higher SNRs) all seem to "scale better" ?

I think any similar method with an SNR criteria "reasonably" above 1 would be well behaved.
In the cases presented above the criteria is SNR = 4; I'm not sure I'd classify that as "much higher".

Nor would I.

However, it would surprise me if at SNR=26 dB (a linear factor of 20), sensor Readout/Dark noise components (these days) were not quite effectively "numerically swamped" by Photon Shot Noise.

I never use dB because I'm thinking photography not engineering; but in those terms my SNR = 4 is 12 dB

Photography uses log base 2 extensively. "I share your pain" in having to convert the log base 10 values published by DxO Labs. Perhaps they enjoy the panache of seeming "engineering-like" ? ...

Perhaps the base used in calculating star magnitude (2.512) might be the most appropriate of all ?

J A C S
J A C S Forum Pro • Posts: 14,781
Re: Photographic Dynamic Range (PDR) is a More Useful Measure than the DxOMark Landscape Score

bclaff wrote:

The DxOMark criteria has the effect of pushing their result into the portion of the Photon Transfer Curve (PTC) which is near or below SNR = 1 (read noise).
In this region differences in the shape/slope of the foot of the PTC are lost. This results in DxOMark Landscape Scores that sometimes don't really fit. A prime recent example would be the DxOMark ranking the D7200 (a DX/APS-C sensor) ahead of numerous FX/Full Frame sensors such as the D750, D800, D800E, D600 and Sony A7R. (See below)

The PDR criteria operates in the PTC region well above SNR=1 (at least until we get 384Mp sensors!).
In this region the subtle interplay of read noise, other noise, small amounts of photon noise, slope due to Full Well Capacity (FWC), etc. is captured.

But that is to be expected. The noise curve depends on other factors, as well and one number cannot determine it all. On the other hand, SNR=1 is too low for photographic purposes, I agree.

What is your definition of PDR? I see SNR=4 there but that limits the DR numbers a lot?

Jack Hogan Veteran Member • Posts: 7,012
DxO Print/Landscape DR vs Photographic Dynamic Range
2

Hi Bill,

I am going to have to read this more carefully in a few hours when I have more time. I agree in general that PDR may be more practical than eDR for photographers, because of the threshold it chooses for an acceptable minimum signal. Aside from potential errors both give valuable information imho, though, DxO being more of a Sensor score.

DxO starts with the pixel. Their choice of SNR = 1 for the threshold means that total random noise (standard deviation due to shot and read noise) is equal to mean signal out of the pixel. For read noise of around 1.9e- (D7200 at base ISO), that would mean a signal of about 2.46e- out of the photosite. Read noise is somewhat but not much larger in the quadrature sum for total noise (3.6 vs 2.5e- under the square root). As such in this case it tends to take into account relative sensor performance, as exemplified by the figure of merit rn^2. That's probably why this criterion has been used for a long time in engineering applications. In the context of 24MP raw captures DxO then asks the question: what if the pixels were binned 3:1? They assume that the numerator in DR would grow three times but the denominator only sqrt(3). I don't think they read this signal off the SNR curve.

PDR on the other hand starts with the viewer and suggests that in order to achieve the lower acceptable threshold when viewing a final image in standard conditions, a 24MP raw capture needs to show SNR = 4 at the pixel level. For read noise of around 1.9e- that means a mean signal of about 19e- out of the photosite. Shot noise is now way dominant in the quadrature sum compared to read noise (19.0 vs 3.6e- under the square root). It's arguably a fair criterion from more of a practical perspective. But in this case tends to de-emphasize relative sensor performance.

So imho they both have their place: DxO tells you clearly which is the better sensor, which deepest shadows you can push more; PDR tells you how much of a difference it may make in (its view of) typical practice.

Jack

Horshack Veteran Member • Posts: 7,922
Re: Photographic Dynamic Range (PDR) is a More Useful Measure than the DxOMark Landscape Score
3

Detail Man wrote:

bclaff wrote:

... The PDR criteria operates in the PTC region well above SNR=1 (at least until we get 384Mp sensors!).

In this region the subtle interplay of read noise, other noise, small amounts of photon noise, slope due to Full Well Capacity (FWC), etc. is captured.

It seems that (especially thses days, with higher performance image-sensors), Readout/Dark noise component are effectively "numerically swamped" by Photon Shot Noise. Rather "subtle", indeed.

But there's a distinct qualitative difference of read noise vs shot noise. John Sheehy once posted a simulation of an ISO 1,000,000 image with zero read noise and it looked more natural than a much lower ISO image with read noise.

Horshack Veteran Member • Posts: 7,922
Engineering DR still a needed data point
5

IMO engineering DR is still very useful because it tells the reader what the deepest shadows will look like irrespective of the SNR further up the tonal scale (what PDR provides). For example, here is a D5000 vs D3s shadow push comparison I did many years ago - although the D3s outperforms the D5000 significantly in terms of QE and shot noise, the D5000 still produces better deep shadows at base ISO, even after discounting the D3s's banding:

Full link: http://www.dpreview.com/forums/post/36104230

Francois2 New Member • Posts: 1
Re: Photographic Dynamic Range (PDR) is a More Useful Measure than the DxOMark Landscape Score
1

Detail Man wrote:

bclaff wrote:

Detail Man wrote:

bclaff wrote:

PDR has scaled better with shifting technology.

Would not methods such as the "PDR" method (using much higher SNRs) all seem to "scale better" ?

I think any similar method with an SNR criteria "reasonably" above 1 would be well behaved.
In the cases presented above the criteria is SNR = 4; I'm not sure I'd classify that as "much higher".

Nor would I.

However, it would surprise me if at SNR=26 dB (a linear factor of 20), sensor Readout/Dark noise components (these days) were not quite effectively "numerically swamped" by Photon Shot Noise.

I think that you have chosen the criteria of SNR=20 since long.

Did you change it suddenly ? But why ?

Detail Man
Detail Man Forum Pro • Posts: 17,073
Re: Photographic Dynamic Range (PDR) is a More Useful Measure than the DxOMark Landscape Score

Francois2 wrote:

Detail Man wrote:

bclaff wrote:

Detail Man wrote:

bclaff wrote:

PDR has scaled better with shifting technology.

Would not methods such as the "PDR" method (using much higher SNRs) all seem to "scale better" ?

I think any similar method with an SNR criteria "reasonably" above 1 would be well behaved.
In the cases presented above the criteria is SNR = 4; I'm not sure I'd classify that as "much higher".

Nor would I.

However, it would surprise me if at SNR=26 dB (a linear factor of 20), sensor Readout/Dark noise components (these days) were not quite effectively "numerically swamped" by Photon Shot Noise.

I think that you have chosen the criteria of SNR=20 since long.

Did you change it suddenly ? But why ?

I am simply here noting that it seems your inquiry is (presumably) directed specifically to Bill Claff.

OP bclaff Veteran Member • Posts: 8,710
Clarification - PDR SNR

The PDR SNR critieria is for an SNR of 20 normalized to a standard Circle of Confusion (CoC).

This works out to 16000 / (height of sensor).

For the examples given the heights are all about 4000 so the target SNR at a pixel level is 4

-- hide signature --
OP bclaff Veteran Member • Posts: 8,710
See Clarification - PDR SNR

Clarification - PDR SNR

-- hide signature --
OP bclaff Veteran Member • Posts: 8,710
See Clarification - PDR SNR

Clarification - PDR SNR

-- hide signature --
OP bclaff Veteran Member • Posts: 8,710
Yes - nt
-- hide signature --
OP bclaff Veteran Member • Posts: 8,710
Clarification - Read Noise and Engineering Dynamic Range

There is no doubt that read noise (engineering dynamic range) is a key sensor characterization.

And perhaps normalizing read noise to an area provides some way of comparison across sensors.

However, normalized read noise is not photographically relevant; and the DxOMark Landscape Use Case score is supposed to be a photographic use case.

-- hide signature --
OP bclaff Veteran Member • Posts: 8,710
Re: DxO Print/Landscape DR vs Photographic Dynamic Range

Jack Hogan wrote:

So IMHO they both have their place: DxO tells you clearly which is the better sensor, which deepest shadows you can push more; PDR tells you how much of a difference it may make in (its view of) typical practice.

Great overview (which I snipped out).

Semantics perhaps ...
I disagree that DxO "tell you clearly which is the better sensor".
Perhaps I'd agree that it tells you which has the better pixel architecture.

I mean we have to say "better sensor" for what purpose.
If it's to take photographs I think DxO is less relevant than PDR.

Regards,

-- hide signature --
Detail Man
Detail Man Forum Pro • Posts: 17,073
Re: Photographic Dynamic Range (PDR) is a More Useful Measure than the DxOMark Landscape Score

Horshack wrote:

Detail Man wrote:

bclaff wrote:

... The PDR criteria operates in the PTC region well above SNR=1 (at least until we get 384Mp sensors!).

In this region the subtle interplay of read noise, other noise, small amounts of photon noise, slope due to Full Well Capacity (FWC), etc. is captured.

It seems that (especially thses days, with higher performance image-sensors), Readout/Dark noise component are effectively "numerically swamped" by Photon Shot Noise. Rather "subtle", indeed.

But there's a distinct qualitative difference of read noise vs shot noise.

Seems to be a reasonable statement to make [surrounding specific visual observation(s) made].

John Sheehy once posted a simulation of an ISO 1,000,000 image with zero read noise and it looked more natural than a much lower ISO image with read noise.

Tried to find such a graphic (without success). Can you recall any particular "keywords" used in that post that I could "plug into" a (anything but the on-site DPReview, that is) "search engine" ?

Or, might you have a reference-link to John Sheehy's publishing of that graphic to provide ?

DM

OP bclaff Veteran Member • Posts: 8,710
Re: Photographic Dynamic Range (PDR) is a More Useful Measure than the DxOMark Landscape Score

Horshack wrote:

Detail Man wrote:

bclaff wrote:

... The PDR criteria operates in the PTC region well above SNR=1 (at least until we get 384Mp sensors!).

In this region the subtle interplay of read noise, other noise, small amounts of photon noise, slope due to Full Well Capacity (FWC), etc. is captured.

It seems that (especially thses days, with higher performance image-sensors), Readout/Dark noise component are effectively "numerically swamped" by Photon Shot Noise. Rather "subtle", indeed.

But there's a distinct qualitative difference of read noise vs shot noise. John Sheehy once posted a simulation of an ISO 1,000,000 image with zero read noise and it looked more natural than a much lower ISO image with read noise.

In Detail Man's comment I think "swamped" is too strong a term.
In this area of the PTC we are still very much in the read noise dominated region.

If an image or an area of an image is entirely read noise then it has no detail and is essentially black.

To have a photograph you must have photons and therefore photon noise.

Characterizing the photographic use of a sensor in a region of the PTC that is devoid of photon noise is, as I have stated more subtly above, brain-dead.

Here's a hypothetical.
Two sensors that are the same in every way except that one has a higher FWC.
(You can even imagine them both with zero read noise, DSNU, etc.)
Photographically these sensors are not identical.
You will get more useful dynamic range out of the one with the higher FWC.

Regards,

-- hide signature --
Keyboard shortcuts:
FForum MMy threads