DxOmark: pixel level vs image level DR

Started Mar 31, 2012 | Discussions
Randplaty Contributing Member • Posts: 504
DxOmark: pixel level vs image level DR

Why does DxOmark do dynamic range on an image level? Shouldnt they do it on a pixel level? I understand doing ISO on an image level, but with DR, the pixel should either clip, record the photon of light, or blow out. That is all you need to measure dynamic range.

Doing dynamic range on an image level gets weird results like cropping the same image leading to lower DR and downsampling the same image leading to higher DR.

BadBeta Regular Member • Posts: 454
Re: DxOmark: pixel level vs image level DR

Why does DxOmark do dynamic range on an image level? Shouldnt they do it on a pixel level? I understand doing ISO on an image level, but with DR, the pixel should either clip, record the photon of light, or blow out. That is all you need to measure dynamic range.

The logic is that as you downsample and combine pixels you average out noise, and that in turn affects the dynamic range. (You don't actually have to downsize, but it helps visualize the effect on a picture level).

 BadBeta's gear list:BadBeta's gear list
Ricoh GR Digital II Ricoh GR Digital III Canon EOS 5D Canon EOS 5D Mark II Canon EF 16-35mm F2.8L II USM +7 more
OP Randplaty Contributing Member • Posts: 504
Re: DxOmark: pixel level vs image level DR

BadBeta wrote:

Why does DxOmark do dynamic range on an image level? Shouldnt they do it on a pixel level? I understand doing ISO on an image level, but with DR, the pixel should either clip, record the photon of light, or blow out. That is all you need to measure dynamic range.

The logic is that as you downsample and combine pixels you average out noise, and that in turn affects the dynamic range. (You don't actually have to downsize, but it helps visualize the effect on a picture level).

I can see that, but the problem is, by averaging out noise, you average out details also. DR is trying to recover the details, so by downsampling, you actually lose detail. The bottom range (correct me if I'm wrong) of DR is how much detail can you get which is above the base read noise level. So by reducing the read noise you can potentially reveal more detail. But simple downsampling isn't smart enough to know the difference between read noise and actual detail.

I mean think about it in practical terms. Can you take a photo, try to recover detail in the shadows but get nothing. Then instead, you downsample and all of a sudden details appear? I mean if you can do that, show me an example, because I've never seen that happen with my personal photo processing experience.

And that still doesn't answer the question of why cropping reduces DR.

Phil Hill Senior Member • Posts: 2,739
Re: DxOmark: pixel level vs image level DR

Randplaty wrote:

And that still doesn't answer the question of why cropping reduces DR.

Useable shadow DR is highly dependent on noise. Cropping requires greater enlargement, increasing noise.

BadBeta Regular Member • Posts: 454
Re: DxOmark: pixel level vs image level DR

Randplaty wrote:

BadBeta wrote:

Why does DxOmark do dynamic range on an image level? Shouldnt they do it on a pixel level? I understand doing ISO on an image level, but with DR, the pixel should either clip, record the photon of light, or blow out. That is all you need to measure dynamic range.

The logic is that as you downsample and combine pixels you average out noise, and that in turn affects the dynamic range. (You don't actually have to downsize, but it helps visualize the effect on a picture level).

I can see that, but the problem is, by averaging out noise, you average out details also. DR is trying to recover the details, so by downsampling, you actually lose detail. The bottom range (correct me if I'm wrong) of DR is how much detail can you get which is above the base read noise level. So by reducing the read noise you can potentially reveal more detail. But simple downsampling isn't smart enough to know the difference between read noise and actual detail.

Like I said you don't actually have to downsize. It was just to make the concept easier to think about in terms of comparing different sized pixels. The picture scale noise level is the same even if you don't actually sacrifice the resolution (in order to compare on equal terms with different sized pixels).

I mean think about it in practical terms. Can you take a photo, try to recover detail in the shadows but get nothing. Then instead, you downsample and all of a sudden details appear?

Of course not. Like I said, you don't actually have to downsize. It is just a way of visually comparing DR of different pixel densities.

And that still doesn't answer the question of why cropping reduces DR.

It does? Why? You still have the same pixels and density?

 BadBeta's gear list:BadBeta's gear list
Ricoh GR Digital II Ricoh GR Digital III Canon EOS 5D Canon EOS 5D Mark II Canon EF 16-35mm F2.8L II USM +7 more
OP Randplaty Contributing Member • Posts: 504
Re: DxOmark: pixel level vs image level DR

BadBeta wrote:

Randplaty wrote:

BadBeta wrote:

Why does DxOmark do dynamic range on an image level? Shouldnt they do it on a pixel level? I understand doing ISO on an image level, but with DR, the pixel should either clip, record the photon of light, or blow out. That is all you need to measure dynamic range.

The logic is that as you downsample and combine pixels you average out noise, and that in turn affects the dynamic range. (You don't actually have to downsize, but it helps visualize the effect on a picture level).

I can see that, but the problem is, by averaging out noise, you average out details also. DR is trying to recover the details, so by downsampling, you actually lose detail. The bottom range (correct me if I'm wrong) of DR is how much detail can you get which is above the base read noise level. So by reducing the read noise you can potentially reveal more detail. But simple downsampling isn't smart enough to know the difference between read noise and actual detail.

Like I said you don't actually have to downsize. It was just to make the concept easier to think about in terms of comparing different sized pixels. The picture scale noise level is the same even if you don't actually sacrifice the resolution (in order to compare on equal terms with different sized pixels).

I mean think about it in practical terms. Can you take a photo, try to recover detail in the shadows but get nothing. Then instead, you downsample and all of a sudden details appear?

Of course not. Like I said, you don't actually have to downsize. It is just a way of visually comparing DR of different pixel densities.

But the reason why DxOmark is to equalize "prints" from different resolution sensors. If the DR does not show up in the print, why should we normalize the DR? Just use the original DR figure. For the D800 it would be 13.8 or something instead of 14.4.

And that still doesn't answer the question of why cropping reduces DR.

It does? Why? You still have the same pixels and density?

I don't know why it does... but in bclaff's measurements, all of the DX modes have lower DR than their corresponding FX modes.

BadBeta Regular Member • Posts: 454
Re: DxOmark: pixel level vs image level DR

And that still doesn't answer the question of why cropping reduces DR.

It does? Why? You still have the same pixels and density?

I don't know why it does... but in bclaff's measurements, all of the DX modes have lower DR than their corresponding FX modes.

Well, actually I believe it is the very same logic as comparing many small pixels to a large one. You simply use less of the sensor, have less information, and get a higher noise level for that reason.

Extreme example: Say if the sensor consisted of only 10 pixels, and you took a picture of a grey surface. Due to noise some would be darker, some ligher, but on average it would be about 18% grey. (If the camera metering did its job). If you now crop down to 1 pixel it will likely not be 18% grey. If you add another pixel and average you will likely be closer to 18%, and so on and so forth. More information strengthen the signal to noise ratio, and cropping removes information.

 BadBeta's gear list:BadBeta's gear list
Ricoh GR Digital II Ricoh GR Digital III Canon EOS 5D Canon EOS 5D Mark II Canon EF 16-35mm F2.8L II USM +7 more
OP Randplaty Contributing Member • Posts: 504
Re: DxOmark: pixel level vs image level DR

Yeah I think you're right.

But that's not really what we photographers care about when we talk about dynamic range. We're not concerned with the overall information level of photo. We're concerned with whether or not the sky will blow out or if the shadows have any detail left. Cropping doesn't reduce that at all, yet it reduces the DR score.

BadBeta wrote:

And that still doesn't answer the question of why cropping reduces DR.

It does? Why? You still have the same pixels and density?

I don't know why it does... but in bclaff's measurements, all of the DX modes have lower DR than their corresponding FX modes.

Well, actually I believe it is the very same logic as comparing many small pixels to a large one. You simply use less of the sensor, have less information, and get a higher noise level for that reason.

Extreme example: Say if the sensor consisted of only 10 pixels, and you took a picture of a grey surface. Due to noise some would be darker, some ligher, but on average it would be about 18% grey. (If the camera metering did its job). If you now crop down to 1 pixel it will likely not be 18% grey. If you add another pixel and average you will likely be closer to 18%, and so on and so forth. More information strengthen the signal to noise ratio, and cropping removes information.

bclaff Veteran Member • Posts: 4,841
Re: DxOmark: pixel level vs image level DR

DxOMark reports two sets of values:
Screen which are "pixel" values.
And Print which is normalized to an 8"x12" 300ppi print.

And one single value:
The highest Print DR is reported as their Landscape Dynamic Range overall value.

The Print values are appropriate for comparing different cameras because, after all, regardless of the camera, you will make the same sized print and view it at the same distance.

DxOMark and I use different criteria so my Photographic Dynamic Range (PDR) is 3.24 EV lower than their Landscape DR.
Comparison here:
http://home.comcast.net/~NikonD70/Charts/PDR_Landscape_scatter.htm

Dynamic Range has nothing to do with detail. It's simply a measure of the ratio of brightest to darkest signal.

Since the human eye can only resolve a cone of a certain angle it's entirely appropriate to statistically combine all the pixels you can't tell apart into one value.

Regards

-- hide signature --
BadBeta Regular Member • Posts: 454
Re: DxOmark: pixel level vs image level DR

Thanks, that cleared my mumbled mind up a bit.

 BadBeta's gear list:BadBeta's gear list
Ricoh GR Digital II Ricoh GR Digital III Canon EOS 5D Canon EOS 5D Mark II Canon EF 16-35mm F2.8L II USM +7 more
BadBeta Regular Member • Posts: 454
Re: DxOmark: pixel level vs image level DR

Randplaty wrote:

Yeah I think you're right.

But that's not really what we photographers care about when we talk about dynamic range. We're not concerned with the overall information level of photo. We're concerned with whether or not the sky will blow out or if the shadows have any detail left. Cropping doesn't reduce that at all, yet it reduces the DR score.

I can only agree with that.

 BadBeta's gear list:BadBeta's gear list
Ricoh GR Digital II Ricoh GR Digital III Canon EOS 5D Canon EOS 5D Mark II Canon EF 16-35mm F2.8L II USM +7 more
Phil Hill Senior Member • Posts: 2,739
Re: DxOmark: pixel level vs image level DR

Randplaty wrote:

or if the shadows have any detail left. Cropping doesn't reduce that at all, yet it reduces the DR score.

Cropping and then printing at the same size as the uncropped file does reduce the available detail in the shadows because of the additional enlargement required to equalize the size. And it also increases the visibility of noise.

Shadow DR is all about how much you can lift the shadows before noise becomes objectionable and/or detail degrades. Cropping adversely affects both.

OP Randplaty Contributing Member • Posts: 504
Re: DxOmark: pixel level vs image level DR

Yes that's exactly my question. If by downsampling, the detail and noise are both reduced, isn't that dynamic range worthless in real use?

If the dynamic range measurement doesn't include a detail measurement, couldn't a sensor record some sort of light value at lower stops of light, but that light is virtually worthless?

bclaff wrote:

DxOMark reports two sets of values:
Screen which are "pixel" values.
And Print which is normalized to an 8"x12" 300ppi print.

And one single value:
The highest Print DR is reported as their Landscape Dynamic Range overall value.

The Print values are appropriate for comparing different cameras because, after all, regardless of the camera, you will make the same sized print and view it at the same distance.

DxOMark and I use different criteria so my Photographic Dynamic Range (PDR) is 3.24 EV lower than their Landscape DR.
Comparison here:
http://home.comcast.net/~NikonD70/Charts/PDR_Landscape_scatter.htm

Dynamic Range has nothing to do with detail. It's simply a measure of the ratio of brightest to darkest signal.

Since the human eye can only resolve a cone of a certain angle it's entirely appropriate to statistically combine all the pixels you can't tell apart into one value.

Regards

-- hide signature --
OP Randplaty Contributing Member • Posts: 504
Re: DxOmark: pixel level vs image level DR

Phil Hill wrote:

Randplaty wrote:

or if the shadows have any detail left. Cropping doesn't reduce that at all, yet it reduces the DR score.

Cropping and then printing at the same size as the uncropped file does reduce the available detail in the shadows because of the additional enlargement required to equalize the size. And it also increases the visibility of noise.

Shadow DR is all about how much you can lift the shadows before noise becomes objectionable and/or detail degrades. Cropping adversely affects both.

I can see that mathematically, but for real world use, either the detail is above the noise level or isn't above the noise level. Cropping and printing won't make the detail sink back below the noise level. It's only if the measurement cannot measure detail that that could possibly happen. bclaff above said that DR measurements don't consider detail at all.

But our eyes can measure detail.

Phil Hill Senior Member • Posts: 2,739
Re: DxOmark: pixel level vs image level DR

Well, it’s more about whether you can make a good print from the file. The acceptable DR threshold is going to be different if you crop.

The measurements may be the same at the pixel level but the effects are going to be different at the image level for a crop printed at the same size.

drwho9437 Senior Member • Posts: 1,992
They do both

Some of the extractions attempt to let you compare cameras of different resolutions to one another.

Essentially the DR plots come from the SNR plots, if you want the pixel level just look at the raw log-log SNR plots for a camera.

Because a higher resolution camera with the same sensor area will have fewer photons per photo site they normalize the pixel sizes for comparing cameras basically... (IE they use the resolution to rescale), this lets you see the readout noise of the sensor rather than the worse SNR due to smaller number of electrons/photons.

In readout amplifiers, charge pumps on Canon sensors are certainly inferior in a significant way from the Sony derived sensors (Nikon included).

The DxO numbers for DNR show what setting a luminosity value of 0 to SNR of 1 would be (0 dB). Probably you don't want to do that, if noise in dark tones can't get anywhere close to 0 dB then Canon and Nikon are much closer. The reality is this is so and that is why it technical superior photon noise limited Sony sensors aren't so clearly better than Canon's that everyone jumps ship.

I am very curious to see if Canon follows Sony's lead at some point or if there is a base engineering choice for a sensor that isn't purely photon/electron count noise limited.

Given the 5D mk III sensor is probably developed in parallel with 7D's I'd guess we won't see it in this sensor but I'd love to be wrong since there was a very small change in MP.
--
http://www.flickr.com/photos/bbryce/

OP Randplaty Contributing Member • Posts: 504
Re: They do both

That makes sensento me. Is it possible for one sensor to have better highlights but worse overall DR score?

drwho9437 wrote:

Some of the extractions attempt to let you compare cameras of different resolutions to one another.

Essentially the DR plots come from the SNR plots, if you want the pixel level just look at the raw log-log SNR plots for a camera.

Because a higher resolution camera with the same sensor area will have fewer photons per photo site they normalize the pixel sizes for comparing cameras basically... (IE they use the resolution to rescale), this lets you see the readout noise of the sensor rather than the worse SNR due to smaller number of electrons/photons.

In readout amplifiers, charge pumps on Canon sensors are certainly inferior in a significant way from the Sony derived sensors (Nikon included).

The DxO numbers for DNR show what setting a luminosity value of 0 to SNR of 1 would be (0 dB). Probably you don't want to do that, if noise in dark tones can't get anywhere close to 0 dB then Canon and Nikon are much closer. The reality is this is so and that is why it technical superior photon noise limited Sony sensors aren't so clearly better than Canon's that everyone jumps ship.

I am very curious to see if Canon follows Sony's lead at some point or if there is a base engineering choice for a sensor that isn't purely photon/electron count noise limited.

Given the 5D mk III sensor is probably developed in parallel with 7D's I'd guess we won't see it in this sensor but I'd love to be wrong since there was a very small change in MP.
--
http://www.flickr.com/photos/bbryce/

John Sheehy Forum Pro • Posts: 18,806
Re: They do both

Randplaty wrote:

That makes sensento me. Is it possible for one sensor to have better highlights but worse overall DR score?

What do you mean by better highlights? Less noise in the highlights, or more of them above metered gray?

In either case, yes, it is possible.

-- hide signature --

John

drwho9437 Senior Member • Posts: 1,992
Re: They do both

They aren't measuring "highlights", highlights only exist if you apply a tone curve and map the ADC values to some RGB values. They aren't doing that in the case of DR, they are looking purely at the number that come out of the ADC pipe in the raw file.

(Though I don't know the raw file formats in detail I am assuming it is just a 12 or 14 bit matrix of numbers and you need to know which are RGB).

So what they do may do is take a picture of a gradient (white to black) and then increase the exposure (or illumination) until the raw file says the white is the max value the camera will let it output.

Then they effectively look at bars in their gradient from white to black and look at the range RAW luminance values the file has, any variance is noise. From this variance they have the SNR.

(Alternatively they use a grey card and just turn up and down a light until it clips in the RAW file and goes to 0 in the raw file).

From the SNR you they define DR as 0 dB noise to the SNR at clipping point of the file. I believe it is 6 dB per stop.

However you won't want to use an image with 0 dB SNR. It would just all be noise, so this definition of DR while useful quantitatively isn't useful in the field, however the plotted data is , you just look at what the SNR curves look like above a certain number of dB. Probably 18, 24 or 30 dB for most people I would think.

In sum they are measuring the linear properties of a linear sensor, whereas what people think of as a picture has a log curve applied to it, this is where people start to say oh that DxO number doesn't make sense it is junk, because how to take linear sensor data and turn it into the log image does matter... RAW + a fixed converter is the fairest way but even that could have differences camera to camera given different color filters, and choices of tone curves for the RAW converter authors.
--
http://www.flickr.com/photos/bbryce/

Great Bustard Forum Pro • Posts: 38,500
If you want better highlights...

Randplaty wrote:

That makes sensento me. Is it possible for one sensor to have better highlights but worse overall DR score?

...then use a lower exposure and/or apply a different tone curve to the captured photo.

The DR is the number of stops from the noise floor to the saturation limit. The only way you can get more DR is to lower the read noise and/or raise the saturation limit.

In other words, there is a maximum DR that the sensor can capture, and if the DR of the scene exceeds the DR of the sensor, then the competent photographer needs to select which portion of the DR from the scene they wish to record.

Keyboard shortcuts:
FForum MMy threads