How about Dynamic range?

[MAC]

I was wondering what makes the images from my 40D the images so special (other than the fact that I took them ). It may be dynamic range. Here is a compilation of dpr's recent DR results for in-camera jpegs from Canon and Nikons. These are prevailing values up to ISO 1600:

Cam... EV
30D... 8.4
40D... 9.1
50D... 8.3
7D... 8.3
5Dii... 8.4

D200... 8.2
D300S... 8.4

The 40D DR is a standout. I wonder why?

There is no magic in the 40D; must be just a difference in tone curves used.

--
John

I think there is magic in the 40d!

MAC

 

[John Sheehy]

Here:

OK, that's a start. These are 14-bit NEFs, I assume. That assumed, I would say we are looking at about a 1.3 stop DR benefit for the D3X over the 1Ds3.

The standard deviations are 1.42 ADU for the ISO 100 NEF and 2.05 for the ISO 200 NEF. Looking at the histograms, I can see that the black clipping occurs at true mean black, as opposed to some older Nikons which clipped higher.

Clipping at black reduces apparent standard deviation of read noise, since the mean rises above black, as zeros which should be negative numbers pull the mean up. I have found in the past that a factor of about 1.62 needs to be applied, so now we are looking at 2.3 and 3.31, respectively, for ISOs 100 and 200. The 1Ds3 is about 5.4 at ISO 100 (as also are the 7D, 40D, 5D2, and most Canons are close these days), so pixel-level DR is better in the D3X at ISO 100, and I would extrapolate from this data that ISO 400 on the D3X has about the same pixel-level noise floor as ISO 100 on the Canons.

Since Canon tends to clip their RAW data low, and Nikon does not, there's probably an additional 1/4 stop benefit to the D3X.

--
John

 

[John Sheehy]

I think there is magic in the 40d!

Perhaps, if by magic you mean "slight of hand trick".

--
John

 

[MAC]

John Sheehy wrote:
Perhaps, if by magic you mean "slight of hand trick".

yes, you got it!

 

[GaborSch]

How do you think it varies? You say the read noise is not constant; what observations lead you to that conclusion?

Following measurements are from an ISO 800 shot of the A800, from 25% illumination (-2 EV); I don't have the 50% value. First value: illumination in EV from clipping, second value: noise in ADU, third value: shot noise if the read noise is 5 ADU, as it is on a black frame.

-2 30.0 29.6
-3 21.0 20.4
-4 15.5 14.7
-5 11.1 9.91
-6 8.80 7.24
-7 7.00 4.90
-8 6.00 3.32
-9 5.12 1.10
-10 5.08 0.90

It looks good down to -6 EV, but not further.

I corrected pxlppr's math. 20000 electrons at 50% intensity would lead to 141 electrons of photon noise, or .7%, not 7%

I'm sure that's what you meant with 3.5%

Conversely, .5% noise at 50% intensity means that S/sqrt=200, or S=40000 electrons

Yes, and that's what I hardly believe. This indicates that the full capacity is 80000 electrons, on a 24 megapixel FF sensor? I think that is far too high to be true.

--
Gabor

http://www.panopeeper.com/panorama/pano.htm

 

[pixelpepper]

Just trying to get the sums right this time.

How do you think it varies? You say the read noise is not constant; what observations lead you to that conclusion?

Following measurements are from an ISO 800 shot of the A800, from 25% illumination (-2 EV); I don't have the 50% value. First value: illumination in EV from clipping, second value: noise in ADU, third value: shot noise if the read noise is 5 ADU, as it is on a black frame.

I concur with your calculation here. I've added som more columns, showing:

4 - illumination value (in ADU) (assuming full count of 4096 and black of 0, neither right, I should think)
5 - shot noise fraction (shot noise/illumination)
6 - electron count at this illumination (1/(shot noise fraction)^2)
7 - full scale electron count (electron count * 2^ev)

-2 30.0 29.6 1024 0.029 1198 4793
-3 21.0 20.4 512 0.040 630 5041
-4 15.5 14.7 256 0.057 304 4871
-5 11.1 9.91 128 0.077 166 5338
-6 8.80 7.24 64 0.113 78 4999
-7 7.00 4.90 32 0.153 43 5461
-8 6.00 3.32 16 0.207 23 5957
-9 5.12 1.10 8 0.138 53 26983
-10 5.08 0.90 4 0.224 20 20317

It looks good down to -6 EV, but not further.

Mostly due to measurement inaccuracies, I'd guess. Redoing the table, assuming a value of 1200 electrons at -2ev. Column 2: electrons at this ev, 3: theoretical shot noise, column 4, shot noise referred back to ADU, column 5, your measured shot noise in ADU.

-2 1200 34.6 29.6 29.6
-3 600 24.5 20.9 20.4
-4 300 17.3 14.8 14.7
-5 150 12.25 10.4 9.91
-6 75 8.7 7.4 7.24
-7 37.5 6.1 5.2 4.90
-8 18.8 4.3 3.7 3.32
-9 9.4 3.1 2.6 1.10
-10 4.7 2.2 1.85 0.90

As you see, agreement is good down to -6ev, but bey that stage you're trying to measure noises of a handful fo electrons. Down there, your rough estimate of read noise makes a big difference. At high illumination levels, it makes no difference at all.

Conversely, .5% noise at 50% intensity means that S/sqrt=200, or S=40000 electrons

Yes, and that's what I hardly believe. This indicates that the full capacity is 80000 electrons, on a 24 megapixel FF sensor? I think that is far too high to be true.

That was a example figure, your measurements give around 5000 at 800 ISO, scaling by 800/160, that's around 25000 at base ISO, which seems more reasonable.

 

[cpw]

..

Wrong. It works under the assumption that read noise is negligible. The point is to do the white frame test at an illumination much higher than the dark level.
...

I would not describe "the method" as relying upon this in order to work. Rather, I would describe this method, i.e. photon transfer, as working and giving meaningful results whether or not the read noise is negligible. Indeed, read noise, among other parameters, is one of the things that comes out of it. Please see the works of J. Janesick for his original descriptions of this method. The main things that this method relies upon is: sensor linearity, and the shot noise obeying Poisson statistics. In the most general case, there will be 3 noise sources: read, shot, and Pixel Response Non-Uniformity (PRNU). When the method is done in a way that eliminates PRNU, then one only has read noise and shot noise to worry about. As you say, we measure things in DN (digital number, others describe as ADU), and the main purpose of the method, as Emil describes, is to transfer to electron units (hence its name). The result in the presence ot the 2 noise sources, if we plot the data as noise variance (square of those standard deviations) vs. signal, will be a straight line. The slope and intercept of this line will give us inverse gain (DN per e-) and read noise, respectively. If one, in addition, evaluates the data in a way that does not exclude the PRNU, then one can come up with an estimate of this parameter also. One can also plot in log-log scale, as Janesick describes. It's pretty interesting.

Fine at high ISO, but at low ISO's there is more read noise than there needs to be. If the read noise was the same as it is at high ISO, there would be more DR at high ISO's, nor would there be any need for variable gain amplification, all ISO adjustment could be done by digital scaling.

Indeed, I've stated this for months now, please see at bottom here:

http://forums.dpreview.com/forums/read.asp?forum=1019&message=32061500&q=cpw&qf=m

Chris

 

[pixelpepper]

..

Wrong. It works under the assumption that read noise is negligible. The point is to do the white frame test at an illumination much higher than the dark level.
...

I would not describe "the method" as relying upon this in order to work. Rather, I would describe this method, i.e. photon transfer, as working and giving meaningful results whether or not the read noise is negligible.

Indeed, with more sophisticated analysis, the effect of the read noise can be eliminated (and the read noise can be estimated in cameras such as Nikons which clip at zero).

Indeed, read noise, among other parameters, is one of the things that comes out of it. Please see the works of J. Janesick for his original descriptions of this method. The main things that this method relies upon is: sensor linearity, and the shot noise obeying Poisson statistics. In the most general case, there will be 3 noise sources: read, shot, and Pixel Response Non-Uniformity (PRNU). When the method is done in a way that eliminates PRNU, then one only has read noise and shot noise to worry about. As you say, we measure things in DN (digital number, others describe as ADU), and the main purpose of the method, as Emil describes, is to transfer to electron units (hence its name). The result in the presence ot the 2 noise sources, if we plot the data as noise variance (square of those standard deviations) vs. signal, will be a straight line. The slope and intercept of this line will give us inverse gain (DN per e-) and read noise, respectively. If one, in addition, evaluates the data in a way that does not exclude the PRNU, then one can come up with an estimate of this parameter also. One can also plot in log-log scale, as Janesick describes. It's pretty interesting.

Thanks for the clarification and additional detail. It adds to one thing - Gabor's mistrust of the 'method' is misplaced.

Fine at high ISO, but at low ISO's there is more read noise than there needs to be. If the read noise was the same as it is at high ISO, there would be more DR at high ISO's, nor would there be any need for variable gain amplification, all ISO adjustment could be done by digital scaling.

Indeed, I've stated this for months now, please see at bottom here:

http://forums.dpreview.com/forums/read.asp?forum=1019&message=32061500&q=cpw&qf=m

Emil and John Sheehy have been stating it for years!

 

[ejmartin]

How do you think it varies? You say the read noise is not constant; what observations lead you to that conclusion?

Following measurements are from an ISO 800 shot of the A800, from 25% illumination (-2 EV); I don't have the 50% value. First value: illumination in EV from clipping, second value: noise in ADU, third value: shot noise if the read noise is 5 ADU, as it is on a black frame.

-2 30.0 29.6
-3 21.0 20.4
-4 15.5 14.7
-5 11.1 9.91
-6 8.80 7.24
-7 7.00 4.90
-8 6.00 3.32
-9 5.12 1.10
-10 5.08 0.90

It looks good down to -6 EV, but not further.

Well duh! Take the difference of two errors, and amplify it by a factor of 100. Time to learn a bit about error estimation and significant figures in data measurement.

The error bars are growing as one goes down the list. The further down you go, the less reliable your estimate is.

I corrected pxlppr's math. 20000 electrons at 50% intensity would lead to 141 electrons of photon noise, or .7%, not 7%

I'm sure that's what you meant with 3.5%

Yeah, I was a bit sloppy in my math. Last time around I did use a calculator, so hopefully that number (.7%) is correct.

Conversely, .5% noise at 50% intensity means that S/sqrt=200, or S=40000 electrons

Yes, and that's what I hardly believe. This indicates that the full capacity is 80000 electrons, on a 24 megapixel FF sensor? I think that is far too high to be true.

The number .5% was illustrative. Or do you know for a fact that this is the noise level of the D3x at half saturation, base ISO?

--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/

 

[GaborSch]

Mostly due to measurement inaccuracies, I'd guess. Redoing the table, assuming a value of 1200 electrons at -2ev

Taking advantage of artistic licence is not useful for calculations .

Down there, your rough estimate of read noise makes a big difference

I wonder what makes you think I made a rough estimation of the read noise. In fact, that (measured on black frames) is the most reliable value calculated with.

That was a example figure, your measurements give around 5000 at 800 ISO, scaling by 800/160, that's around 25000 at base ISO, which seems more reasonable.

That was not an example figure but the result of the calculation based on noise measured on ISO 100. I have not posted my measurement: the shot noise at 50% intensity with ISO 100 (which uses the "native" ISO) is 0.5% if the read noise at that intensity is the same as on the black frame . If this is true, then this is the result of 40,000 electrons at 50% intensity , thus the capacity is 80,000.

If this is not correct, then the shot noise must be much lower at 50% intensity, i.e. the read noise must be much higher at that intensity than on the black frame.

That's all I concluded.

--
Gabor

http://www.panopeeper.com/panorama/pano.htm

 

[GaborSch]

--
Gabor

http://www.panopeeper.com/panorama/pano.htm

 

[landmarks4711589173]

I was wondering what makes the images from my 40D the images so special (other than the fact that I took them ). It may be dynamic range. Here is a compilation of dpr's recent DR results for in-camera jpegs from Canon and Nikons. These are prevailing values up to ISO 1600:

Cam... EV
30D... 8.4
40D... 9.1
50D... 8.3
7D... 8.3
5Dii... 8.4

D200... 8.2
D300S... 8.4

The 40D DR is a standout. I wonder why?
--
JerryG

It is Jpeg dynamic range.

But in Raw the 40D stands out as well

 

[zappa1976]

Thanks for the explanation.
But less noise floor means automaticallly more DR in term of stop?

How you can evaluate which camera is better, for example, at 18% grey in term of noise?

How do you found my 1Ds3?

In EV, in your opinion how much is the difference comparing with D3X in real shots?
Are you sure that not at floor but at about the 18% grey, the D3X is better?

Thanks for the info.

Here:

OK, that's a start. These are 14-bit NEFs, I assume. That assumed, I would say we are looking at about a 1.3 stop DR benefit for the D3X over the 1Ds3.

The standard deviations are 1.42 ADU for the ISO 100 NEF and 2.05 for the ISO 200 NEF. Looking at the histograms, I can see that the black clipping occurs at true mean black, as opposed to some older Nikons which clipped higher.

Clipping at black reduces apparent standard deviation of read noise, since the mean rises above black, as zeros which should be negative numbers pull the mean up. I have found in the past that a factor of about 1.62 needs to be applied, so now we are looking at 2.3 and 3.31, respectively, for ISOs 100 and 200. The 1Ds3 is about 5.4 at ISO 100 (as also are the 7D, 40D, 5D2, and most Canons are close these days), so pixel-level DR is better in the D3X at ISO 100, and I would extrapolate from this data that ISO 400 on the D3X has about the same pixel-level noise floor as ISO 100 on the Canons.

Since Canon tends to clip their RAW data low, and Nikon does not, there's probably an additional 1/4 stop benefit to the D3X.

--
John

 

[ejmartin]

Mostly due to measurement inaccuracies, I'd guess. Redoing the table, assuming a value of 1200 electrons at -2ev

Taking advantage of artistic licence is not useful for calculations .

Down there, your rough estimate of read noise makes a big difference

I wonder what makes you think I made a rough estimation of the read noise. In fact, that (measured on black frames) is the most reliable value calculated with.

That was a example figure, your measurements give around 5000 at 800 ISO, scaling by 800/160, that's around 25000 at base ISO, which seems more reasonable.

That was not an example figure but the result of the calculation based on noise measured on ISO 100. I have not posted my measurement: the shot noise at 50% intensity with ISO 100 (which uses the "native" ISO) is 0.5% if the read noise at that intensity is the same as on the black frame . If this is true, then this is the result of 40,000 electrons at 50% intensity , thus the capacity is 80,000.

If this is not correct, then the shot noise must be much lower at 50% intensity, i.e. the read noise must be much higher at that intensity than on the black frame.

Not quite. Noises add (in quadrature), so the photon noise is not larger than the observed noise -- it must be the same or less. The photon noise is always the square root of the photon count. The observed noise thus provides an upper bound on the photon noise, which yields a lower bound on the S/N=sqrt, and therefore a lower bound on S. If the read noise were higher, the inferred signal would only increase .

If your noise measurement is correct, 80K photons is the lower bound on the well capacity.

--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/

 

[pixelpepper]

Mostly due to measurement inaccuracies, I'd guess. Redoing the table, assuming a value of 1200 electrons at -2ev

Taking advantage of artistic licence is not useful for calculations .

The calculated value was 1198.37, but since I didn't have good levels for white level, so assumed 1024 or black level, so assumed 0, there didn't seem much point complicating things by expressing it to two decimal places. If your measurements give better values for black and white levels (and the actual level that '-2ev' maps to in ADU), I can redo them (I still have the spreadsheet here. Unless Sony's not using a large proportion of the 4096 levels available, or your -2ev is seriously out, it's not going to make a big difference though.

Down there, your rough estimate of read noise makes a big difference

I wonder what makes you think I made a rough estimation of the read noise. In fact, that (measured on black frames) is the most reliable value calculated with.

And it was exactly 5 ADU. There's a co-incidence. How big was you patch of black?

That was a example figure, your measurements give around 5000 at 800 ISO, scaling by 800/160, that's around 25000 at base ISO, which seems more reasonable.

That was not an example figure

Emil confirmed that it was

but the result of the calculation based on noise measured on ISO 100. I have not posted my measurement: the shot noise at 50% intensity with ISO 100 (which uses the "native" ISO) is 0.5%

What was that in ADU? not sure whether you're referring to 0.5% of pixel value or 0.5% of full scale. Neither seems consistent with your other measurements.

 

[GaborSch]

If this is not correct, then the shot noise must be much lower at 50% intensity, i.e. the read noise must be much higher at that intensity than on the black frame.

The same but slower:

if the actual well capacity is less than 80,000,

then the shot noise has to be lower at 50% intensity than calculated above,

thus the read noise component of the noise has to be higher than assumed, i.e. higher than the measured noise on the black frame ,

therefor the read noise is not constant (it depends on the intensity).

--
Gabor

http://www.panopeeper.com/panorama/pano.htm

 

[GaborSch]

The calculated value was 1198.37, but since I didn't have good levels for white level, so assumed 1024 or black level, so assumed 0

I am lost. I have no idea, what you are referring to.

And it was exactly 5 ADU. There's a co-incidence. How big was you patch of black?

I don't understand the question; I posted I measured it on a black frame . Therefor the black patch is 6080x4044 pixels large. I don't remember how large are I measured, but it is totally irrelevent .

That was not an example figure

Emil confirmed that it was

Emil calculated that based on the sample I posted in the previous message, which was just the measurement from an A900 shot with ISO 100, at 50% intensity.

However, I conceed that this is not an important detail.

What was that in ADU? not sure whether you're referring to 0.5% of pixel value or 0.5% of full scale. Neither seems consistent with your other measurements.

1. It refers to the pixel value range ("full scale" in your terminology).

2. Forget about the ADU, it is useless. If the raw data is lossless, then the pixel value range is from 128 to 4095, but if it is lossy, then the range is from 256 to 8191; all measured values double. However, if you convert the lossy raw file with Adobe's DNG converter, it spreads the values up to 16383.

Therefor, if one refers to pixel values, one has to say in which format they relate to.

(Referring to Nikon data can be even worse.)

--
Gabor

http://www.panopeeper.com/panorama/pano.htm

 

[ejmartin]

If this is not correct, then the shot noise must be much lower at 50% intensity, i.e. the read noise must be much higher at that intensity than on the black frame.

The same but slower:

if the actual well capacity is less than 80,000,

then the shot noise has to be lower at 50% intensity than calculated above,

thus the read noise component of the noise has to be higher than assumed, i.e. higher than the measured noise on the black frame ,

therefor the read noise is not constant (it depends on the intensity).

Perhaps I'm misunderstanding your private terminology. When you say the noise is .5%, do you mean .5% of signal, or .5% of the range of RAW levels?

Well capacity of 80K leads to shot noise of sqrt[40K]=200 electrons at 50% of saturation. This is .5% of 40K but .25% of full scale. So one gets 80K as the full well if what you mean by .5% noise is N/S=.005.

If on the other hand by .5% noise you mean .5% of the range of RAW levels, then .5% noise at S=50% is S/N=100, which means S=10000 photons, which means saturation is 20000 photons.

So please define your private, nonstandard terminology for noise. If it corresponds to the second possibility above, then a very reasonable figure for full well is obtained (actually, one that is rather low by modern standards for the size of the pixels).

BTW, my previous post was entirely correct -- the measured noise provides a lower bound for the well capacity. Subtracting out read noise leads to lower shot noise estimate, leads to larger S/N estimate, leads to larger photon count estimate.

--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/

 

[GaborSch]

ejmartin wrote:

The noise percentage I posted relates to the pixel value range, i.e. 100%; however, I forgot to calculate with the full capacity.

Thus your calculation of 20000 electrons is "right" - but this appears to be far off, now in the other direction.

I don't know where the actual well capacity of the D3X/A900 can be found.

--
Gabor

http://www.panopeeper.com/panorama/pano.htm

 

[zappa1976]

All this is not important, I was only wondering how those numbers have been arrived at.

It would be nice if someone with the resources could host a site where RAW blackframes, OOF color-checkers, clipped files, and other such useful files could be hosted.

--
John

Here:

http://www.mediafire.com/?sharekey=f7be5d1881c0faddab1eab3e9fa335ca59273cf1b5086b04

I'm uploading (pls wait some time because my adsl is very slow today!) 2 blackframes (100 and 200 iso) of a D3X and a blackframe at 100 iso of my 1Ds3.
I hope these could be useful for some analysis.
Pls can you tell me/us what do you think about these files?
Thanks.

P.S.: here are available, at the moment, from the D3X:
http://www.zshare.net/download/684056477a40b936/
http://www.zshare.net/download/684064164b054e88/

For John Sheehy that asked the blackframes also of a A900:

iso100 http://www.mediafire.com/?dmhllbnedyl

iso200 http://www.mediafire.com/?dtjymzj2h2m

iso400 http://www.mediafire.com/?nzzyojzzjon

Cheers.