bclaff
Forum Pro
Ah, as I read it the electrons freed from the silicon all have the same energy and any excess energy heats the silicon (at least in the visible range). (Chapter 2 Photon Interaction)
Regards,
-
Welcome to the new forums! Please read this first. For known issues we are working to resolve, click here.
Curve fitting and read noise, FWC, gain
- Thread starter Jack Hogan
- Start date
Regards,
I got the camera simulator working. Here's what I get for a 12 bit camera with a FWC of 100000 electrons, preamp RN of 1e-, postamp RN of 0 at ISO 100:
Horizontal axis mu, stops from full scale; Vertical axis sigma stops from full scale
We see more ringing than that on the D810 and the a7S.
Then I added a variable offset before the ADC, and set it to half an LSB. I got this:
Aha!
My guess is that, with the right combination of RN and offset, we'll be able to replicate the 12 bit camera behavior.
Jim
--
http://blog.kasson.com
Last edited:
bclaff
Forum Pro
Probably OT but your work reminds me of an old post I made regarding the D800 and a mystery for values at low signal levels:Then I added a variable offset before the ADC, and set it to half an LSB. I got this:
Aha!
My guess is that, with the right combination of RN and offset, we'll be able to replicate the 12 bit camera behavior.
D800 Photographic Dynamic Range
Regards,
--
Bill (visit me at http://home.comcast.net/~NikonD70/ )
Jack Hogan
Veteran Member
You are right, yes, that pesky offset could be the culprit in combination with a small read noise compared to LSB size - and does the D810 have optical black pixels that I think are used to estimate it before subtraction in many Nikon cameras? I haven't found them yet.I got the camera simulator working. Here's what I get for a 12 bit camera with a FWC of 100000 electrons, preamp RN of 1e-, postamp RN of 0 at ISO 100:
Horizontal axis mu, stops from full scale; Vertical axis sigma stops from full scale
We see more ringing than that on the D810 and the a7S.
Then I added a variable offset before the ADC, and set it to half an LSB. I got this:
Aha!
My guess is that, with the right combination of RN and offset, we'll be able to replicate the 12 bit camera behavior.
Jim
--
http://blog.kasson.com
On the other hand we are looking at 14-bit data with read noise around 80% of an LSB. if the error is of the order of 50% of an LSB it could still cause trouble.
Jack Hogan
Veteran Member
Except that I tried that and while it fixes the FWC it screws up the relative read noise. Original set of D810 data (columns = R,G,B, rows = ISO outlined below):
D810 read (noise) in e- and sat(uration) * 10ke-. Columns=R,G,B channels. Rows=ISO as shown at bottom.
FWCs are not the same. Implied pre-conditioning (normalized to 1 = FWC of one green channel):
Pre-conditioning implied by FWC (yellows=green ch, blue=red ch, purple=blue ch)
So applying a 1.14 factor to the red mean signal and re-fitting for read noise and saturation (columns represent the red and green channel resp.):
Hypothesis busted, as it looks like it messes with an otherwise fine looking read noise :-(
Last edited:
Jack Hogan
Veteran Member
Last edited:
Jack Hogan
Veteran Member
Actually, hypothesis unbusted - as I made a mistake in the post just above: I only multiplied the signal means by the WB pre-conditioning factors but not the standard deviations (doh!, both were incorrectly normalized by the pre-scaling). By instead multiplying both numerator and denominator by the same factor the SNR curve shape results unchanged - and therefore so does the relative read noise, as originally suspected. FWC on the other hand is affected as expected, as the curve is plotted against mean signal which has shifted as a result of the operation.
So here is my new effort at attempting to properly present FWC/saturation estimates for WB pre-conditioned data. I took a look at the histogram of a dark D810 raw image at base ISO and counted the missing codes in the red and blue channels in order to get a rough estimate of the pre-conditioning factors (the green channels are fully populated): red = 34 codes missing in 250 raw levels; blue 36 codes missing.
This implies that the red channel was pre-scaled by a factor of about 1.136 (similar to what I estimated earlier based on the average fwc difference) and the blue one by 1.144 (about 10% lower than my fwc estimate). This is what happens when I run the fit with these factors applied to the red and blue channel means and standard deviations (read noise is unchanged):
D810, r and b data corrected for pre-scaling, Jim Kasson Data
Other than for the first couple of ISO stops where we saw that ringing was messing with the fit and skewing results slightly, the red channel has almost caught up with the greens as expected, with FWC within 1% or so. The blue channel on the other hand has gotten much closer but it is still 2-3% off. I wonder why?
In any case I think that the case for just using the green channel data for RN and FWC estimates - or at least attempt to correct for pre-scaling - is getting a little stronger in my book.
Jack
Last edited:
ThrillaMozilla
Veteran Member
This is a pretty common error with least-squares fitting.
I haven't read through this, but any fitting method requires the appropriate measure. Minimizing the sum of squares of errors is appropriate if the uncertainties of all points are equal. Otherwise, the errors should be normalized to the statistical uncertainty of each measurement. In other words, a weighted fit.
In the first case, the uncertainties are kind of on a linear scale, but the fit was on a log scale! As you noted, that is a substantial mismatch. Just judging visually from the plot, I can't see the random deviations from the line. However, it appears that you have assumed that all the uncertainties are equal on a log scale. This does appear to give you a good fit, at least visually on the scale of the plots.
I'm sorry, I don't have time to explain (or remember!) this in a rigorous form, but that is the gist of it.
There is another possible problem. You and Jim Kasson may have this covered well, but it if the zero offset is incorrect, the signal will be calculated incorrectly, and the relative errors will be greatest for lowest signals. I have seen quite a few test photos in which the zero offset appears to have been determined incorrectly. In particular, I'm not sure it is always exactly equal to what is read on the dark strip.
I haven't read through this, but any fitting method requires the appropriate measure. Minimizing the sum of squares of errors is appropriate if the uncertainties of all points are equal. Otherwise, the errors should be normalized to the statistical uncertainty of each measurement. In other words, a weighted fit.
In the first case, the uncertainties are kind of on a linear scale, but the fit was on a log scale! As you noted, that is a substantial mismatch. Just judging visually from the plot, I can't see the random deviations from the line. However, it appears that you have assumed that all the uncertainties are equal on a log scale. This does appear to give you a good fit, at least visually on the scale of the plots.
I'm sorry, I don't have time to explain (or remember!) this in a rigorous form, but that is the gist of it.
There is another possible problem. You and Jim Kasson may have this covered well, but it if the zero offset is incorrect, the signal will be calculated incorrectly, and the relative errors will be greatest for lowest signals. I have seen quite a few test photos in which the zero offset appears to have been determined incorrectly. In particular, I'm not sure it is always exactly equal to what is read on the dark strip.
Jack Hogan
Veteran Member
This thread is almost 10 years old. Many questions here have been worked out since (for instance FWC differences in the raw color channels of Nikon cameras and ringing in low ISO PTC curves).
Should anyone have any relevant questions feel free to fire away. Otherwise let it rest in peace or start a new thread.
Jack
Should anyone have any relevant questions feel free to fire away. Otherwise let it rest in peace or start a new thread.
Jack
Last edited:
ThrillaMozilla
Veteran Member
Ah, I would have hoped so. It popped up, so I didn't notice the date.
But what is this ringing that keeps coming up?
Nikon D850 photon transfer curves - the last word
This is a continuation of a series of posts on the Nikon D850. You should be able to find all the posts about that camera in the Category List on the right sidebar, below the Articles widget. There's a drop-down menu there that you can use to get to all the posts in this series. You
blog.kasson.com
Sony a7RIII photon transfer curves - the last word
This is a continuation of a series of posts on the Sony a7RIII. You should be able to find all the posts about that camera in the Category List on the right sidebar, below the Articles widget. There's a drop-down menu there that you can use to get to all the posts in this series. [This
blog.kasson.com
ThrillaMozilla
Veteran Member
Ah, yes, thank you. I had seen ripples like that and wondered.
Last edited:
At this late date, I'd like to say that Jack deserves all the credit for the idea of adding and subtracting pairs of images. Not only does it get rid of PRNU, but it means that uneven lighting of the target and falloff of the lens is not an issue.
Jack Hogan
Veteran Member
Standing on the shoulders of giants Jim, I believe I saw Emil Martinec doing it. An additional benefit is that one can get relative noise without having to know the black level.
What I did not realize at the time is that pair subtraction also minimizes 1/sqrt(12) quantization rounding noise from the ADC, which adds in quadrature and is normally swept under the read noise carpet. Together with FPN, this is one of the reasons why the read noise calculated in the OP from pair subtraction is lower than simply reading standard deviation in a uniform patch of a single raw frame.
Jack
bclaff
Forum Pro
Jack,
I'm sure you know this as it's implied in your answer but subtraction doesn't eliminate noise it simply lowers it.
Regards,
Bill
Keyboard shortcuts
- f
- Forum