R5 Low ISO Noise Reduction (cont.)

Started 2 months ago | Discussions
J A C S
J A C S Forum Pro • Posts: 16,960
R5 Low ISO Noise Reduction (cont.)
10

A continuation of this interesting thread. It took me a while but I have now the auto-correlation of the two G channels combined. The bright blue squares have values about 23% of the center one, all eight of them.

This is based on a 2048x2048 centered square. Dealing with the missing (on purpose) R and B values due to the Bayer structure was a RPITA.

R5, ISO 100, G+G2 auto-correlation

It is amazing how much order is there in the noise, isn't it?

Canon EOS R5
If you believe there are incorrect tags, please send us this post using our feedback form.
stoffer Regular Member • Posts: 275
Re: R5 Low ISO Noise Reduction (cont.)
1

J A C S wrote:

A continuation of this interesting thread. It took me a while but I have now the auto-correlation of the two G channels combined. The bright blue squares have values about 23% of the center one, all eight of them.

This is based on a 2048x2048 centered square. Dealing with the missing (on purpose) R and B values due to the Bayer structure was a RPITA.

R5, ISO 100, G+G2 auto-correlation

It is amazing how much order is there in the noise, isn't it?

Interesting! So you don’t think this is really noise reduction, or am I missing something? Sorry for being a bit dense with these technical discussions!

-- hide signature --

Best regards, Stoffer
May the Light be with you!

 stoffer's gear list:stoffer's gear list
Sony a7R III Sony FE 35mm F1.4 Sony FE 24mm F1.4 GM Apple iPhone X +1 more
sobrien
sobrien Senior Member • Posts: 1,435
Re: R5 Low ISO Noise Reduction (cont.)
3

Probably no harm to link the original (and indeed interesting) thread for future reference:

https://www.dpreview.com/forums/thread/4508766
Lots of people (myself included) will be interested to understand this better and the practical implications for photography.

P.S. Great work to all the contributors so far.

-- hide signature --

"The simple things are also the most extraordinary things and only the wise can see them."
https://www.flickr.com/photos/135843555@N03/

 sobrien's gear list:sobrien's gear list
Canon EOS R Canon EOS R5 Canon EF 85mm F1.8 USM Canon EF 135mm F2L USM Canon Extender EF 2x III +16 more
J A C S
OP J A C S Forum Pro • Posts: 16,960
Re: R5 Low ISO Noise Reduction (cont.)
2

stoffer wrote:

J A C S wrote:

A continuation of this interesting thread. It took me a while but I have now the auto-correlation of the two G channels combined. The bright blue squares have values about 23% of the center one, all eight of them.

This is based on a 2048x2048 centered square. Dealing with the missing (on purpose) R and B values due to the Bayer structure was a RPITA.

R5, ISO 100, G+G2 auto-correlation

It is amazing how much order is there in the noise, isn't it?

Interesting! So you don’t think this is really noise reduction, or am I missing something? Sorry for being a bit dense with these technical discussions!

Hard to say, I do not know enough about the engineering part to make guesses. It could be better controlled read noise, which would be a good thing.

BTW, the auto-correlation is not the blur kernel. The latter is smaller. I will post something later today.

J A C S
OP J A C S Forum Pro • Posts: 16,960
Blur kernel of B
3

I had some time to try to recover the actual blur kernel of B. Presumably, the same thing applies to R.

The auto-correlation is the blur kernel convolved with itself. Here is the auto-correlation of B again:

Each pixel make sense as a shift. Shift=(0,0) corresponds to the yellow square. The auto-correlation there is always one. We see that the neighboring pixels are correlated with coefficients approximately 0.14. Pixels at distance two or greater are very weakly correlated.

Now, to find the blur kernel is not that trivial, especially given noisy data. I had to fit two parameters - the actual standard deviation before NR (or whatever it is) and the blur coefficients. The resulting standard deviation is known to be 1.3. Playing with some numbers and doing some computations, I found a very good fit with actual standard deviation 2 (or 1.9 or so), which is what can be found by analyzing the border pixels (see the old thread). Then the blur kernel looks like this (I only claim to be an approximation):

blur kernel of B, black frame

Disregard the fact that we have 9x9 pixels vs. 13x13 as above, it has no significance. The bright blue pixels have values approximately 0.07. The eight neighboring pixels contribute to about 1/3 of the total light at the center. This shows that in the autocorrelation above, we must have nonzero values of the second "ring" (at distance=2) which looks the same as the background; but they are really small.

To validate that, I created a synthetic noisy image with the same parameters as the one I know or predicted (st.dev.=2), applied the blur kernel I recovered, and computed the auto-correlation. It looks virtually the same as the first picture here, coming from the data.

You may say that the two pictures look more or less the same - they do but the second one has 1/2 of the neighboring values, approximately.

Now, about G - it is more complicated. I posted the auto-correlation earlier. I tuned in my code further and the neighboring coefficients need to be corrected a bit but the picture looks more or less the same. I will post something later.

EthanCam Forum Member • Posts: 65
Re: Blur kernel of B

J A C S wrote:

I had some time to try to recover the actual blur kernel of B. Presumably, the same thing applies to R.

The auto-correlation is the blur kernel convolved with itself. Here is the auto-correlation of B again:

Each pixel make sense as a shift. Shift=(0,0) corresponds to the yellow square. The auto-correlation there is always one. We see that the neighboring pixels are correlated with coefficients approximately 0.14. Pixels at distance two or greater are very weakly correlated.

Now, to find the blur kernel is not that trivial, especially given noisy data. I had to fit two parameters - the actual standard deviation before NR (or whatever it is) and the blur coefficients. The resulting standard deviation is known to be 1.3. Playing with some numbers and doing some computations, I found a very good fit with actual standard deviation 2 (or 1.9 or so), which is what can be found by analyzing the border pixels (see the old thread). Then the blur kernel looks like this (I only claim to be an approximation):

blur kernel of B, black frame

Disregard the fact that we have 9x9 pixels vs. 13x13 as above, it has no significance. The bright blue pixels have values approximately 0.07. The eight neighboring pixels contribute to about 1/3 of the total light at the center. This shows that in the autocorrelation above, we must have nonzero values of the second "ring" (at distance=2) which looks the same as the background; but they are really small.

To validate that, I created a synthetic noisy image with the same parameters as the one I know or predicted (st.dev.=2), applied the blur kernel I recovered, and computed the auto-correlation. It looks virtually the same as the first picture here, coming from the data.

You may say that the two pictures look more or less the same - they do but the second one has 1/2 of the neighboring values, approximately.

Now, about G - it is more complicated. I posted the auto-correlation earlier. I tuned in my code further and the neighboring coefficients need to be corrected a bit but the picture looks more or less the same. I will post something later.

Thanks for your ongoing investigations!

I played around with the ISO100 dark frames provided by Hoka Key and on a 2048x2048 grid I found very similar autocorrelation patterns for the R, G, G2 and B channels as you posted earlier. I visualized the plot in log-scale to be able to better see small values, and indeed as you mentioned (for R and B at least, I have not checked G) the second ring has quite small values but still seems to contribute to the center value.

What are the numerical values of the blur coefficients you found for B channel?

(as a side note, I think the blur kernel should be normalized such that the sum of the coefficients is one, but I am not sure it is the case here)

Cheers

Horshack Veteran Member • Posts: 8,544
Re: R5 Low ISO Noise Reduction (cont.)

This looks very promising, great work. Can you share the actual values of the full 3x3 kernel? It's hard to get the precise values from the color-coded charts.

J A C S
OP J A C S Forum Pro • Posts: 16,960
Re: Blur kernel of B

EthanCam wrote:

J A C S wrote:

I had some time to try to recover the actual blur kernel of B. Presumably, the same thing applies to R.

The auto-correlation is the blur kernel convolved with itself. Here is the auto-correlation of B again:

Each pixel make sense as a shift. Shift=(0,0) corresponds to the yellow square. The auto-correlation there is always one. We see that the neighboring pixels are correlated with coefficients approximately 0.14. Pixels at distance two or greater are very weakly correlated.

Now, to find the blur kernel is not that trivial, especially given noisy data. I had to fit two parameters - the actual standard deviation before NR (or whatever it is) and the blur coefficients. The resulting standard deviation is known to be 1.3. Playing with some numbers and doing some computations, I found a very good fit with actual standard deviation 2 (or 1.9 or so), which is what can be found by analyzing the border pixels (see the old thread). Then the blur kernel looks like this (I only claim to be an approximation):

blur kernel of B, black frame

Disregard the fact that we have 9x9 pixels vs. 13x13 as above, it has no significance. The bright blue pixels have values approximately 0.07. The eight neighboring pixels contribute to about 1/3 of the total light at the center. This shows that in the autocorrelation above, we must have nonzero values of the second "ring" (at distance=2) which looks the same as the background; but they are really small.

To validate that, I created a synthetic noisy image with the same parameters as the one I know or predicted (st.dev.=2), applied the blur kernel I recovered, and computed the auto-correlation. It looks virtually the same as the first picture here, coming from the data.

You may say that the two pictures look more or less the same - they do but the second one has 1/2 of the neighboring values, approximately.

Now, about G - it is more complicated. I posted the auto-correlation earlier. I tuned in my code further and the neighboring coefficients need to be corrected a bit but the picture looks more or less the same. I will post something later.

Thanks for your ongoing investigations!

I played around with the ISO100 dark frames provided by Hoka Key and on a 2048x2048 grid I found very similar autocorrelation patterns for the R, G, G2 and B channels as you posted earlier. I visualized the plot in log-scale to be able to better see small values, and indeed as you mentioned (for R and B at least, I have not checked G) the second ring has quite small values but still seems to contribute to the center value.

It should; if we assume a symmetric blur kernel of "radius" at least one (not a delta), the auto-correlation should span twice as much but with very small values in the second ring (of second order, like squares of the first ring).

What are the numerical values of the blur coefficients you found for B channel?

Something like 0.14-0.15 for all eight in the first ring, with the corners close to 0.14 and the N, E, S and W close to 0.15; for the AC. This indicates approx. equal values for the blur kernel in that ring, roughly a half of those before normalization.

(as a side note, I think the blur kernel should be normalized such that the sum of the coefficients is one, but I am not sure it is the case here)

Sure. I have it normalized that way in the computations but for the purpose of the visualization, I rescaled it to increase the contrast. I should have kept the original scale and changed the color bar style. When I said that the eight neighboring pixels contribute to about 1/3 of the total light at the center, I had the normalization into account - they have values about 0.07 (with the center 1.00) before normalization; eight of them is 0.56, which is about 1/3 of 1.00+0.56.

Cheers

Good to see someone else working on that. I will analyze the dark but not completely dark shot next. I have the G blur kernel and will post it in a moment.

J A C S
OP J A C S Forum Pro • Posts: 16,960
Re: R5 Low ISO Noise Reduction (cont.)

Horshack wrote:

This looks very promising, great work. Can you share the actual values of the full 3x3 kernel? It's hard to get the precise values from the color-coded charts.

Here:

0.0449 0.0449 0.0449 
0.0449 0.6410 0.0449 
0.0449 0.0449 0.0449

This is done with (manual) fitting, the off center values are 0.07 when the center is 1, then normalized. I chose st.dev.=2 before NR, and 1.3 after, and approximately equal values in the second ring, equal to the computed ones, more or less, with a slight edge to the up/down/left/right ones.

J A C S
OP J A C S Forum Pro • Posts: 16,960
Blur kernel of G+G2

The blur kernel of G+G2, ISO 100, black frame (as the rest):

blur kernel of G+G2

This time, normalized properly. The "diamond" values are 0.0303, the center is 0.7576. In this particular example, I put some small values (about 1/4 of 0.0303) in the four corners of the 5x5 square as well but you cannot see that on the plot.

I did not get such a great fit this time. I had to chose st.dev.=1.75 before NR, while we measured ~1.9-1.95 in the borders before. The output st.dev. is 1.3 with those values, as it should be. With those values, it fits the AC well but the st.dev. in reality is probably higher than 1.75 before NR. Also, I accounted for the rounding to an integer (511, 512, etc.) and chose mean=512. In fact, it is a bit lower than that but choosing, say 511.4 in the simulation part does not make a visible difference.

The blur kernel can just be guessed first from the AC since the latter has a strong central value=1 and the rest is small. Then divide the other values by 2, and you get a good approximation before normalization. I compared it to the data though, what you see is not just the guess.

Horshack Veteran Member • Posts: 8,544
Re: R5 Low ISO Noise Reduction (cont.)

J A C S wrote:

Horshack wrote:

This looks very promising, great work. Can you share the actual values of the full 3x3 kernel? It's hard to get the precise values from the color-coded charts.

Here:

0.0449 0.0449 0.0449
0.0449 0.6410 0.0449
0.0449 0.0449 0.0449

This is done with (manual) fitting, the off center values are 0.07 when the center is 1, then normalized. I chose st.dev.=2 before NR, and 1.3 after, and approximately equal values in the second ring, equal to the computed ones, more or less, with a slight edge to the up/down/left/right ones.

Thanks. When reverse-engineering the kernel keep in mind there's a good chance they're using a median filter, so in addition to trying to impute the relative pixel weightings for a mean-type kernel I would also consider the possible strategies employed for a median filter. Getting the precise base kernel/method is important but I'm actually more interested in what the cut-off is, since that directly impacts what ADU threshold or noise relationships the kernel is being applied to. For example, how the correlation decreases as ISO increases. Also, in an unrelated thread on FM about long-exposure noise I was able to get a hold of a 60-second ISO 100 blackframe and the correlations are noticeably less on it than on a normal, fast-exposure ISO 100 blackframe, implying the cut-off may be noise or distribution related rather than simply ADU-threshold related.

Lastly, in addition to using the stdev relationship of the border vs non-border pixels to reverse-engineer the kernel I would also consider using the top-left and top-right corner cases, where you have 5 of the 9 original input pixel values available to you for a 3x3 kernel.

J A C S
OP J A C S Forum Pro • Posts: 16,960
Re: R5 Low ISO Noise Reduction (cont.)

Horshack wrote:

J A C S wrote:

Horshack wrote:

This looks very promising, great work. Can you share the actual values of the full 3x3 kernel? It's hard to get the precise values from the color-coded charts.

Here:

0.0449 0.0449 0.0449
0.0449 0.6410 0.0449
0.0449 0.0449 0.0449

This is done with (manual) fitting, the off center values are 0.07 when the center is 1, then normalized. I chose st.dev.=2 before NR, and 1.3 after, and approximately equal values in the second ring, equal to the computed ones, more or less, with a slight edge to the up/down/left/right ones.

Thanks. When reverse-engineering the kernel keep in mind there's a good chance they're using a median filter, so in addition to trying to impute the relative pixel weightings for a mean-type kernel I would also consider the possible strategies employed for a median filter.

Right. The auto-correlation assumes nothing, it just computes it. The computation of the blur kernel assumes a convolution already, which may not be the case. On the other hand, I get a remarkable fit for the B channel, and I would imagine that a median filter would be more destructive. Just speculating.

Getting the precise base kernel/method is important but I'm actually more interested in what the cut-off is, since that directly impacts what ADU threshold or noise relationships the kernel is being applied to. For example, how the correlation decreases as ISO increases.

I am planning to stay with ISO for a while to see how the NR or whatever it is changes with the signal level.

Also, in an unrelated thread on FM about long-exposure noise I was able to get a hold of a 60-second ISO 100 blackframe and the correlations are noticeably less on it than on a normal, fast-exposure ISO 100 blackframe, implying the cut-off may be noise or distribution related rather than simply ADU-threshold related.

Yeah, everything I do is based on one frame only, who knows how things change with the speed. I think I am going to ask our generous contributor to shoot, say 1/125 sec, f/8, ISO frame in complete darkness before going to long exposures.

Lastly, in addition to using the stdev relationship of the border vs non-border pixels to reverse-engineer the kernel I would also consider using the top-left and top-right corner cases, where you have 5 of the 9 original input pixel values available to you for a 3x3 kernel.

I was thinking about that but I am not sure that they do the same thing there.

Horshack Veteran Member • Posts: 8,544
"Noise reduction" measurement for all ISOs
2

Expanding on the observation/theory from this sub-thread, here is a graph of the stdev noise delta between the first sensor column and all other columns for R5 fast-exposure blackframes. This quantifies the amount of noise reduction being applied at each ISO, presuming the border-pixel theory presented in the aforementioned link is correct.

R5 Stdev noise differential between 1st sensor column and all other columns, across all Full-EV ISOs

EthanCam Forum Member • Posts: 65
Re: R5 Low ISO Noise Reduction (cont.)

Horshack wrote:

Also, in an unrelated thread on FM about long-exposure noise I was able to get a hold of a 60-second ISO 100 blackframe and the correlations are noticeably less on it than on a normal, fast-exposure ISO 100 blackframe, implying the cut-off may be noise or distribution related rather than simply ADU-threshold related.

Interesting. Do you know if „Long Exposure NR“ was turned on in the camera?

EthanCam Forum Member • Posts: 65
Re: "Noise reduction" measurement for all ISOs
1

Horshack wrote:

Expanding on the observation/theory from this sub-thread, here is a graph of the stdev noise delta between the first sensor column and all other columns for R5 fast-exposure blackframes. This quantifies the amount of noise reduction being applied at each ISO, presuming the border-pixel theory presented in the aforementioned link is correct.

R5 Stdev noise differential between 1st sensor column and all other columns, across all Full-EV ISOs

Thanks!

It seems so far that the dark values in ISO100 and ISO400 are the most affected by NR, at least for short enough exposures.

If this is done to improve general image quality, it is questionable that only ISO100 and ISO400 would need a „special NR treatment“.

On the other hand, they are the respective base ISOs of the dual conversion gain sensor, and the two ISOs that deliver the maximal DR in photos resp. in C-Log videos. Interesting.

J A C S
OP J A C S Forum Pro • Posts: 16,960
Re: R5 Low ISO Noise Reduction (cont.)
5

Digging a bit deeper, with new images, 1/125 sec., ISO 100, black frame, the camera a bit warmer, here is what I found (correlates well with what Bill found out as well):

Noise/NR

Mechanical shutter: similar as before. Noise is a bit higher but the camera is hotter: 40+ C.

EFCS: A bit higher noise (by 10-15%), like 1.5 or so, NR the same.

Electronic Shutter: Noise 4.0 (vs 1.5), much higher! Almost no NR!

High Speed plus: Noise 2.16 (higher), NR a bit lower bit still there. Are those 12 bit? I cannot tell.

Banding

There is a very faint horizontal banding, nothing to worry about but there are also a few fat vertical strips as well. We are talking about some channels going up or down 1 ADU or maybe less! What is interesting is that the vertical strips were the same from shot to shot even though Hoka Hey (thank you, Hoka!) did not shot them in succession! EFCS, ES, etc., all have their own signatures. Well, this is based on 8 images only.

NR as a function of the signal level

This is hard to assess. Somewhere in the visible shadows which are not completely black in JPEG, the NR is gone. In the very deep shadows. NR seems to increase when going to the bottom. Of course, I do not know the algorithm. I has to use different images to get an idea of what is going on; ideally, I should be able to do it with a single image of a gradient type but it is hard to shoot one. The bottom line is that unless you lift shadows heavily, the NR or whatever it is there is not doing anything you will notice.

Hoka Hey
Hoka Hey Senior Member • Posts: 2,039
Re: R5 Low ISO Noise Reduction (cont.)

J A C S wrote:

Digging a bit deeper, with new images, 1/125 sec., ISO 100, black frame, the camera a bit warmer, here is what I found (correlates well with what Bill found out as well):

Noise/NR

Mechanical shutter: similar as before. Noise is a bit higher but the camera is hotter: 40+ C.

EFCS: A bit higher noise (by 10-15%), like 1.5 or so, NR the same.

Electronic Shutter: Noise 4.0 (vs 1.5), much higher! Almost no NR!

High Speed plus: Noise 2.16 (higher), NR a bit lower bit still there. Are those 12 bit? I cannot tell.

Banding

There is a very faint horizontal banding, nothing to worry about but there are also a few fat vertical strips as well. We are talking about some channels going up or down 1 ADU or maybe less! What is interesting is that the vertical strips were the same from shot to shot even though Hoka Hey (thank you, Hoka!) did not shot them in succession! EFCS, ES, etc., all have their own signatures. Well, this is based on 8 images only.

NR as a function of the signal level

This is hard to assess. Somewhere in the visible shadows which are not completely black in JPEG, the NR is gone. In the very deep shadows. NR seems to increase when going to the bottom. Of course, I do not know the algorithm. I has to use different images to get an idea of what is going on; ideally, I should be able to do it with a single image of a gradient type but it is hard to shoot one. The bottom line is that unless you lift shadows heavily, the NR or whatever it is there is not doing anything you will notice.

Thanks for all your hard work on this!

-- hide signature --

Joe

J A C S
OP J A C S Forum Pro • Posts: 16,960
Re: R5 Low ISO Noise Reduction (cont.)
3

Thanks.

I played with some underexposed DNG files posted by a youtube reviewer (shots of a boy with the RF 50). The darkest one is 3 stops underexposed. I do not see NR in the border pixels! With the uniformly lit images, I see it in the really dark images only which are not images for any photographic purposes. The ones that are half completely black and the other half terribly dark but not black show NR in the borders.

I can speculate that the NR triggers on images that are horribly underexposed, like the ones dpreview shoots! Another option would be to be applied on very dark parts only but this is hard to test, and the DNG images do not support it.

Again, this is based on a limited number of images only, take it with a grain of salt.

juanmaasecas
juanmaasecas Senior Member • Posts: 1,102
Re: R5 Low ISO Noise Reduction (cont.)

J A C S wrote:

Thanks.

I played with some underexposed DNG files posted by a youtube reviewer (shots of a boy with the RF 50). The darkest one is 3 stops underexposed. I do not see NR in the border pixels! With the uniformly lit images, I see it in the really dark images only which are not images for any photographic purposes. The ones that are half completely black and the other half terribly dark but not black show NR in the borders.

I can speculate that the NR triggers on images that are horribly underexposed, like the ones dpreview shoots! Another option would be to be applied on very dark parts only but this is hard to test, and the DNG images do not support it.

Again, this is based on a limited number of images only, take it with a grain of salt.

From those samples you mention, I don’t see significant differences with the files with the R regarding DR and noise, specially since the R files need to be pushed much less to achieve the same exposure. The R5 has some magenta cast in the shadows, the R is more neutral to my eyes.

 juanmaasecas's gear list:juanmaasecas's gear list
Canon EOS M50 Sony a7 III Canon EOS R Canon EF 50mm f/1.2L USM Canon EF 50mm F1.4 USM +9 more
J A C S
OP J A C S Forum Pro • Posts: 16,960
Re: R5 Low ISO Noise Reduction (cont.)

juanmaasecas wrote:

J A C S wrote:

Thanks.

I played with some underexposed DNG files posted by a youtube reviewer (shots of a boy with the RF 50). The darkest one is 3 stops underexposed. I do not see NR in the border pixels! With the uniformly lit images, I see it in the really dark images only which are not images for any photographic purposes. The ones that are half completely black and the other half terribly dark but not black show NR in the borders.

I can speculate that the NR triggers on images that are horribly underexposed, like the ones dpreview shoots! Another option would be to be applied on very dark parts only but this is hard to test, and the DNG images do not support it.

Again, this is based on a limited number of images only, take it with a grain of salt.

From those samples you mention, I don’t see significant differences with the files with the R regarding DR and noise, specially since the R files need to be pushed much less to achieve the same exposure. The R5 has some magenta cast in the shadows, the R is more neutral to my eyes.

All this is academic, done with some statistical and Fourier analysis. The real differences might be in the non-linearity responsible for that shadow cast you mention (which I do not see in those samples, BTW), possible banding, heat related noise, etc. With the 5D4, I have seen bands in the deep shadows created by bright spots elsewhere in the image; a black frame would not reveal them at all.

Keyboard shortcuts:
FForum MMy threads