Moon Maid's sensor comparison

Started Feb 29, 2012 | Discussions
Marianne Oelund Veteran Member • Posts: 7,753
Moon Maid's sensor comparison
1

Alright, I'll admit it: Every time I see someone claim that camera A is "X stops" better than camera B for high-ISO use, it makes me cringe. Generally, it will fail to be true, and there isn't a single number that can characterize sensor performance - or the difference between sensors. In fact, one of the points of this exercise is to demonstrate how close the current-generation cameras are, at least across the mid- to light-tone range.

Here, I would like to share with you how my engineering mind visualizes sensor performance with respect to noise, considering a few cameras which are presently quite topical. These are SNR (signal-to-noise ratio) curves at some selected ISO settings, which give an assessment of freedom from noise over a wide tonal range.

The vertical scale is in dB. At 40dB, SNR is 100:1 and noise is barely perceptible. At 30dB, noise is fairly visible at large print size, but would not be considered objectionable by most users. At 20dB, SNR is only 10:1 and you will want to use noise-reduction software to obtain a quality print, and at 10dB, results may still be considered only fair after careful NR. The horizontal scale is EV, corresponding approximately to what your camera's exposure meter would indicate on a blank wall; sensor saturation is typically just above +3EV.

Since this includes cameras which are not yet available in production form, please keep in mind that the data is preliminary and subject to change. However, the analysis was performed with great care, and I am not expecting significant changes.

The curves represent only shot noise (photon noise) and read noise. Pattern noise, which is only of practical import across the upper part of the tonal range at low ISO settings, has not been included. This is partly due to the difficulty of making the measurement - the sensor must be extremely clean and my cameras are rarely in this condition.

Often, cameras are compared on a full-image basis, i.e., as though they would all be used to make the same size prints. To support this particular point of view, 12Mp-normalized curves are included. Personally, I also believe that per-pixel performance is a valid comparison approach for certain output sizes, so those curves are included as well. You may interpolate between the two curve types to find how the camera would perform at various downsampling ratios.

One final point I would like to emphasize: Recent marketing claims regarding high-ISO improvements are referring to the camera's JPEG output, not the basic sensor performance, and rely on camera processing throughput and NR algorithm improvements. This is great news for JPEG shooters, but such claims have little bearing on post-processed results from RAW files.

Canon EOS 10D Canon EOS 20D Canon EOS 30D Canon EOS 40D
If you believe there are incorrect tags, please send us this post using our feedback form.
Antal I Kozma Senior Member • Posts: 2,833
Re: Moon Maid's sensor comparison

Marianne,

Thanks for the comparison. So, for clarification to the scientifically challenged: your analysis expects the D800 to perform below the D3 in image noise. Am I correct?

The reason for asking is since I have plenty of experience in shooting with the D3 when light conditions were challenging, concerts in dimly lit venues and so forth. The D3 did a very good job shooting at ISO 3200.

I am considering getting a D800 but would like to use it in similar dimly lit situations at ISO 3200. Am I correct to say that it will under perform in comparison to the D3?

Thanks,
AIK

 Antal I Kozma's gear list:Antal I Kozma's gear list
Nikon AF-S Micro-Nikkor 105mm F2.8G IF-ED VR Phase One Capture One Pro Nikon D810 Nikon D500 Nikon AF-S Nikkor 24-70mm f/2.8G ED +17 more
OP Marianne Oelund Veteran Member • Posts: 7,753
Keep output size in mind

Antal I Kozma wrote:

Thanks for the comparison. So, for clarification to the scientifically challenged: your analysis expects the D800 to perform below the D3 in image noise. Am I correct?

That's according to the red curve, showing the D800's per-pixel performance, which is only important for huge prints or heavy cropping. If you print at about the same size with the D800, it generally out-performs the D3/D700 (see the green curves).

At the most common print sizes, the D800 performs admirably at high ISO.

ssh33
ssh33 Regular Member • Posts: 182
Re: Moon Maid's sensor comparison

Marianne Oelund wrote:

These are SNR (signal-to-noise ratio) curves at some selected ISO settings

Good data, thank you Marianne!

alabaster Senior Member • Posts: 1,581
Re: Moon Maid's sensor comparison

ssh33 wrote:

Marianne Oelund wrote:

These are SNR (signal-to-noise ratio) curves at some selected ISO settings

Good data, thank you Marianne!

Thank you very much. This is an extremely useful way of presenting the data. What the 6400 ISO curves are telling me is that the D800 will be a bit noisier than the D4 deep in the shadows, but will otherwise be extremely competitive.
--
Geoff B
http://www.gbphoto.com.au/

 alabaster's gear list:alabaster's gear list
Nikon D810 +1 more
AgentHEX Contributing Member • Posts: 676
Re: Moon Maid's sensor comparison

How is the SNR measured for these graphs?

Marianne Oelund wrote:

Alright, I'll admit it: Every time I see someone claim that camera A is "X stops" better than camera B for high-ISO use, it makes me cringe. Generally, it will fail to be true, and there isn't a single number that can characterize sensor performance - or the difference between sensors. In fact, one of the points of this exercise is to demonstrate how close the current-generation cameras are, at least across the mid- to light-tone range.

Here, I would like to share with you how my engineering mind visualizes sensor performance with respect to noise, considering a few cameras which are presently quite topical. These are SNR (signal-to-noise ratio) curves at some selected ISO settings, which give an assessment of freedom from noise over a wide tonal range.

geronimo Regular Member • Posts: 304
Re: Moon Maid's sensor comparison

Where did you get your "preliminary" and "estimated" data from?

Where can we go to see that data for ourselves?

Please post the images used for gathering your data.

fabgo Senior Member • Posts: 1,300
Re: Moon Maid's sensor comparison

Thanks Marianne.

Awesome work. Both the D4 and the D800 look very, very good.

-- hide signature --

Fabian

Ray Soares Veteran Member • Posts: 3,121
Thanks for sharing! nt
-- hide signature --

Ray Soares

See my pictures at http://www.pbase.com/raysoares

 Ray Soares's gear list:Ray Soares's gear list
Nikon D7000 Nikon D4 Nikon AF-S Nikkor 70-200mm f/2.8G ED VR II Nikon AF-S Nikkor 24mm f/1.4G ED Nikon AF-S Nikkor 85mm f/1.4G +1 more
Luke Kaven Veteran Member • Posts: 5,712
Re: Moon Maid's sensor comparison

Thanks for doing this service to the community, and for this added perspective.

I'm just trying to assimilate it. In the ISO6400 exposure, I was surprised to see the behavior of the D3s on a par with the D4-at-12MP, and surprised to see both of those cameras diverge from the rest beginning at -2EV.

Do you think that there is any noise reduction going on there? Or what? I wonder if this corresponds to the bump around ISO25600 on the earlier DR measurements? On the DR chart this ran contrary to my expectations.

sebastian73 Contributing Member • Posts: 598
Thanks Marianne

Thanks a lot Marianne

Could you give us (the non technicians) a short statement your opinion regarding the D4 and D800.
--

Nikon D3s, AFS 16-35/4.0G, AFS 35mm/1.4G, AFS 24-70/2.8G, AFS 85mm/1.4G, AFS 105mm/2.8VR, AFS 70-200/2.8VRII, AFS 200/2.0VR, TC-1.4E II, TC-2.0E III, SB-800

Gitzo GT3541LS tripod with Markins M20, Gitzo GM3551 monopod with RRS MH-01 head

 sebastian73's gear list:sebastian73's gear list
Nikon D4 Nikon D810 Nikon AF-S Nikkor 16-35mm F4G ED VR Nikon AF-S Nikkor 70-200mm f/2.8G ED VR II Nikon AF-S Nikkor 200mm f/2G ED-IF VR +5 more
GlobalGuyUSA Senior Member • Posts: 2,016
Re: Keep output size in mind

This is very interesting -- but I'm confused as to "how much better" the D4 & D800 perform than their predecessors. I guess your numbers are specific, but reading this chart just isn't easy for me.

How much better is each camera than the D3 (at 12MP) in "stops"?

What do your X and Y axes represent?? Thanks!

-- hide signature --

Sincerely,

GlobalGuyUSA

 GlobalGuyUSA's gear list:GlobalGuyUSA's gear list
Nikon D810 Nikon AF Nikkor 135mm f/2D DC Nikon AF-S Nikkor 85mm f/1.4G Nikon AF-S Nikkor 58mm f/1.4G Nikon AF-S Nikkor 20mm f/1.8G ED +9 more
brunobarolo Senior Member • Posts: 1,118
Re: Moon Maid's sensor comparison

So much for the bad noise produced by higher pixel numbers Thanks, Marianne!

BTW, considering how close the D4 is to the D3s, these graphs seem to show that sensor quality reached sort of a plateau with the D3s sensor and any further improvements will be slow and small (until some completely new technology may come around some day).

Flashlight Veteran Member • Posts: 7,867
Great, thanks...

...even I can understand this...

...I think

-- hide signature --

Philip

AgentHEX Contributing Member • Posts: 676
Re: Keep output size in mind

GlobalGuyUSA wrote:

This is very interesting -- but I'm confused as to "how much better" the D4 & D800 perform than their predecessors. I guess your numbers are specific, but reading this chart just isn't easy for me.

How much better is each camera than the D3 (at 12MP) in "stops"?

Well, that's rather the point the of her rant, but the a "stop" above would be either a unit of 7 in Y or 1 in X so you can judge for yourself.

What do your X and Y axes represent?? Thanks!

X is exposure and Y is SNR.

-- hide signature --

Sincerely,

GlobalGuyUSA

AgentHEX Contributing Member • Posts: 676
Re: Keep output size in mind

OMG lol this is hilarously wrong as I was trying to be cute and precise w/ maple but log() is E instead of 10.

So, I lied, it's nominal 3, not 6.93 on the Y scale.

AgentHEX wrote:

GlobalGuyUSA wrote:

This is very interesting -- but I'm confused as to "how much better" the D4 & D800 perform than their predecessors. I guess your numbers are specific, but reading this chart just isn't easy for me.

How much better is each camera than the D3 (at 12MP) in "stops"?

Well, that's rather the point the of her rant, but the a "stop" above would be either a unit of 7 in Y or 1 in X so you can judge for yourself.

What do your X and Y axes represent?? Thanks!

X is exposure and Y is SNR.

knickerhawk Veteran Member • Posts: 6,050
Re: Moon Maid's sensor comparison

Thanks! Very nice illustration of what's going on. Are you in any position yet to comment on differences in color performance of the new sensors relative to the old ones?
--
My photos: http://www.pbase.com/imageiseverything/root

J Mankila
J Mankila Veteran Member • Posts: 4,234
Thank you! :) Would the D3X fit in there?

Thank you for taking the time to help us. But would it be easy to include the D3X? I think that's quite interesting, especially at lower ISOs. A graph showing D3X at 24MP and D800 at both 24MP and 36MP would be interesting, especially at ISO 100.

Or is the image quality so high there that it makes no practical difference?

-- hide signature --

regards
Janne Mankila, Finland

jenella
jenella Senior Member • Posts: 1,009
Re: Moon Maid's sensor comparison

Marianne Oelund wrote:

Since this includes cameras which are not yet available in production form, please keep in mind that the data is preliminary and subject to change. However, the analysis was performed with great care, and I am not expecting significant changes.

Did she actually handle the cameras herself or gather info from other sources? If the latter is true... its just more speculation IMO despite all the fancy charts and graphs.

 jenella's gear list:jenella's gear list
Fujifilm X100S Nikon D700 Nikon D7000 Nikon D4 Nikon AF-S Nikkor 14-24mm f/2.8G ED +8 more
RicksAstro
RicksAstro Veteran Member • Posts: 3,856
Re: Moon Maid's sensor comparison

If these graphs hold true and if pattern noise isn't an issue, the D800 will really be an ISO-less camera. If you push the ISO200 graph 5 stops, it seems to line up about perfectly with the ISO6400. I don't know if that was your extrapolation or based on actual measurements. If on measurements, that's great news! I could see being able to SW push process the ISO 200 image, assuming the SW applies the correct tone curves and such with such extreme processing.

Even at the pixel level at ISO200, the D800 has cleaner shadow than the D3s starting about EV -5.5. At the normalized level, it beats everything but the D4 normalized and even there, the D800 has better shadows.

Thanks so much for the effort you put into this!

-- hide signature --
Keyboard shortcuts:
FForum MMy threads