My 5DM2 vs 5DM3 High ISO measurements

Started Mar 8, 2012 | Discussions
Horshack
Veteran MemberPosts: 3,921
Like?
My 5DM2 vs 5DM3 High ISO measurements
Mar 8, 2012

In an effort to roughly calculate the High ISO improvement of the 5DM3 vs 5DM2, I've done std. deviation measurements of 5DM2 and 5DM3 High ISO images from the images at imaging resource (summary/prediction at very bottom of this post). All raws processed in ACR 6.7 RC using 'Process 2010' with all ACR defaults except chrominance and luminance NR were disabled and white-balance was custom set (5DM2 wb=4950/-21 and 5DM3 wb=5350/-20). I generated custom ACR linear color profiles for each camera using the Adobe DNG editor on the color-checker chart in each respective ISO 100 image.

I compared the 5DM2 ISO 6400 image to the 5DM3 ISO 12800 image..with the working assumption that the ISO improvement of the 5DM3 is somewhere between 0 stops and 1 stop.

I matched exposures using the CC white patch, which produced a -0.021 stop exposure adjustment on the 5DM3 image, which was normalized using a negative exposure adjustment on the 5DM3 image layer.

Here's a 100% crop of the two images, 5DM2 ISO 6400 on left, 5DM3 ISO 12800 on right:

Using that same crop, here are the std. deviation measurements for luminance only:

Here are the std. deviation color measurements for the Red, Green, and Blue patches: (mean/std. dev)

Red patch:
5DM2 ISO 6400: red=88.72/10.62, green=43.42/6.79, blue=33.20/7.91
5DM3 ISO 12800: red=82.85/11.17, green=42.26/7.29, blue=33.93/9.24

Green patch:
5DM2 ISO 6400: red=57.72/7.91, green=84.10/7.18, blue=46.44/6.82
5DM3 ISO 12800: red=59.45/8.17, green=86.66/7.81, blue=49.95/7.90

Blue patch:
5DM2 ISO 6400: red=37.52/7.54, green=34.96/6.89, blue=83.25/9.03
5DM3 ISO 12800: red=40.62/7.86, green=37.24/7.84, blue=86.82/10.71

To help make sense of the 5DM2 ISO 6400 vs 5DM3 ISO 12800 measurements and provide back-correlation, I performed the same measurements on the 5DM2 ISO 6400 vs 5DM2 ISO 12800 images.

White patch:
5DM2 ISO 6400: 169.29/7.04
5DM2 ISO 12800: 167.96/9.17

Middle-darker gray:
5DM2 ISO 6400: 68.78/6.72
5DM2 ISO 12800: 67.02/9.69

Black patch:
5DM2 ISO 6400: 23.94/7.20
5DM2 ISO 12800: 21.63/10.33

Red patch:
5DM2 ISO 6400: red=88.58/10.60, green=43.31/6.81, blue=33.19/7.94
5DM2 ISO 12800: red=85.02/14.60, green=43.15/9.53, blue=30.49/13.27

Green patch:
5DM2 ISO 6400: red=57.78/7.79, green=84.08/7.19, blue=46.56/6.78
5DM2 ISO 12800: red=53.69/13.03, green=82.88/9.96, blue=44.56/10.60

Blue patch:
5DM2 ISO 6400: red=37.27/7.41, green=34.89/6.80, blue=83.27/8.86
5DM2 ISO 12800: red=35.11/11.98, green=34.40/10.53, blue=81.92/13.08

Here is a 100% crop of the 5DM2 ISO 6400 and 5DM2 ISO 12800 images:

Summary/Prediction

Based solely on the noise measurements above and not detail (thus assuming Canon didn't add some type of raw NR on the 5DM3), and assuming the nominal ISO calibrations between the bodies are the same and my exposure measurement/adjustment methodology is correct...by eyeballing the std. deviation numbers for 5DM2 ISO 6400 vs 5DM3 ISO 12800 and then correlating those to 5DM2 ISO 6400 vs 5DM2 ISO 12800 numbers, it appears to me that the 5DM3's improvement for ISO 6400/12800 over the 5DM2 is between 0.60 and 0.80 stops. My margin of error is 0.60 to 0.80 stops

Canon EOS 5D
If you believe there are incorrect tags, please send us this post using our feedback form.
aaront
Senior MemberPosts: 1,120Gear list
Like?
Re: My 5DM2 vs 5DM3 High ISO measurements
In reply to Horshack, Mar 8, 2012

Horshack wrote:

...by eyeballing the std. deviation numbers for 5DM2 ISO 6400 vs 5DM3 ISO 12800 and then correlating those to 5DM2 ISO 6400 vs 5DM2 ISO 12800 numbers, it appears to me that the 5DM3's improvement for ISO 6400/12800 over the 5DM2 is between 0.60 and 0.80 stops. My margin of error is 0.60 to 0.80 stops

So, after all is said and done, we get 3/4 stop more ISO improvement, better AF, 2 FPS more speed, oh, and a 1 MP increase. After 3+ years and $1,400 more than the 5D MKII.

I think I can easily pass this generation. Should have been called the 5D MKIIn.

And before the forum flames me, check my profile for equipment.

-- hide signature --

Aaron Thomson

 aaront's gear list:aaront's gear list
Canon EOS 5D Mark II +2 more
Reply   Reply with quote   Complain
teodorian2
Senior MemberPosts: 1,860
Like?
Re: My 5DM2 vs 5DM3 High ISO measurements
In reply to Horshack, Mar 8, 2012

Horshack wrote:

Summary/Prediction

Based solely on the noise measurements above and not detail (thus assuming Canon didn't add some type of raw NR on the 5DM3), and assuming the nominal ISO calibrations between the bodies are the same and my exposure measurement/adjustment methodology is correct...by eyeballing the std. deviation numbers for 5DM2 ISO 6400 vs 5DM3 ISO 12800 and then correlating those to 5DM2 ISO 6400 vs 5DM2 ISO 12800 numbers, it appears to me that the 5DM3's improvement for ISO 6400/12800 over the 5DM2 is between 0.60 and 0.80 stops. My margin of error is 0.60 to 0.80 stops

Here is another measurement:

http://translate.google.se/translate?hl=sv&sl=sv&tl=en&u=http%3A%2F%2Fwww.fotosidan.se%2Fforum%2Fshowpost.php%3Fp%3D1797654%26postcount%3D391

Reply   Reply with quote   Complain
Dominique Dierick
Veteran MemberPosts: 4,467Gear list
Like?
Very informative. Thanks n/t
In reply to Horshack, Mar 8, 2012
-- hide signature --

My equipment: Nikon D3s, D700, D7000, EOS 450D
The ladies equipment: Panasonic LX5 and GH2 for video

 Dominique Dierick's gear list:Dominique Dierick's gear list
Fujifilm X100S
Reply   Reply with quote   Complain
Horshack
Veteran MemberPosts: 3,921
Like?
FFTs
In reply to Horshack, Mar 8, 2012

Here are FFTs of the 5DM2 and 5DM3 ISO 12800 images using a 512x512 background patch of the Imaging Resource images. FFTs can be used to check for the presence of raw noise reduction.

Composite (RGB), 5DM2 on top and 5DM3 on bottom

Red Channel, 5DM2 on top and 5DM3 on bottom

Green Channel, 5DM2 on top and 5DM3 on bottom

Blue Channel, 5DM2 on top and 5DM3 on bottom

For comparison, here's what the 5DM2 FFT looks like after noise reduction as been applied using the CS5 NR filter:

Reply   Reply with quote   Complain
stephenmelvin
Veteran MemberPosts: 4,547
Like?
the problem with your comparison...
In reply to Horshack, Mar 8, 2012

...is that it only compares quantity of noise, but not the quality of the noise.

In other words, it completely ignores pattern noise, which some people inexplicably refer to as "banding."

The samples I've seen have been nearly free of high ISO pattern noise. To the point where even the ISO 50,000 looks usable for some purposes.

Previously, I'd reject a 5D II image at 12,800 for any purpose whatsoever, other than surveillance. The pattern noise is just too objectionable.

I look forward to tests once these cameras start hitting the market. Mostly, I look forward to comparing them myself.

-- hide signature --
Reply   Reply with quote   Complain
Mighty Mouse
Regular MemberPosts: 223
Like?
Re: 1D Mk IV
In reply to Horshack, Mar 8, 2012

So I would guess that the 5D III would be about only 1/2 stop better than the 1D Mk IV since the 1D Mk IV is about 1/2 to 3/4 better than the 5D II in Raw?

I also would think that the 1Dx would be about 1 stop better than the 1D Mk IV in raw due to the bigger pixels. 2 stops better in jpg.
--
Jim

Reply   Reply with quote   Complain
Horshack
Veteran MemberPosts: 3,921
Like?
Re: My 5DM2 vs 5DM3 High ISO measurements
In reply to teodorian2, Mar 8, 2012

Thanks for that link. His measurements, which I'm sure are much more precise than mine, give the 5DM3 a 0.50 stop advantage over the 5DM2 (for raw).

Reply   Reply with quote   Complain
stephenmelvin
Veteran MemberPosts: 4,547
Like?
could you explain these?
In reply to Horshack, Mar 8, 2012

I'm not sure how to interpret these. If there's zero noise reduction, what would they look like? A solid gray patch? Do these show NR?

-- hide signature --
Reply   Reply with quote   Complain
Horshack
Veteran MemberPosts: 3,921
Like?
Re: the problem with your comparison...
In reply to stephenmelvin, Mar 8, 2012

stephenmelvin wrote:

...is that it only compares quantity of noise, but not the quality of the noise.

In other words, it completely ignores pattern noise, which some people inexplicably refer to as "banding."

The samples I've seen have been nearly free of high ISO pattern noise. To the point where even the ISO 50,000 looks usable for some purposes.

Previously, I'd reject a 5D II image at 12,800 for any purpose whatsoever, other than surveillance. The pattern noise is just too objectionable.

I look forward to tests once these cameras start hitting the market. Mostly, I look forward to comparing them myself.

I agree, this is only a quantitative measurement not qualitative. I also agree that the absence of High ISO banding makes a big difference in the subjective noise improvement of the 5DM3.

Reply   Reply with quote   Complain
Horshack
Veteran MemberPosts: 3,921
Like?
Re: could you explain these?
In reply to stephenmelvin, Mar 8, 2012

stephenmelvin wrote:

I'm not sure how to interpret these. If there's zero noise reduction, what would they look like? A solid gray patch? Do these show NR?

To my eyes the FFTs indicate there is no NR but I'm a noob at FFT analysis. Here are some useful links:

http://qsimaging.com/ccd_noise_interpret_ffts.html
http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p4.html

Reply   Reply with quote   Complain
stephenmelvin
Veteran MemberPosts: 4,547
Like?
indeed...
In reply to Horshack, Mar 8, 2012

Horshack wrote:

I agree, this is only a quantitative measurement not qualitative. I also agree that the absence of High ISO banding makes a big difference in the subjective noise improvement of the 5DM3.

This is where things get interesting. The objective reduction of noise may be under a stop (looking forward to direct testing of production models by people who know what they're doing), it looks like the practical improvement in image quality could be enough to let us use the entire ISO range of the camera.

-- hide signature --
Reply   Reply with quote   Complain
jeromeindahood
New MemberPosts: 18
Like?
Re: My 5DM2 vs 5DM3 High ISO measurements
In reply to Horshack, Mar 8, 2012

wow. basically spending 3.5 k for essentially an updated firmware then!?

Reply   Reply with quote   Complain
tony field
Veteran MemberPosts: 5,804
Like?
Re: My 5DM2 vs 5DM3 High ISO measurements
In reply to Horshack, Mar 8, 2012

Excellent comparison. It is very nice to see a person technically inclined who is completely willing to show the methodology used for his analysis. The process you have seems completely reasonable to me and quantifies my "eyeball guesses" when I compare my 5D-II images with the web's 5D-III.

Thank you
--
tony
http://www.tphoto.ca

Reply   Reply with quote   Complain
Peter 13
Veteran MemberPosts: 8,301
Like?
Re: My 5DM2 vs 5DM3 High ISO measurements
In reply to Horshack, Mar 8, 2012

Horshack wrote:

Thanks for that link. His measurements, which I'm sure are much more precise than mine, give the 5DM3 a 0.50 stop advantage over the 5DM2 (for raw).

This is my prediction, just based on eyeballing. Between 1/3 stop and 1/2 stop at ISO 1600-3200.

BTW, what do you get at ISO 1600 and ISO 3200? Many people would not go higher than that.

Reply   Reply with quote   Complain
Pultzar
Senior MemberPosts: 1,424
Like?
ISO 100 noise is the problem
In reply to stephenmelvin, Mar 8, 2012

There are some new studio shots on imaging-resources of a few kids with a dark background at ISO 100. Even without pushing you can see banding when viewing 100%. If you push the exposure then it comes out more clearly.

It is vertical on the sensor and not part of the background. This is proven by looking at images shot in landscape mode and portrait mode. The portrait mode has horizontal banding because the image has been rotated.

Reply   Reply with quote   Complain
didd
Regular MemberPosts: 261
Like?
Re: FFTs
In reply to Horshack, Mar 8, 2012

Thanks for doing that, Horshack

I was suspecting some trick in the red channel due to the few hot pixels in the images at high iso, and the lack of resolution discussed on the IR images at low iso in the red channel.
Seems not the case at 12k8.
Could be interesting to look at iso 100.

I have another question
In this test,
http://forums.dpreview.com/forums/read.asp?forum=1032&message=40825115

if we forget the banding, I'm very surprised by the high level of the standard deviation, compared to the 5D2 . 9.3 vs 6.5, that's quite incredible ...

Reply   Reply with quote   Complain
Robert H Crutcher
Junior MemberPosts: 38
Like?
Re: My 5DM2 vs 5DM3 High ISO measurements
In reply to Horshack, Mar 9, 2012

Interesting analysis. Could you indicate where you find the RAW 5DII files on IR. I've searched all over the site and all i find is the jpeg files at 0 noise reduction. These files are significantly worse than the 5DIII RAW files, i.e., blurry.

Regards,

Bob Crutcher

Reply   Reply with quote   Complain
rwbaron
Veteran MemberPosts: 7,707Gear list
Like?
Re: ISO 100 noise is the problem
In reply to Pultzar, Mar 9, 2012

Pultzar wrote:

There are some new studio shots on imaging-resources of a few kids with a dark background at ISO 100. Even without pushing you can see banding when viewing 100%. If you push the exposure then it comes out more clearly.

Really?

I was the one who asked for the push and I thought the 100% crops looked decent, certainly not perfect but very usable for anything other than 100% pixel peeping.

It is vertical on the sensor and not part of the background. This is proven by looking at images shot in landscape mode and portrait mode. The portrait mode has horizontal banding because the image has been rotated.

What do you shoot with your camera? Dark subjects under moon light?

Bob
--
http://www.pbase.com/rwbaron

 rwbaron's gear list:rwbaron's gear list
Canon PowerShot G9 Canon PowerShot S90 Canon PowerShot G1 X Canon EOS 50D Canon EOS 7D +20 more
Reply   Reply with quote   Complain
bronxbombers
Forum ProPosts: 18,226
Like?
Re: My 5DM2 vs 5DM3 High ISO measurements
In reply to Horshack, Mar 9, 2012

Horshack wrote:

it appears to me that the 5DM3's improvement for ISO 6400/12800 over the 5DM2 is between 0.60 and 0.80 stops. My margin of error is 0.60 to 0.80 stops

well my 2/3 fits nicely right in there

so does my margin of error

Reply   Reply with quote   Complain
Keyboard shortcuts:
FForum MMy threads