D300s review: false premises, false conclusions

Started Nov 16, 2009 | Discussions thread
ForumParentFirstPreviousNextNext unread
Flat view
Tom Christiansen
Senior MemberPosts: 2,239
Like?
D300s review: false premises, false conclusions
Nov 16, 2009

http://www.dpreview.com/reviews/nikond300s/page15.asp reads:

Finally let's take a look a the D300's RAW output next to the competition. Removing any in-camera noise reduction and processing the images using Adobe Camera Raw gives us the nearest thing to a 'level playing field' for assessing the relative noise levels of the four cameras' sensors.

That statement is provably false. All they are testing is how effective ACR is at decoding one camera's raw format compared with how effective ACR is at decoding another camera's raw format. That's it. They are not testing the cameras. No conclusions can be drawn about the cameras from these tests.

The proof is simple. Take your own raw shot at an ISO anywhere between 1600 and 3200. Convert to TIFF once using ACR and once using NX2. In both, explicitly disable all noise reduction and sharpening, and make sure they use the same white balance.

Flip back and forth between the two images. You may do this in ViewNX easily enough, or you may pull both images into Photoshop superimposing one over the other in separate layers flipping the opacity.

Watch how much the image changes. ACR is much worse. Therefore, what they write above is false, and what they write below is indeterminable.

If you don't have a D300 raw file handy, you may use mine:

http://mox.perl.com/photos/D300-RawNoiseTest/2009-11-08_17-18-27__TSC4708.NEF

It's shot at ISO 1600. The difference between ACR and Nikon is astounding. But don't trust me: use your own eyes. That will be a lot better than my posting comparison shots. Do your own; use my raw file or use your own. You'll see.

With noise reduction turned off we get a more accurate idea of how noisy these sensors are

No, they don't. See above.

and the image looks slightly different to what we've seen above in the JPEG section of this page. There's not a significant difference between any of these cameras - if anything, the most telling thing is how similarly the sensors of three different manufacturers perform.

Using a common raw converter makes clear that

No; it makes nothing clear. ACR applies different settings to different cameras. That's why they have different camera profiles. That's why the profile for 12-bit and 14-bit raws is different. etc etc. All this does is test ACR's profiles, nothing more. And we know historically suspect those are.

much of the difference between the cameras' performances in JPEG are, unsurprisingly, down to decisions made by the manufacturers about noise reduction levels. In RAW, these differences are removed from the equation, giving a better impression of what the sensors are doing. As you can see, there's very little to choose between them.

That's not what their own tests are telling them. I believe their tests are fundamentally flawed, but even accepting their results, you cannot accept their summary because it does reflect what their own data shows.

Examine the final raw chroma noise graph they claim to drawing their conclusions from. The D300s is in red; the Canon 7D is in green.

Look where the green line is at ISO 100. Where does the red line cross that same point? At ISO 250. That shows the D300s is fully 4/3 of a stop less noisy at low ISO than the 7D is. This is what their graph shows at calibrated sensitivities:

C (n/a) ≠ N “100”
C (n/a) ≠ N 200
C 100 ≅ N 250
C 200 ≅ N 400
C 400 ≅ N 550
C 800 ≅ N 1100
C 1600 ≅ N 1800
C 2000 ≅ N 2000
C 2500 ≅ N 2200
C 3200 ≅ N 2500
C “4000” ≅ N 3200

This is their owb data, not mine. Even with their bogus ACR use, it's pretty dramatic, isn't it? So how can they possibly come to the counter-factual conclusion above? Their own data says otherwise, but they say nothing. What's that about?

And why does it all look like boiler-plate text that we've read before? Could that be because it is?

Take nothing for granted. Run your own tests. Compare ACR with Nikon, then decide for yourselves if their comparison is even vald. Look at the graph for yourself. Do you believe it justifies the conclusion they say they draw from it? Why or why not?

-- hide signature --

tom
--
colicky, havocker, picnicky, panicking, picnicking,
panicky, magicked, colicking, picnicked, bivouacking,
colicked, mimicked, frolicked, picnicker, demosaicked ,
garlicky, mimicker, havocking, bivouacked, demosaicker ,
havocked, panicked, mimicking, frolicking, demosaicking .

ForumParentFirstPreviousNextNext unread
Flat view
Post (hide subjects)Posted by
....New
ForumParentFirstPreviousNextNext unread
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark post MMy threads
Color scheme? Blue / Yellow