Unbiased noise test.

Started Feb 28, 2009 | Discussions thread
Shop cameras & lenses ▾
Flat view
maluminas Regular Member • Posts: 178
Unbiased noise test.

Hello DPR!

I have been lurking here for many many years, but never created an account because for some reason DPR did not allow registration with web emails such as Gmail. But i checked recently and now they do so here i am, with a gift.

An unbiased noise test. Or as unbiased as i could possibly make it.

As everyone else, i have been frustrated by disinformation stemming from religious brand fanatism. So i decided to make a noise test that everyone would agree on, though i know this is impossible, but here's my best try, anyway.

I post it on the Nikon forums, because i own a Nikon camera, but i'm sure the other camps will see it too.

It stars the latest high-MP cameras, the D3X, the A900 and the 5D2. I also included the D3, to see if it is still the high-ISO king. The 1Ds3 makes an appearance, because i was just curious to see if there is any visible improvement in the 5D2, which uses a very similar sensor. The D300 insisted on having a cameo, because its the camera i own, but it also has the densest sensor of the selection, just for fun.

The test was done in two parts. A "pixel performance" test, using crops from full-size images, to assess the performance of individual pixels regardless of density. The second part is made using crops from images resized to 12MP. I call it the "print performance" test because it simulates a comparison of the final output on paper.

I consider both methods to be valid, they just provide different conclusions.

Here is the description of my method.


1) Download "bottle test scene" RAW files from Imaging Resource. (Many thanks! Sorry for the bandwidth spike!)
2) Load all of them in Bibble 5 (pre-release, see notes below).

3) Apply same profile to all RAWs. (no NR, no sharpening, portrait color profile, no lens corrections, "As Shot" WB).

4) Carefully adjust exposure compensation of each file using rightmost green spike on the histograms as a reference. Fine-tune by eye, very painstakingly...

5) When exposure consistency across all files is satisfactory, batch to JPEG (highest quality).

6) For the print simulation only: use IrfanView's batch conversion function to downsample to D3 resolution (Lanczos filter used for best quality). I reused the full-size crops for the D3 and D300 since the differences would have been negligible, plus it saved time...

7) Crop two 100x100 squares on each files (save as PNG). One from the WhiBal card at the top, one from the center of the black cup at the lower right.
8) Assemble all crops in GIMP, along with proper labels.

9) Include method at the bottom because this is going on DPR and we all know what happens to tests like these on DPR when there is any hint of doubt...
10) Export to JPEG, 90% quality.
11) Post on DPR
12) Put on flame-proof suit and sit in a bath of flame retardant.

Now i understand the exposure isn't EXACTLY the same on each crop, but i decided to stop adjusting before i lost my sanity.

Notes on Bibble 5 pre-release: Why did i choose it? Because it doesn't perform hidden operations on the images, and because it is the only converter i found that could convert all the RAWs from these very recent cameras. It is very promising, but since its a pre-release test edition it is plagued by bugs for some people. It converted the 24MP images in 3 seconds. The interface is very nice. The new functions are also very nice.

And now! I suspect no one read my elaborate description because they wanted to see the tests, so here they are!

Draw your own conclusions!

Flat view
Post (hide subjects) Posted by
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark MMy threads
Color scheme? Blue / Yellow