Unbiased noise test.

Started Feb 28, 2009 | Discussions thread
landmarks4711 Contributing Member • Posts: 906
Brilliant !!!! Includes the Downsample

maluminas wrote:

Hello DPR!

I have been lurking here for many many years, but never created an
account because for some reason DPR did not allow registration with
web emails such as Gmail. But i checked recently and now they do so
here i am, with a gift.

An unbiased noise test. Or as unbiased as i could possibly make it.

As everyone else, i have been frustrated by disinformation stemming
from religious brand fanatism. So i decided to make a noise test that
everyone would agree on, though i know this is impossible, but here's
my best try, anyway.

I post it on the Nikon forums, because i own a Nikon camera, but i'm
sure the other camps will see it too.

It stars the latest high-MP cameras, the D3X, the A900 and the 5D2. I
also included the D3, to see if it is still the high-ISO king. The
1Ds3 makes an appearance, because i was just curious to see if there
is any visible improvement in the 5D2, which uses a very similar
sensor. The D300 insisted on having a cameo, because its the camera i
own, but it also has the densest sensor of the selection, just for
fun.

The test was done in two parts. A "pixel performance" test, using
crops from full-size images, to assess the performance of individual
pixels regardless of density. The second part is made using crops
from images resized to 12MP. I call it the "print performance" test
because it simulates a comparison of the final output on paper.

I consider both methods to be valid, they just provide different
conclusions.

Here is the description of my method.

METHODOLOGY:
1) Download "bottle test scene" RAW files from Imaging Resource.
(Many thanks! Sorry for the bandwidth spike!)
2) Load all of them in Bibble 5 (pre-release, see notes below).
3) Apply same profile to all RAWs. (no NR, no sharpening, portrait
color profile, no lens corrections, "As Shot" WB).
4) Carefully adjust exposure compensation of each file using
rightmost green spike on the histograms as a reference. Fine-tune by
eye, very painstakingly...
5) When exposure consistency across all files is satisfactory, batch
to JPEG (highest quality).
6) For the print simulation only: use IrfanView's batch conversion
function to downsample to D3 resolution (Lanczos filter used for best
quality). I reused the full-size crops for the D3 and D300 since the
differences would have been negligible, plus it saved time...
7) Crop two 100x100 squares on each files (save as PNG). One from the
WhiBal card at the top, one from the center of the black cup at the
lower right.
8) Assemble all crops in GIMP, along with proper labels.
9) Include method at the bottom because this is going on DPR and we
all know what happens to tests like these on DPR when there is any
hint of doubt...
10) Export to JPEG, 90% quality.
11) Post on DPR
12) Put on flame-proof suit and sit in a bath of flame retardant.

Now i understand the exposure isn't EXACTLY the same on each crop,
but i decided to stop adjusting before i lost my sanity.

Notes on Bibble 5 pre-release: Why did i choose it? Because it
doesn't perform hidden operations on the images, and because it is
the only converter i found that could convert all the RAWs from these
very recent cameras. It is very promising, but since its a
pre-release test edition it is plagued by bugs for some people. It
converted the 24MP images in 3 seconds. The interface is very nice.
The new functions are also very nice.

And now! I suspect no one read my elaborate description because they
wanted to see the tests, so here they are!

Draw your own conclusions!

Post (hide subjects) Posted by
(unknown member)
(unknown member)
(unknown member)
(unknown member)
(unknown member)
(unknown member)
(unknown member)
(unknown member)
(unknown member)
(unknown member)
(unknown member)
(unknown member)
(unknown member)
(unknown member)
(unknown member)
(unknown member)
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark MMy threads
Color scheme? Blue / Yellow