A Sound Method for Evaluating Signal/Noise Ratio

For inter-camera RAW comparisons of cams with the same sized image-sensor and pixel-resolution:

(1) Use the very same lens on each camera to be compared;

(2) Use Manual shooting mode, setting the F-Number and the Shutter Speed to be the same;

(3) For base ISO Gain testing, use the lowest ISO Gain setting available. For higher ISO Gain testing, use the manufacturer's rated ISO Gain (meaningful only in relation to JPG recording levels);

(4) Set each camera's white-balance reference color-temperature manually (meaning using the mode which allows the user to manually set the color-temperature, not the preset WB modes). Choose a common color-temperature appropriate for the brightest lighting (i.e, 5500 K for a sunlit day). So that the SNR at low light-levels can be analyzed, include dark shaded areas in image-frame

(5) Set any active Exposure Compensation to 0 EV, allowing camera metering to operate normally;

(6) Record test shots of the same scene (same camera-subject distance, same AOV, using the same focal-length) at close to the same time (i.e., on a clear day when light levels are not varying);

(7) Also record one intentionally highly overexposed test-shot (for each camera, at each ISO Gain).
.

(8) Download the free, stable, non-invasive RawDigger 0.9.11 for your platform here: http://www.rawdigger.com/news/rawdigger-0-9-11 . This will be used to determine inter-camera differences in RAW-level scaling. Install RawDigger . Leave all of the settings in Preferences - Data Processing in their default settings (so that black-level-subtraction will be properly applied).

(8)(A) For each camera, and for each ISO Gain tested, open the highly overexposed test-shot. Look (in main program screen, in the image statistics data column) for Max , for a common number reported in at least two (if not all), of the RGBG color-channels. Make a note of all those numbers.

(8)(B) For each test-shot from each camera at each ISO Gain tested, open the normally exposed test-shot. To maintain linearity in the test-shots to be analyzed, reject any test-shot where any one (or more) of the RGBG channels reaches a value that is equal to the recorded Max for that camera operating at that ISO Gain.

(8)(C) For the remaining valid test-shots to be analyzed, look (in main program screen, in the image statistics data column) for Avg . Add all of those numbers (for each of the RGBG channels) together using a calculator, and make a note of all those numbers.

(8)(D) For each valid test-shot from each camera at each ISO Gain used, calculate the Peak/Average Ratio of each of the individual test-shots by dividing the Max value [from (8)(A) above] by the Avg value [from (8)(B) above]. Make a note of the results of the calculation. The RAW-level scaling of each test-shot is known. This information will be used in making comparisons.
.

(9) Download the the free, stable, non invasive RAW Therapee 4.09.x for your platform here: http://rawtherapee.com/downloads . This will be used to convert the RAW test-shots recorded - free from from the (silent, unknown, and differing) RAW-level scalings that various commercial RAW processors (i.e., Adobe LR/CR, Silkypix, DxO Optics Pro, etc.) commonly applying to RAW image-files. RPP (which runs on Mac op-systems only) is another good option to RAW Therapee 4.09x .

Use the following common settings when RAW processing all test-shots in RAW Therapee 4.09x :

Exposure tools:
Neutral
Highlight recovery amount=0
Black=0
Shadow recover=0
Brightness=0
Contrast=0
Saturation=0
Lab Brightness=0
Lab Contrast=0
Lab Saturation=0
Tone Curve=Linear
All other functions disabled.

Exp. Comp . [ See Section (10) below for instructions for choosing this setting-value ]
.

Detail tools:
Contrast by detail levels disabled.
All other functions disabled.
.

Color tools:
White Balance = Camera Settings
Color Management:
Input profile=No profile
Working profile=sRGB
Output profile=sRGB
Outpt gamma=standard_g=2.2
All other controls in their default state.
.

Raw tools:
De-mosaicing algorithm=AMaZE
False color suppression steps=0
All other functions disabled.
.

(10) How to adjust Exp. Comp . for accurate SNR comparisons at all levels of "pushing shadows":
.

(10)(A) When comparing crops from the larger image-frame (for all except for the very darkest shadow areas) , increase the value of the Exp. Comp. (which is in EV units) for the test-image having the higher   Peak/Average Ratio ( PARhigh below) by the following amount (more positive in value) than the test-image having the lower Peak/Average Ratio ( PARlow below):

( Log[base 2] of ( PARhigh / PARlow ) ) / 2

Here is the same formula restated for the (more commonly available) Natural Logarithm function:

( Log[base e] of ( PARhigh / PARlow ) ) / 2.885390
.

(10)(B) When comparing crops from the larger image-frame (for only the darkest shadow areas, approaching black ) , increase the value of the Exp. Comp. (which is in EV units) for the test-image having the higher   Peak/Average Ratio ( PARhigh below) by the following amount (more positive in value) than the test-image having the lower   Peak/Average Ratio ( PARlow below):

( Log[base 2] of ( PARhigh / PARlow ) )

Here is the same formula restated for the (more commonly available) Natural Logarithm function:

( Log[base e] of ( PARhigh / PARlow ) ) / 1.442695