D4 vs. 1Dx cont'd.: What would be a better test methodology?

Started Sep 8, 2012 | Discussions thread
KLO82 Contributing Member • Posts: 907
Re: D4 vs. 1Dx cont'd.: What would be a better test methodology?

Here is a previous post of mine (with one correction, I mentioned D65 lighting previously instead of D55 lighting):

(Open Quote)

I was just thinking about how we can shoot with different cameras so that we can fairly compare their outputs visually. Here are some points I came up with (many are just my preferences not necessarily absolute requirements for fair testing):

1. We will use D55 daylight illuminant for testing. WB should be set for this light.

2. We will use a light meter which will show exposure as per Standard Output Sensitivity (SOS) definition of ISO speed.

3. Our test scene will contain an 18% gray card (among other things). The objects in the test scene will be arranged in such a way that even with relatively shallow DoF, all the objects will be in focus. This is to minimize the difference of DoF of different format cameras. This may also help to minimize focusing errors. We will also have something with high contrast which will be used as point of focus.

4. We will determine exposures for SOS full stop ISOs (eg ISO 100, 200, 400,….) using the light meter and gray card.
5. We will use sRGB for output color space.

6. We will use those manufacturer ISO settings that will give closest output in terms of brightness of 18% gray card (118RGB @ sRGB) for OOC jpeg. For example, let’s say we have determined recommended exposure for ISO 6400 using the meter and gray card. Using that exposure D4 needs ISO 5000 to show the gray card as mid gray, whereas 5D3 needs ISO 8000. We will use ISO 5000 for D4 and ISO 8000 with 5D3.

7. For comparing jpegs, even after selecting the closest ISOs, the gray card may show slightly higher or lower value than 118. We will adjust those in PP using the camera's bundled software. In this case, different camera's will show output with different tone curve. (This is to compare OOC jpegs, though we actually have changed brightness slightly in PP)

8. If we want to visually compare “raw images” (actually converted jpegs with minimum processing), we will use the sRGB tone curve and match the brightness to make the gray card as 118.
9. We will resize the images to a common size.

10. We will use stopped down prime lens with known T stop for the testing. Best is, like imaging-resource, using same lens which can be used in different cameras.

There are many other points which I (still) do not know what to do about. For example:

- which raw converter to use? The raw converter has to be “fair” to all brands/ cameras
- which resizing algorithms to use for upsizing/ downsizing?

- The objects in the test scene – what are the things should we include? There should be objects which will show luminance noise, color performance/ chroma noise, loss of sharpness/ detail, noise at the darkest tones etc.

this comparison will work properly for cameras which have manual exposure control and manual selection of ISO values.

I do not have any academic background in imaging science, and I know a lot less than many posters in this forum. I may have made many mistakes in this post. Would really appreciate if you can give constructive criticism.
(end quote)

One thing I should add now: the test scene must be uniformly lit.
My discussion was about visual comparison only.

 KLO82's gear list:KLO82's gear list
Canon EOS 550D Canon EOS 6D Canon EF 85mm f/1.8 USM Canon EF 24-105mm f/4L IS USM Canon EF 50mm f/1.8 STM
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark MMy threads
Color scheme? Blue / Yellow