A Level Playing Field? Not for Sony.

Started Nov 8, 2008 | Discussions thread
Flat view
Mark VB Senior Member • Posts: 2,912
A Level Playing Field? Not for Sony.

In the recent blog posting regarding the use of Adobe Camera RAW as a standard converter for comparing the performance of different cameras ( http://blog.dpreview.com/editorial/2008/11/adobe-camera-ra.html#more ), reference was made to trying to "create a level playing field" and making "results - as much as possible - comparable between cameras." Also, reference was made to wanting "to show a camera's image output in its "purest" form, i.e. as captured by the sensor and as little as possible altered by sharpening algorithms, noise reduction or any other forms of image processing."

I agree with this approach and have no issue with this approach as described in the blog posting.

But, this approach is not similarly applied when it comes to the use of camera created jpegs, which Phil has stated in another post are "the most relevant and important" ( http://forums.dpreview.com/forums/read.asp?forum=1000&message=29861966 ).

For example, in the recent review of the Sony A900, "fine" jpegs are used for analyzing the image quality of the A900. For the A900, this is a middle level jpeg. The best quality jpeg in the camera is "extra fine." Thus, in the case of the A900 (and other Sony DSLRs), the reviews/tests are not using the image output with as little image processing as possible. By contrast, for each of the other three cameras to which the A900 was compared, the D700, 1DS Mk III and 5D, the use of "fine" jpegs for those cameras represented their highest quality jpegs (according to the dpreview tests for each one). Thus, for all three cameras against which the A900 was compared, the jpegs were the highest quality jpeg the cameras produced, but not the A900.

Nowhere in the review was this stated or described. I have no idea whether there are any visible differences between an "extra fine" and "fine" jpeg from the A900 (or any other Sony DSLR). But, if there are such differences due to the higher compression used to create the smaller file size "fine" jpeg, they would most likely show up in the actual pixel views that are used to compare the image quality of different cameras.

Now, there may be a logical reason why this was done (such as the fact that when shooting RAW+jpeg in the A900 and other Sony's you are limited to a "fine" jpeg, and one assumes that for efficiency reasons the camera testers are shooting RAW+jpeg in order to do a test shot once to get both a RAW and jpeg file). But, this is not in keeping with the previously stated desire to compare cameras on a level playing field.

I have tried raising this privately (through the feedback link), and in questions posted in other threads in which the subject arose, all to no avail - i.e., I have not seen or received a response as to why this different test procedure is used for the Sony cameras. Given that I shoot only RAW, this doesn't really have an impact on me personally, but for the many other site readers that led Phil to state that jpegs are the "most relevant and important," it does.

So, in the absence of an explanation for this anomalous test methodology, I decided to post this comment in a new thread and see what happens.

While it is a criticism of the dpreview tests, it is intended as a constructive one. That is, with the hope that it will lead to more consistent tests which will only serve to improve the reliability and usefulness of the tests.

Finally, as I am on the road and will have no access to the internet for several days after tonight, I will not be able to monitor this thread or respond to any posts made for about six days. So, for those who are interested, I look forward to seeing your comments when I am again able to access the internet.

-- hide signature --

Mark Van Bergh

Flat view
Post (hide subjects) Posted by
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark MMy threads
Color scheme? Blue / Yellow