READ THIS--Questions Regarding the Validity of Popular PHotographys SD14 tests

Started Jun 16, 2007 | Discussions thread
Flat view
Gary Dean Mercer Clark
Gary Dean Mercer Clark Veteran Member • Posts: 5,551
READ THIS--Questions Regarding the Validity of Popular PHotographys SD14 tests

I question the validity of the resolution tests performed by Popular Phootgraphy magazine as while they boast a million dollar testing lab and their lens test results are superb, their camera tests results are questionable especially in regards to the recent SD14 tests in the July 2007 Issue. In the article, you'll notice that they make a note that images were converted to .tiff files prior to the resolution tests. Why? Because the software that they use only supports Bayer sensored cameras and and there is not raw support for the foveon .x3f file format. What does this mean? First of all is the question of whether Pop Photo used the Sigma SPP 3.0 software to convert the Sigma raw images to .tiff format and what settings were used. Secondly, if Adobe Raw converter 4.1 was used there is some controversy regarding this plug in to adobe photoshop, as some users have found that it actually seems to blur raw files imported into the program forcing the users to have to sharpen the images significantly prior to importing into the Adobe raw converter. I'm hoping that Popular Photography will take a serious look at the the Adobe raw converter program and report their findings to readers.

With our lack of knowledge as to what program was used to convert the the Sigma SD14's raw images to .tiff files, we are left scratching our heads wondering how accurate were the image quality test performed by Popular Photography? Could this account for the lower than expected resolution results in the tests?

The programs used for testing were designed for testing bayered sensored cameras, not the unique foveon sensor and if the raw images from the camera were not used for the tests, but converted to .tiff files--how accurate are the results?--while I have respect for Popular Photography's effort, the lower than expected test results for a camera billed as a 14 megapixal camera are suspect at the least and in my opinion do not tell the whole story. Its hard to believe that McNamara, one of the industry's brilliant minds and experts would stand by these test results and not question whether the testing methods used are 100 percent accurate and a true reflection of the SD14's optical performance.

Finally, in order to insure that the lense tests were as accurate as possible, Sigma Corporation sent two Sigma SD14s for the test and calibrated a Sigma 30mm ex lense and kit lense for each camera respectively. This would insure that the tests results were as accurate as possible and give the highest possible results. This is a common industry practise. Popular Photography does not tell readers what lenses are used for testing resolution and longtime Sigma DSLR users know that the quality of the lense used has a big impact on the optical resolution achieved with their cameras. So with specially calibrated lenses specific to each camera, I would expect much better test results than were reported by Popular Photography. Did they leave the lenses on the cameras they were calibrated for, or did they interchanged between cameras, further putting the test results into question?

In conclusion, industry methods of testing cameras and explaining the results to consumers have always been a point of argument between manufacturers and reviewers. Without accuracy and fairness, it is the consumers who lose in this argument.

Popular Photography further makes the statement that the only people this camera is for are current Sigma SLR film camera users who want to upgrade to digital. This is not a fair statement in regards to the SD14. Why are only current Sigma camera users the only consumers who would be interested in this camera? Sound like someone's opinion without being based on fact to me.

Finally,Other reviewers, like the Editor of Australia's ProPhoto magazine have stated that the SD14 blows away the D200 in image quality and resolution. His review was fair and balance, noting the SD14's strengths and weaknesses and indicated performance of the SD14 in league with the highest megapixal Bayer sensored SLRs currently available, but for much less. How is it possible for such a huge difference in reviewer tests of the SD14 in the industry? In my opinion, the current tests being used by many industry reviewers are biased towards Bayer sensored cameras and Foveon has even proposed new testing methods that have not yet been adopted by the industry. Foveon claims that the color resolution tests they use give a more accurate reflection of the foveon chip's capabilities. Why are popular photography and other reviewers so reluctant adopt new higher testing standards for color resolution?

Popular Photography has a responsiblity to perform accurate and fair tests of cameras as it has a large readership and can easily deal a death blow to any camera's sales which is something that should never be taken lightly. In my opinion having used the SD14 in real life shooting, I find its performance and image quality extremely high and it performs much better than reported by Popular Photography in real life use.

McNamara's claims that Foveon is hyping the foveon chip and misleading consumers is no different than Popular Photography's test results on the SD14.

I hope this discussion will encourage others to contact Popular Photography in writing and give their opinions regarding the testing protocols used. While I still believe that Pop Photo's lense testing is about the best in the industry, the magazine has lost it credibility, in my opinion with it's bayer-centric camera testing and questionable testing methods.
Gary Mercer
Test Photos Gallery:

 Gary Dean Mercer Clark's gear list:Gary Dean Mercer Clark's gear list
Canon EOS 5D Mark II Canon EOS 7D Sony a77 II Canon EF 50mm f/1.4 USM Canon EF 85mm f/1.8 USM +17 more
Flat view
Post (hide subjects) Posted by
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark MMy threads
Color scheme? Blue / Yellow