No, I agree with your conclusion. But ahhhghh! Not another
reference to the wretched meaningless scores. There is NO
difference between the top three in that list. Two tenths is indeed
good enough. It means there is zero difference!
I'll have to stop looking at this thread, I think.
Sorry I'm late to this party! Claypaws, you've hit the nail on the head with the conclusions being drawn by shootsmarter.com.
1) They ranked the test results from best to worst totally ignoring the precision of the data. Psychophysical test scores based on integer value rank scoring with untrained observers using only 76 respondants is not going to produce scoring accuracy to two decimal places and in all likelihood not even to the tenths unit that separates these results. The top three cameras undoubtedly have no statistical difference as Claypaws noted, and even the whole list from best to worst may or may not have statistically differentiable scores depending on the standard deviation about the mean in the data set. Excel could easily have calculated it and then we'd know, but Smartshooter.com didn't publish it!
2) They instructed the participants poorly. They describe it this way: "Remember, sharpness, color accuracy and contrast was not judged - just how satisfied you would be with the quality of the skintone if you were viewing the print of your wife or daughter at home. Nothing more". I hate to be critical because they did make a very valiant effort to perform an objective test, but how does a human being do face detection anyway? You have to locate the human's face by looking at patterns of color and tone, i.e, all of which involves making conscious or subconscious decisions about color, sharpness and contrast. Otherwise you couldn't find the eyes, lips, nose, cheekbones, etc.
3). I downloaded the images representing "best" and "worst" scores (wish they'd let us see them all, but I didn't find them), opened in PS, converted to LAB mode and looked at histograms of L, a, and b data for the skin tone areas. Best image was slightly more red (~ +5a), and with a hint more yellow (~ +2b) on average. This made the skin in the "best" file slightly more saturated and a hint more reddish compared to the image from the worst score, but this tolerance would never be maintained by typical photo lab prints so on any given day in any given lab, the color component from either camera might have produced more pleasing print color. The biggest difference in these files is the tonal break points. If you analyze the background neutral gray in each print you will see that they did nail the exposure very tightly. Yet the skintone reproduction in the "worst" image has noticeably higher contrast with shadows being rendered about the same as those in the best image but highlights being rendered noticeably lighter. Tonal breakpoints, i.e, where highlights, midtones, and shadows fall can have a very large influence on the perceived shape and colorfulness of a human face. Portrait photographers exploit tonal rendering, for example, by choosing broad or short portrait lighting techniques to help make skinny faces look fuller and round faces look thinner. The tonal curve shape is the most significant factor between the "best" and "worst" cameras in this test. It isn't huge but it is noticeable. Moreover, the cameras are close enough in performance that either file could be tweaked to get to a match print, so does the test imply the "worst" camera will consistently need more PP to produce optimum prints? One can hardly conclude that based on this test's sample size of just one scene!
4). As noted by numerous others in this thread, shooting the S3 in standard and not wide DR mode took away any advantage or disadvantage it might have had in this test. S3 was not represented in this test the way most S3 owners shoot with it, but then again,none of the cameras should be declared victorious based on the "default" image of just one scene and judged for default printed output from a fuji
Frontier mini lab.
My conclusion: It is a fun, interesting, but statistically meaningless test. Pick your favorite camera, declare it the winner.
--
Mark McCormick