DXOMARK.com, did you notice

Started Jan 28, 2014 | Discussions thread
agentul Contributing Member • Posts: 619
Re: DXO scores are like MTF charts

tt321 wrote:

I agree with most of what you said, but synthetic benchmarks serve other purposes as well, such as providing a stable and well-defined standard for a common discussion to be based on. A lot of academic work make huge uses of these benchmarks and that's a widely accepted practice so long as the methodology is supportable.

Your objections are not with the benchmarks themselves, but the way they have been misused.

indeed. working in a similar field (i basically do synthetic benchmarks for a living) i can tell you that certain requirements can be bypassed by sometimes optimizing specifically for the test environments. also, we actually have disclaimers that the results are not meant to show real world performance, but only relative performance. in the real world, the actual performance depends heavily on the end user.

On the other hand, it's probably very difficult and not worth the trouble to design a lens that is brilliant at MTF-50 but otherwise crappy in picture taking

well, a design team might not do it exactly like that, but what if they were to try to improve the performance at known test parameters, so that it would look better on paper? take the Panasonic 25 MM: if you read a lot of reviews, the general opinion seems to be that, while it is not as sharp as the 20mm, the images look better. also, the AF performance is improved in low light. so it becomes a subjective interpretation for each buyer as to what the advantages and disadvantages are, and these don't show up on any MTF charts.

as everyone would see with their own eyes from the results, even if this is possible.

well, don't underestimate the power of self-delusion. tell a person that a Nikon D3100 takes better pictures than a E-M1 enough times and with enough technical arguments, and they'll believe it. look at how people are starting to be obsessed about shutter shock in this forum. besides, we all know that bigger and more expensive is always better, right?

Fuel economy is somewhat different as a lot of people don't measure how much they are getting out of their fuel with any precision at all

but a lot of people do. besides, if it takes you a tank of fuel to get to somewhere with your old car, you're going to notice that it also takes about a tank with your new car, even though the dealer praised its fuel efficiency, right? most people will start asking questions at that point.

and sometimes even large differences are unnoticed.

people notice more things when they're the ones paying the bills.

Post (hide subjects) Posted by
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark MMy threads
Color scheme? Blue / Yellow