DXOMARK.com, did you notice

Started Jan 28, 2014 | Discussions thread
Anders W Forum Pro • Posts: 21,468
Re: DXO scores are like MTF charts

agentul wrote:

tt321 wrote:

I agree with most of what you said, but synthetic benchmarks serve other purposes as well, such as providing a stable and well-defined standard for a common discussion to be based on. A lot of academic work make huge uses of these benchmarks and that's a widely accepted practice so long as the methodology is supportable.

Your objections are not with the benchmarks themselves, but the way they have been misused.

indeed. working in a similar field (i basically do synthetic benchmarks for a living) i can tell you that certain requirements can be bypassed by sometimes optimizing specifically for the test environments. also, we actually have disclaimers that the results are not meant to show real world performance, but only relative performance. in the real world, the actual performance depends heavily on the end user.

On the other hand, it's probably very difficult and not worth the trouble to design a lens that is brilliant at MTF-50 but otherwise crappy in picture taking

well, a design team might not do it exactly like that, but what if they were to try to improve the performance at known test parameters, so that it would look better on paper? take the Panasonic 25 MM: if you read a lot of reviews, the general opinion seems to be that, while it is not as sharp as the 20mm, the images look better.

The 20 is indeed sharper than the 25 (at least wide open and thereabout, especially at the edges). A number of times, I have tried to ask people who claim that images from the 25 nevertheless look better, on the basis of the reasonably comparable test images provided by SLR Gear, what specifically they think looks better (in any other regard than those we can get from the difference in specs). I have so far come up empty-handed. Consequently, I strongly suspect that what we are talking about here is a Leica-badge placebo effect, which doesn't have a whole lot to do with benchmarks or the way the manufacturers adapt to them.

also, the AF performance is improved in low light. so it becomes a subjective interpretation for each buyer as to what the advantages and disadvantages are, and these don't show up on any MTF charts.

as everyone would see with their own eyes from the results, even if this is possible.

well, don't underestimate the power of self-delusion. tell a person that a Nikon D3100 takes better pictures than a E-M1 enough times and with enough technical arguments, and they'll believe it. look at how people are starting to be obsessed about shutter shock in this forum. besides, we all know that bigger and more expensive is always better, right?

Fuel economy is somewhat different as a lot of people don't measure how much they are getting out of their fuel with any precision at all

but a lot of people do. besides, if it takes you a tank of fuel to get to somewhere with your old car, you're going to notice that it also takes about a tank with your new car, even though the dealer praised its fuel efficiency, right? most people will start asking questions at that point.

and sometimes even large differences are unnoticed.

people notice more things when they're the ones paying the bills.

 Anders W's gear list:Anders W's gear list
Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Olympus E-M1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS Panasonic Lumix G Vario 7-14mm F4 ASPH +28 more
Post (hide subjects) Posted by
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark MMy threads
Color scheme? Blue / Yellow