DXOMARK.com, did you notice

Started 8 months ago | Discussions thread
agentul
Contributing MemberPosts: 597Gear list
Like?
Re: DXO scores are like MTF charts
In reply to tt321, 8 months ago

tt321 wrote:

agentul wrote:

DonSC wrote:

There simply isn't any perfect way to make the comparison.

I think they try their best. It's just not that easy.

the best way is to look at some actual photos (real life and test scenes). in the end, that is what you are buying the camera to do.

In order for this to work, ultimately one needs to do one's own testing, on scenes that one is interested in and explore issues one cares most about, and this is not realistic for most people, especially in the current commercial environment where population centres of substantial size (even over 1M inhabitants) sometimes don't have even one comprehensive camera shop or camera+lens rental facility and most people have to or prefer to rely on web retailers.

the internet has many review sites and blogs. it does take a while to look at them, but the advantage is that you can spread out the process across a month or two, so that you can get a better idea. doing one's own testing probably works for people who are professional photographers and can set aside dedicated time for this. as i said, test photos don't tell the whole story, nor do synthetic benchmarks of any kind. i prefer to combine everything along with the little technical understanding of cameras that i have in order to arrive to a conclusion. at least that's what i did when i was shopping for a camera.

i see that there are people for which DxO is the final word in camera performance. allow me to share a similar story: there are a few synthetic computer benchmarking tools out there, designed initially to simulate certain workloads such as archiving and gaming performance. one thing that has been noticed over the years in reviews is that some of these programs simulate conditions that are not really often encountered in real life. one such example is 3D Mark, which has grown to be more demanding than many high end modern games. this is one reason why CPU and video card reviews use real life games and programs - most people want to know how fast their game or video encoding will be, even with mid-level hardware. having 20% less points in a synthetic benchmark doesn't necessarily mean that you're getting sub-par hardware. it may mean that you're getting enough performance for your needs at a lower price.the other reason computer hardware reviews have moved away from synthetic benchmark programs is that some manufacturers have been caught cheating - optimizing drivers for the synthetic benchmarks used in reviews, while real life performance was lagging.

also, i recall some embarrassing articles back in 2008 when the X58 chipset was launched, and motherboard manufacturers were rushing to put out models that would be faster than the competition in terms of RAM performance. a closer look by AnandTech revealed that the BIOS did not match what was written on paper: even though a model was supposed to support more than 20GB of RAM, in real life it wasn't working properly with anything above the 3 or 6 GB used in reviews at the time.

another example would be how certain car manufacturers have been caught cheating at fuel efficiency tests by optimizing the ECUs to detect the test cycle and reduce fuel consumption to levels below what would be encountered in real life scenarios.

bottom line: synthetic benchmarks are useful, but the numbers should always be taken with a grain of salt.

 agentul's gear list:agentul's gear list
Panasonic Lumix DMC-GH2 Panasonic Lumix G Vario 14-42mm F3.5-5.6 ASPH OIS Panasonic Leica Summilux DG 25mm F1.4 Panasonic Lumix G Vario 45-150mm F4-5.6 ASPH Mega OIS Panasonic Lumix DMC-LZ2 +1 more
Reply   Reply with quote   Complain
Post (hide subjects)Posted by
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark post MMy threads
Color scheme? Blue / Yellow