Reading mode: Light Dark

Why the religious belief in DxO ratings?

Go to last post
Big Ga said:
OzRay said:
User reviews are one thing, but I just see people quoting DxO scores everywhere, yet in the real world they are pretty much meaningless.
Click to expand...
To clarify:

Are you specifically referring to the DxO final overall Scores only, or are you also stating that All the data from DxO is pretty much meaningless in the real world ?
Click to expand...
I'm simply talking about how people use DxO scores as if they were something writ in stone and reflect what is good and what is not.
 
OzRay said:
Big Ga said:
To clarify:

Are you specifically referring to the DxO final overall Scores only, or are you also stating that All the data from DxO is pretty much meaningless in the real world ?
Click to expand...
I'm simply talking about how people use DxO scores as if they were something writ in stone and reflect what is good and what is not.
Click to expand...
Can you post a few links where people did that? I don't recall seeing it. Thanks.
 
OzRay said:
There appears to be an insatiable urge to use DxOMark ratings as some irrefutable bible of camera/lens capability/quality, when in reality the results are little more than the equivalent of fuel economy ratings for cars worked out from relevant country EPA test requirements.
Click to expand...
Can you post a few links where people did that? I don't recall seeing it. Thanks.
 
cainn24 said:
Moti said:
bclaff said:
I'm only going to address sensors rather than lenses since that's the area I know.

Many people shop for a car without regard to specifications such as engine horsepower.
But then again, some people factor this into their decision process.
Imagine a world where the car manufacturers didn't specify horsepower; wouldn't someone who measured and published horsepower data be helpful?

For camera sensors I think the DxOMark scores have utility.
(But the overall score is not relevant, at least not to me.)

I perform measurements that overlap with those of DxOMark.
Some people appreciate an unbiased, non-commercial double check.
And I provide details that some people find important that cannot be found at DxOMark.

My work relies on dozens of people who have collaborated with me to gather data for their camera.
The M4/3s world is under-represented, probably because many feel as you do that such research is unimportant. (Pentaxians seem to feel the same.)
No problem, if people don't want the information there is no sense in making an effort to provide it.
Click to expand...
Specs are nice and some of them may even be important but I don't think there is any serious driver who buys a car he doesn't know, without drive testing it.

Same with photographic gear.
Click to expand...
Some people drive every car in much the same way even though the handling and performance characteristics of different vehicles can vary quite significantly. Other people make a m

ore concerted effort to adjust their strategy to the point where they are playing to the specific strengths of the vehicle they find themselves in. People in the latter category are more likely to see stated specifications start to translate into real-life experiences because they are specifically pushing those particular boundaries.

Same with photographic gear.
Click to expand...
No it is not, because unlike driving, photography is a visual art and craft, therefore the ultimate test should be based on real world photo tests and not on DXO mark only, as stated in the OP.

Moti

--
http://www.musicalpix.com
 
Last edited:
Henry Richardson said:
OzRay said:
There appears to be an insatiable urge to use DxOMark ratings as some irrefutable bible of camera/lens capability/quality, when in reality the results are little more than the equivalent of fuel economy ratings for cars worked out from relevant country EPA test requirements.
Click to expand...
Can you post a few links where people did that? I don't recall seeing it. Thanks.
Click to expand...
 
Gidday Moti

It's not such a bad analogy, in some ways. Many (most?) people steer cars from point to point, and that's fine. Others really enjoy driving and get a real buzz out of it. Regardless of how much we love driving, we continue to use cars to go to the shops and similar pedestrian purposes. Pun intended ... :-D

Same with photography. Many use a camera solely for personal documentary purposes, and that's equally fine. Some of us like to take it a few steps further and try to become photographers, while (mostly) continuing to take lots and lots of fairly pedestrian documentary shots.

--
Regards, john from Melbourne, Australia.
.
Please do not embed images from my web site without prior permission
I consider this to be a breach of my copyright.
-- -- --
.
The Camera doth not make the Man (nor Woman) ...
Perhaps being kind to cats, dogs & children does ...
.
Galleries: http://canopuscomputing.com.au/gallery2/v/main-page/



C120644_small.jpg





Bird Control Officers on active service.
 
Henry Richardson said:
OzRay said:
There appears to be an insatiable urge to use DxOMark ratings as some irrefutable bible of camera/lens capability/quality, when in reality the results are little more than the equivalent of fuel economy ratings for cars worked out from relevant country EPA test requirements.
Click to expand...
Can you post a few links where people did that? I don't recall seeing it. Thanks.
Click to expand...
Just make a search in the forum. You'll find plenty of time samples.

Moti
 
John King said:
Gidday Moti

It's not such a bad analogy, in some ways. Many (most?) people steer cars from point to point, and that's fine. Others really enjoy driving and get a real buzz out of it. Regardless of how much we love driving, we continue to use cars to go to the shops and similar pedestrian purposes. Pun intended ... :-D
Click to expand...
Sure, but my point was that I don't think anyone would by a car he doesn't know, no matter what the purpose is, without drive testing it. Was I wrong?
John King said:
Same with photography. Many use a camera solely for personal documentary purposes, and that's equally fine. Some of us like to take it a few steps further and try to become photographers, while (mostly) continuing to take lots and lots of fairly pedestrian documentary shots.
Click to expand...
Of course, I agree to that too but this doesn't contradict the fact that the ultimate test for photography gear is real world photo tests, which was the OP topic.

Moti
 
Perhaps a better automotive example would be crash worthiness.
This is something that you can't evaluate with a simple test drive.
Or perhaps you want to tow something heavy; probably horsepower is important to you but again you can't test it easily.
Perhaps you care whether the car has a timing chain or a timing belt or something else internal that is not easily observed.

To some people the fine points of sensor performance and the hints about the internals working are part of their decision process or influence how they use their camera.
For example, generally there is an ISO setting above which raising ISO is not helpful; determining that ISO setting can be done objectively.
Some astro-photographers care about things at the electron level; you can't get that by test driving the camera.

Again, not for everyone, but useful to more people than I think you give credit.
 
I've never looked at them either before or after purchase but so often I see people here getting all hot and bothered about them. May they enjoy long and rewarding lives. :)
 
bclaff said:
Perhaps a better automotive example would be crash worthiness.
This is something that you can't evaluate with a simple test drive.
Or perhaps you want to tow something heavy; probably horsepower is important to you but again you can't test it easily.
Perhaps you care whether the car has a timing chain or a timing belt or something else internal that is not easily observed.
To some people the fine points of sensor performance and the hints about the internals working are part of their decision process or influence how they use their camera.
For example, generally there is an ISO setting above which raising ISO is not helpful; determining that ISO setting can be done objectively.
Some astro-photographers care about things at the electron level; you can't get that by test driving the camera.

Again, not for everyone, but useful to more people than I think you give credit.
Click to expand...
 
OzRay said:
bclaff said:
Perhaps a better automotive example would be crash worthiness.
This is something that you can't evaluate with a simple test drive.
Or perhaps you want to tow something heavy; probably horsepower is important to you but again you can't test it easily.
Perhaps you care whether the car has a timing chain or a timing belt or something else internal that is not easily observed.
To some people the fine points of sensor performance and the hints about the internals working are part of their decision process or influence how they use their camera.
For example, generally there is an ISO setting above which raising ISO is not helpful; determining that ISO setting can be done objectively.
Some astro-photographers care about things at the electron level; you can't get that by test driving the camera.

Again, not for everyone, but useful to more people than I think you give credit.

--
Bill (visit me at http://www.photonstophotos.net )
Click to expand...
But the fact is, in equivalent terms, DxO doesn't test or provide information on any of those issues. That's the whole point of questioning their value and why some hang on to their every word.

DxO does little more than put a car on a dyno and evaluate the maximum power/torque that engine provides, given the dyno that they use and the conditions at the time (dyno results are very sensitive to type, conditions and operator). They may also provide some vague fuel economy evaluations while on the dyno and maybe emissions results.

There is absolutely nothing 'real world' about what they test. DxO is all about getting clicks and being talked about, hopefully generating revenue by being some sort of defacto authority on photographic quality.
Click to expand...
Not unlike VW, Audi, SEAT, Skoda, Porsche diesel engine cars and their emissions testing ...
Reality? What's that?
OzRay said:
--
Thoughts, Musings, Ideas and Images from South Gippsland
http://australianimage.com.au/wordpress/
Click to expand...


--
Regards, john from Melbourne, Australia.
.
Please do not embed images from my web site without prior permission
I consider this to be a breach of my copyright.
-- -- --
.
The Camera doth not make the Man (nor Woman) ...
Perhaps being kind to cats, dogs & children does ...
.
Galleries: http://canopuscomputing.com.au/gallery2/v/main-page/



C120644_small.jpg





Bird Control Officers on active service.
 
Moti said:
Henry Richardson said:
OzRay said:
There appears to be an insatiable urge to use DxOMark ratings as some irrefutable bible of camera/lens capability/quality, when in reality the results are little more than the equivalent of fuel economy ratings for cars worked out from relevant country EPA test requirements.
Click to expand...
Can you post a few links where people did that? I don't recall seeing it. Thanks.
Click to expand...
Just make a search in the forum. You'll find plenty of time samples.
Click to expand...
Well, I looked at the thread he suggested I look at:

http://www.dpreview.com/forums/post/56568003

I didn't see anyone claiming DxO is the "irrefutable bible of camera/lens capability/quality" but maybe you can point it out?
 
OzRay said:
There appears to be an insatiable urge to use DxOMark ratings as some irrefutable bible of camera/lens capability/quality, when in reality the results are little more than the equivalent of fuel economy ratings for cars worked out from relevant country EPA test requirements. As soon as the rubber hits the road, the results become meaningless.
Click to expand...
I think there's a difference between religious belief and scientific certainty. The average score is meaningless but the individual scores do place a very real relative way to compare cameras in sensor key characteristics, and makes it very valuable when making choices based on sensor.
OzRay said:
Much, much, better are actual results from users with lenses/cameras in real world use, all the better if they are actual photography enthusiasts and not journos that are writing for some photography mag or the like. Again with the car analogy, motoring journos are not the ideal source for information on cars that might suit your needs.

Do people really consider their lens/camera selections on DxO results?
Click to expand...
For choosing a camera body? It sure is a data point *if and only if you know how to use it well*. i.e. match the scores and characteristics of a sensor to what you are looking for. And know they are not measuring camera ergonomics, how does a camera feel in use, etc.

If you are aware of those things, DxoMark is very useful in the aspect of sensor measurement- which is what they are doing with those scores.
OzRay said:
--
Thoughts, Musings, Ideas and Images from South Gippsland
http://australianimage.com.au/wordpress/
Click to expand...
--
Raist3d/Ricardo (Photographer, software dev.)- I photograph black cats in coal mines at night...
“The further a society drifts from truth the more it will hate those who speak it.” - George Orwell
 
Last edited:
Moti said:
cainn24 said:
Moti said:
bclaff said:
I'm only going to address sensors rather than lenses since that's the area I know.

Many people shop for a car without regard to specifications such as engine horsepower.
But then again, some people factor this into their decision process.
Imagine a world where the car manufacturers didn't specify horsepower; wouldn't someone who measured and published horsepower data be helpful?

For camera sensors I think the DxOMark scores have utility.
(But the overall score is not relevant, at least not to me.)

I perform measurements that overlap with those of DxOMark.
Some people appreciate an unbiased, non-commercial double check.
And I provide details that some people find important that cannot be found at DxOMark.

My work relies on dozens of people who have collaborated with me to gather data for their camera.
The M4/3s world is under-represented, probably because many feel as you do that such research is unimportant. (Pentaxians seem to feel the same.)
No problem, if people don't want the information there is no sense in making an effort to provide it.
Click to expand...
Specs are nice and some of them may even be important but I don't think there is any serious driver who buys a car he doesn't know, without drive testing it.

Same with photographic gear.
Click to expand...
Some people drive every car in much the same way even though the handling and performance characteristics of different vehicles can vary quite significantly. Other people make a m

ore concerted effort to adjust their strategy to the point where they are playing to the specific strengths of the vehicle they find themselves in. People in the latter category are more likely to see stated specifications start to translate into real-life experiences because they are specifically pushing those particular boundaries.

Same with photographic gear.
Click to expand...
No it is not, because unlike driving, photography is a visual art and craft, therefore the ultimate test should be based on real world photo tests and not on DXO mark only, as stated in the OP.
Click to expand...
People who adjust their strategy until they are playing to the strengths of whatever camera/lens combination they are using are more likely to see a correlation between controlled camera performance testing and real world results.

If I conduct some properly controlled indoor camera performance experiments (which I often do) that reveal that one of my lenses is sharper than one of my other lenses I can always, without exception, realize that advantage out in the real world so long as I effectively deal with the larger number of variables that are typically present, which I generally always can. Furthermore if I had not conducted those controlled tests I might not have known that that additional performance was on offer and therefore might not have tried so hard to find it.

This being the case the notion that the results of controlled tests do not correlate with the results that are achievable out in the real world simply doesn't gel with my own experience.
 
Henry Richardson said:
Moti said:
Henry Richardson said:
OzRay said:
There appears to be an insatiable urge to use DxOMark ratings as some irrefutable bible of camera/lens capability/quality, when in reality the results are little more than the equivalent of fuel economy ratings for cars worked out from relevant country EPA test requirements.
Click to expand...
Can you post a few links where people did that? I don't recall seeing it. Thanks.
Click to expand...
Just make a search in the forum. You'll find plenty of time samples.
Click to expand...
Well, I looked at the thread he suggested I look at:

http://www.dpreview.com/forums/post/56568003

I didn't see anyone claiming DxO is the "irrefutable bible of camera/lens capability/quality" but maybe you can point it out?
Click to expand...
 
Moti said:
John King said:
Gidday Moti

It's not such a bad analogy, in some ways. Many (most?) people steer cars from point to point, and that's fine. Others really enjoy driving and get a real buzz out of it. Regardless of how much we love driving, we continue to use cars to go to the shops and similar pedestrian purposes. Pun intended ... :-D
Click to expand...
Sure, but my point was that I don't think anyone would by a car he doesn't know, no matter what the purpose is, without drive testing it. Was I wrong?
Click to expand...
Actually the more I have personally investigated the camera performance equation in controlled conditions, and seen it correlate with what is achievable out in the real world, the more meaningful the test results produced by outfits like DxO have become to me. This is essentially the end result of a process of demystification. Once you get a handle on all the variables, and understand how they all feed into the final image, considering individual performance characteristics in isolation becomes far more useful.
 
Moti said:
zxaar said:
OzRay said:
peppermonkey said:
Besides, with guys like Robin Wong who can make a brownie take ridiculously good shots, any problems with my photos aren't with the equipment but mostly with the photographer, i.e. me.
Click to expand...
And that's where DxOMark fails miserably.
Click to expand...
If they are not playing i do not know how they fail and do it miserably.
Moti said:
It leads many people to believe that products that are more than capable of high quality results are incapable of such because of DxOMark scores.
Click to expand...
What user do is upto him. I am pretty sure you can buy the best camera according to them and shoot c ra p photos out of it and claim how ridiculously wrong dxo got it.
Click to expand...
Sure, if your purpose is to show off, there is nothing like DXO results to give you a great ego trip.

But if you are a serious photographer and you care about your photographs, nothing comes close to real world photo rests.
Click to expand...
Dxomark does not claim that you can not make good or great photos from lower ranked camera sensors.

Just to complain that a camera sensor testing site does not take into account human element is pretty stupid.

I still use R1 and that camera was released in 2005, you would not get anyone who does what he does regardless of how dxomark ranks sensors.

But it does not change the fact that they do not test what humans will do with that camera.
 
zxaar said:
Just to complain that a camera sensor testing site does not take into account human element is pretty stupid.
Click to expand...
Actually, a camera testing site (one that tests lenses and cameras) that doesn't take into account the human element is pretty stupid.
 
You must log in or register to reply here.

Similar threads

D
Sigma 300-600 4 DG OS Sports – Review After 14 Days of Testing
Replies
1
Views
2K
jerste
J
V
DxO measurements and SoftRAM
2
Replies
31
Views
36
John Sheehy
J
D
NEW member (insert cool line attracting attention)
Replies
0
Views
1
Devil Doc
D
J
This is why DXO should be ignored by photographers.
6 7 8
Replies
143
Views
144
alexisgreat
A
Anders W
The real deal -- FF 28 - 300mm v Oly's pro 40 - 150mm (continued)
4 5 6
Replies
103
Views
207
bobn2
bobn2

Keyboard shortcuts

f
Forum
m
My threads
Mobile site
About
Editorial content
Cameras & Lenses
Community
All content, design, and layout are Copyright © 1998–2025 Digital Photography Review All Rights Reserved.
Reproduction in whole or part in any form or medium without specific written permission is prohibited.
When you use DPReview links to buy products, the site may earn a commission.
©GPS Media - Guides, Products, Services.
Back
Top