Why DXO overall scores are meaningless

Started 3 months ago | Discussions thread
ForumParentFirstPreviousNextNext unread
Flat view
Albert Silver
Contributing MemberPosts: 734Gear list
Like?
Why DXO overall scores are meaningless
3 months ago

DXO provides excellent measurements that are absolutely great tools to compare various aspects of a sensor's performance, and lenses, but the one score that is completely meaningless is the overall score. This is true of both their sensor and lens scores.

Let's consider the Nikon APS-C D5200 and the Canon 1Dx. The Nikon is rated at 84 and the 1Dx is an 82. This would be ok if we know how these overall scores were calculated, but there is no transparency in the process, which means that despite attempts to be a reference, they can never be accepted as such. Let's take a quick look at what this Nikon crop sensor is doing to be considered superior to the top-of-the-line Canon model:

Overall score

OMG!! A $500 camera that outperforms a $6000 camera...

Of course, DXO provides detailed charts showing how each aspect measures up, and how the Nikon sensor is superior. I will filter nothing, selectively highlight nothing. There are only five charts, so bear with me.

ISO Sensitivity

In ISO sensitivity the 1Dx beats the Nikon at all levels, and includes an ISO 50 the Nikon does not offer.

SNR 18% (Signal-to-Noise Ratio)

Here too it is a wash in favor of the Canon sensor. Not a big surprise of course, but we are trying to determine why the Nikon sensor is rated as superior to the 1Dx. The answer comes in:

Dynamic Range

Aha! In the first three stops we can see that Nikon has greater dynamic range, with ISO 800 about equal, after that it is a wash in favor of the Canon. Needless to say the Canon can also reach ISOs the Nikon cannot, and anything at ISO 1600 or higher is hugely in favor of the Canon. Still, we have found one localized area where the Nikon is measurably superior: DR at ISO 100-400.

Tonal Range

Tonal range also favors the Canon by a large amount, and at all ISOs.

Color Sensitivity

This one is also generally better for the Canon. The Nikon does have 11 ISO better color sensitivity at ISO 100 (it scores 80 instead of 69) and similarly for ISO 200, but after that the Canon gets the upperhand.

Conclusion

The only area the Nikon sensor really outperformed the Canon sensor was at ISO 100-400 in dynamic range, but this was nullified at ISO 800, and all other ISOs were massively in favor of the Canon. In every other area measured, the Canon was either even or much superior. Still, the overall score is Nikon 5200 - 84, and Canon 1Dx - 82.

It thus becomes clear that DXO consider DR at ISO 100-400 to be more important than every other measurement and factor put together even if they are in the opposite favor.

This does not detract from the excellent charts that allow one to scrutinize and compare all measurements, but the overall score? Bah.

 Albert Silver's gear list:Albert Silver's gear list
Canon EOS 6D Canon EF 85mm f/1.8 USM
Canon EOS-1D Nikon D5200
If you believe there are incorrect tags, please send us this post using our feedback form.
ForumParentFirstPreviousNextNext unread
Flat view
Post (hide subjects)Posted by
No.New
ForumParentFirstPreviousNextNext unread
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark post MMy threads
Color scheme? Blue / Yellow