DXOMARK lens resolution. A legitimate metric?

In an optical system resolution is multiplicative so every bit, be it from the lens or sensor, helps.
That statement is fundamentally wrong. The accuracy of every measurement chain (optical or other) is determined by the most inaccurate step in the chain. The only exception is when you are dealing with sampling noise and can average multiple samples.
The second paper specifically argues against this commonly held belief (page 24):
The reason for an overly pessimistic view is the misconception that only the resolution limit of the system determines the image quality and that it is identical to the resolution of the weakest link of this chain. This is not the case, though, since the curves are multiplied, or it is the case only if the optical system performs very
poorly.
You mix "resolution" and "image quality" in your response. The two are different. My statement was about resolution. Image quality can improve even when resolution goes down. But your statement about resolution being multiplicative is false (for resolution and for every other measurement chain).
 
Last edited:
In an optical system resolution is multiplicative so every bit, be it from the lens or sensor, helps.
That statement is fundamentally wrong. The accuracy of every measurement chain (optical or other) is determined by the most inaccurate step in the chain. The only exception is when you are dealing with sampling noise and can average multiple samples.
MTF’s do multiply.
Thank you for the non sequitur.
Optical resolution - Wikipedia

b87680106f330cc62257f478d520d7e1977e478b
 
This is a link to their test protocol:


A key quote extracted from that article:

”The score shows the amount of information captured by the lens on a given camera and how well the camera and lens perform together. However, the score does not reflect the intrinsic quality of the camera sensor.”

In other words this is a test of a specific lens on a specific camera model. It is not a measure of that same lens on a different camera model even within the same brand. You may have radically different results. Neither do the scores compare say a 400mm Sony with a 400mm Canon on different bodies. Also carefully note the weighting used for the overall score, making it essentially useless for many lens settings.

A couple years back a blog cited the Yongnuo 85 mm as a hidden gem, one of the top scoring lenses on DXO with a score of 51 demonstrating how one can be misled by misunderstanding the testing.



The lens scored very high at f/2 on a A7RIV. However, if you use the dropdown and pair to a different Sony body you immediately find that the same lens produces some very mundane results.
 
In an optical system resolution is multiplicative so every bit, be it from the lens or sensor, helps.
That statement is fundamentally wrong. The accuracy of every measurement chain (optical or other) is determined by the most inaccurate step in the chain. The only exception is when you are dealing with sampling noise and can average multiple samples.
The second paper specifically argues against this commonly held belief (page 24):
The reason for an overly pessimistic view is the misconception that only the resolution limit of the system determines the image quality and that it is identical to the resolution of the weakest link of this chain. This is not the case, though, since the curves are multiplied, or it is the case only if the optical system performs very
poorly.
You mix "resolution" and "image quality" in your response. The two are different. My statement was about resolution. Image quality can improve even when resolution goes down. But your statement about resolution being multiplicative is false (for resolution and for every other measurement chain).
The term "image quality" is an inherently arbitrary and subjective term, so I'm not going to waste time debating why everyone has to adhere to your definition of it. I don't define "image quality" in the way the author does either.

In this paper "image quality" is basically defined as system resolution, so in the context of its findings the terms are essentially interchangeable. And that system resolution is defined as the multiplication of the resolution MTFs of all the elements in the image chain (sensor * lens). If you have an objectively/empirically better definition of resolution that refutes what seems to be inherent to optics I'd love to see it.......
 
This is a link to their test protocol:

https://www.dxomark.com/dxomark-lens-camera-sensor-testing-protocol/

A key quote extracted from that article:

”The score shows the amount of information captured by the lens on a given camera and how well the camera and lens perform together. However, the score does not reflect the intrinsic quality of the camera sensor.”

In other words this is a test of a specific lens on a specific camera model. It is not a measure of that same lens on a different camera model even within the same brand. You may have radically different results. Neither do the scores compare say a 400mm Sony with a 400mm Canon on different bodies. Also carefully note the weighting used for the overall score, making it essentially useless for many lens settings.

A couple years back a blog cited the Yongnuo 85 mm as a hidden gem, one of the top scoring lenses on DXO with a score of 51 demonstrating how one can be misled by misunderstanding the testing.

https://petapixel.com/2022/08/03/hidden-gem-a-yongnuo-lens-is-ranked-in-the-top-3-best-on-dxomark/

https://www.dxomark.com/Lenses/Yongnuo/Yongnuo-YN-85mm-F18S-DF-DSM-mounted-on-Sony-A7R-IV__1326

The lens scored very high at f/2 on a A7RIV. However, if you use the dropdown and pair to a different Sony body you immediately find that the same lens produces some very mundane results.
We know that is how their data works. I just don't believe they are actually testing every single lens + body combination that they list.

For example they probably have over 100 Canon EF lenses. Each of those has at results for at least 5-6 apertures, and for zooms there's another 3-4 of each of those measurements at different focal lengths. So each EF mount body supposedly has at least 1500 tests done. And there are about 20 or so EF bodies in their database, so now you are talking about 30,000 tests. And that doesn't even get into averaging out multiple samples or decentering, so with that you could be into the hundreds of thousands of tests, with over 10,000 tests done just to add a new body.

Naw, I am certain they are just getting MTFs of each component at each setting and multiplying them to get these results. Which is fine- but I'd rather have the raw data to either do my own analysis on or to view without their subjective filtering and summarization. They have a ton of great data that they have kind of rendered useless. What is a P-mpix? What is acutance? How do either actually translate to photographs? I still don't know.
 
This is a link to their test protocol:

https://www.dxomark.com/dxomark-lens-camera-sensor-testing-protocol/

A key quote extracted from that article:

”The score shows the amount of information captured by the lens on a given camera and how well the camera and lens perform together. However, the score does not reflect the intrinsic quality of the camera sensor.”

In other words this is a test of a specific lens on a specific camera model. It is not a measure of that same lens on a different camera model even within the same brand. You may have radically different results. Neither do the scores compare say a 400mm Sony with a 400mm Canon on different bodies. Also carefully note the weighting used for the overall score, making it essentially useless for many lens settings.

A couple years back a blog cited the Yongnuo 85 mm as a hidden gem, one of the top scoring lenses on DXO with a score of 51 demonstrating how one can be misled by misunderstanding the testing.

https://petapixel.com/2022/08/03/hidden-gem-a-yongnuo-lens-is-ranked-in-the-top-3-best-on-dxomark/

https://www.dxomark.com/Lenses/Yongnuo/Yongnuo-YN-85mm-F18S-DF-DSM-mounted-on-Sony-A7R-IV__1326

The lens scored very high at f/2 on a A7RIV. However, if you use the dropdown and pair to a different Sony body you immediately find that the same lens produces some very mundane results.
We know that is how their data works. I just don't believe they are actually testing every single lens + body combination that they list.

For example they probably have over 100 Canon EF lenses. Each of those has at results for at least 5-6 apertures, and for zooms there's another 3-4 of each of those measurements at different focal lengths. So each EF mount body supposedly has at least 1500 tests done. And there are about 20 or so EF bodies in their database, so now you are talking about 30,000 tests. And that doesn't even get into averaging out multiple samples or decentering, so with that you could be into the hundreds of thousands of tests, with over 10,000 tests done just to add a new body.

Naw, I am certain they are just getting MTFs of each component at each setting and multiplying them to get these results. Which is fine- but I'd rather have the raw data to either do my own analysis on or to view without their subjective filtering and summarization. They have a ton of great data that they have kind of rendered useless. What is a P-mpix? What is acutance? How do either actually translate to photographs? I still don't know.
While I get your point about the number of tests, there were examples in this thread about numbers being all over the place with different [but same mount] cameras
 
In an optical system resolution is multiplicative so every bit, be it from the lens or sensor, helps.
That statement is fundamentally wrong. The accuracy of every measurement chain (optical or other) is determined by the most inaccurate step in the chain. The only exception is when you are dealing with sampling noise and can average multiple samples.
MTF’s do multiply.
Thank you for the non sequitur.
Optical resolution - Wikipedia

b87680106f330cc62257f478d520d7e1977e478b
All those numbers are less than 100%. Every additional multiplication term reduces the value, it does not "help" it.
 
In an optical system resolution is multiplicative so every bit, be it from the lens or sensor, helps.
That statement is fundamentally wrong. The accuracy of every measurement chain (optical or other) is determined by the most inaccurate step in the chain. The only exception is when you are dealing with sampling noise and can average multiple samples.
MTF’s do multiply.
Thank you for the non sequitur.
Optical resolution - Wikipedia

b87680106f330cc62257f478d520d7e1977e478b
All those numbers are less than 100%. Every additional multiplication term reduces the value, it does not "help" it.
Every improvement helps - was the meaning of the original claim.
 
You mix "resolution" and "image quality" in your response. The two are different. My statement was about resolution. Image quality can improve even when resolution goes down. But your statement about resolution being multiplicative is false (for resolution and for every other measurement chain).
The term "image quality" is an inherently arbitrary and subjective term, so I'm not going to waste time debating why everyone has to adhere to your definition of it. I don't define "image quality" in the way the author does either.

In this paper "image quality" is basically defined as system resolution, so in the context of its findings the terms are essentially interchangeable. And that system resolution is defined as the multiplication of the resolution MTFs of all the elements in the image chain (sensor * lens). If you have an objectively/empirically better definition of resolution that refutes what seems to be inherent to optics I'd love to see it.......
There is no such thing as "resolution MFT". MFT is the product of resolution and contrast. Here is a good primer. One quote (boldface is mine):

"Both resolution and contrast are essential for an image to appear sharp. In simple terms, resolution is the ability to transfer details, while contrast is the ability to distinguish between light and dark areas. Even if resolution is high, a low contrast will inhibit your ability to clearly see sample details. MTF measures a lens’ ability to transfer the contrast of a sample to an image using spatial frequency (resolution)."
 
In an optical system resolution is multiplicative so every bit, be it from the lens or sensor, helps.
That statement is fundamentally wrong. The accuracy of every measurement chain (optical or other) is determined by the most inaccurate step in the chain. The only exception is when you are dealing with sampling noise and can average multiple samples.
MTF’s do multiply.
Thank you for the non sequitur.
Optical resolution - Wikipedia

b87680106f330cc62257f478d520d7e1977e478b
All those numbers are less than 100%. Every additional multiplication term reduces the value, it does not "help" it.
Yes, but improving any of the terms in the chain increases the end result. So even an average lens will benefit from a better sensor and vice versa.

--
Sometimes I take pictures with my gear- https://www.flickr.com/photos/41601371@N00/
 
You mix "resolution" and "image quality" in your response. The two are different. My statement was about resolution. Image quality can improve even when resolution goes down. But your statement about resolution being multiplicative is false (for resolution and for every other measurement chain).
The term "image quality" is an inherently arbitrary and subjective term, so I'm not going to waste time debating why everyone has to adhere to your definition of it. I don't define "image quality" in the way the author does either.

In this paper "image quality" is basically defined as system resolution, so in the context of its findings the terms are essentially interchangeable. And that system resolution is defined as the multiplication of the resolution MTFs of all the elements in the image chain (sensor * lens). If you have an objectively/empirically better definition of resolution that refutes what seems to be inherent to optics I'd love to see it.......
There is no such thing as "resolution MFT". MFT is the product of resolution and contrast. Here is a good primer. One quote (boldface is mine):

"Both resolution and contrast are essential for an image to appear sharp. In simple terms, resolution is the ability to transfer details, while contrast is the ability to distinguish between light and dark areas. Even if resolution is high, a low contrast will inhibit your ability to clearly see sample details. MTF measures a lens’ ability to transfer the contrast of a sample to an image using spatial frequency (resolution)."
You are correct. My point was the "image quality" was used to describe MTF in the paper.
 
All those numbers are less than 100%. Every additional multiplication term reduces the value, it does not "help" it.
Yes, but improving any of the terms in the chain increases the end result. So even an average lens will benefit from a better sensor and vice versa.
[/QUOTE]
If "better" means a sensor with higher MFT, I agree with you. But just increasing the sensor resolution does not guarantee that. That was the point I was trying to make.
 
This is a link to their test protocol:

https://www.dxomark.com/dxomark-lens-camera-sensor-testing-protocol/

A key quote extracted from that article:

”The score shows the amount of information captured by the lens on a given camera and how well the camera and lens perform together. However, the score does not reflect the intrinsic quality of the camera sensor.”

In other words this is a test of a specific lens on a specific camera model. It is not a measure of that same lens on a different camera model even within the same brand. You may have radically different results. Neither do the scores compare say a 400mm Sony with a 400mm Canon on different bodies. Also carefully note the weighting used for the overall score, making it essentially useless for many lens settings.

A couple years back a blog cited the Yongnuo 85 mm as a hidden gem, one of the top scoring lenses on DXO with a score of 51 demonstrating how one can be misled by misunderstanding the testing.

https://petapixel.com/2022/08/03/hidden-gem-a-yongnuo-lens-is-ranked-in-the-top-3-best-on-dxomark/

https://www.dxomark.com/Lenses/Yongnuo/Yongnuo-YN-85mm-F18S-DF-DSM-mounted-on-Sony-A7R-IV__1326

The lens scored very high at f/2 on a A7RIV. However, if you use the dropdown and pair to a different Sony body you immediately find that the same lens produces some very mundane results.
We know that is how their data works. I just don't believe they are actually testing every single lens + body combination that they list.

For example they probably have over 100 Canon EF lenses. Each of those has at results for at least 5-6 apertures, and for zooms there's another 3-4 of each of those measurements at different focal lengths. So each EF mount body supposedly has at least 1500 tests done. And there are about 20 or so EF bodies in their database, so now you are talking about 30,000 tests. And that doesn't even get into averaging out multiple samples or decentering, so with that you could be into the hundreds of thousands of tests, with over 10,000 tests done just to add a new body.

Naw, I am certain they are just getting MTFs of each component at each setting and multiplying them to get these results. Which is fine- but I'd rather have the raw data to either do my own analysis on or to view without their subjective filtering and summarization. They have a ton of great data that they have kind of rendered useless. What is a P-mpix? What is acutance? How do either actually translate to photographs? I still don't know.
While I get your point about the number of tests, there were examples in this thread about numbers being all over the place with different [but same mount] cameras
That might have been me. Panasonic lenses being tested on Olympus bodies yielded different results than Panasonic lenses being tested on Panasonic bodies. Taking that further, different Panasonic bodies yielded different results from each other. Too many variables and the only constant was the lens?
 
In an optical system resolution is multiplicative so every bit, be it from the lens or sensor, helps.
That statement is fundamentally wrong. The accuracy of every measurement chain (optical or other) is determined by the most inaccurate step in the chain. The only exception is when you are dealing with sampling noise and can average multiple samples.
MTF’s do multiply.
Thank you for the non sequitur.
Optical resolution - Wikipedia

b87680106f330cc62257f478d520d7e1977e478b
All those numbers are less than 100%. Every additional multiplication term reduces the value, it does not "help" it.
That depends. The 'MTF's there are functions, not numbers. The multiplication is function multiplication. Whether the multiplication 'reduces' or 'helps' the outcome depends on the nature of the function. However, if we take the position that a perfect system would be diffraction limited', that is have the diffraction MTF only, then each additional function there puts us in a worse position than that perfect lens. The point being mate is that MTFsensor() fro a high pixel count sensor puts us in a 'less worse' position than MTFsensor() for a low pixel count sensor.

--
Is it always wrong
for one to have the hots for
Comrade Kim Yo Jong?
 
If "better" means a sensor with higher MFT, I agree with you. But just increasing the sensor resolution does not guarantee that. That was the point I was trying to make.
It's a major contributor, the other being pixel aperture. See Jack Hogan's article . But, if we assume 100% microlenses, the a higher MP produces a 'better' MTF.
 
You mix "resolution" and "image quality" in your response. The two are different. My statement was about resolution. Image quality can improve even when resolution goes down. But your statement about resolution being multiplicative is false (for resolution and for every other measurement chain).
The term "image quality" is an inherently arbitrary and subjective term, so I'm not going to waste time debating why everyone has to adhere to your definition of it. I don't define "image quality" in the way the author does either.

In this paper "image quality" is basically defined as system resolution, so in the context of its findings the terms are essentially interchangeable. And that system resolution is defined as the multiplication of the resolution MTFs of all the elements in the image chain (sensor * lens). If you have an objectively/empirically better definition of resolution that refutes what seems to be inherent to optics I'd love to see it.......
There is no such thing as "resolution MFT". MFT is the product of resolution and contrast. Here is a good primer. One quote (boldface is mine):

"Both resolution and contrast are essential for an image to appear sharp. In simple terms, resolution is the ability to transfer details, while contrast is the ability to distinguish between light and dark areas. Even if resolution is high, a low contrast will inhibit your ability to clearly see sample details. MTF measures a lens’ ability to transfer the contrast of a sample to an image using spatial frequency (resolution)."
I don’t think what Olympus have written here is helpful in the context of this thread. It’s meant as helpful marketing stuff. I note the quote’ doesn’t make any attempt to quantify the term resolution or say how it could be measured.



in the end it comes down to what you mean by resolution. 40 years ago the resolution of a lens happily judged at the spacing of lines that could just be resolved. Contrast was described separately.

But contrast and resolution aren’t seperate things. So the modern approach is to use modular transfer functions.

The old measure of resolution was also a form of MTF. Just a particularly meaningless one. Firstly it was very low contrary ratio, far to low to tell us much meaningful about the images a lens will produce. Secondly the measurements were made with hard edge lines which aren’t a single spatial frequency.

MTF 50 isn’t perfect but it certainly tells us more than the old resolution measurements
 
All those numbers are less than 100%. Every additional multiplication term reduces the value, it does not "help" it.
Yes, but improving any of the terms in the chain increases the end result. So even an average lens will benefit from a better sensor and vice versa.
If "better" means a sensor with higher MFT, I agree with you. But just increasing the sensor resolution does not guarantee that. That was the point I was trying to make.
Give me a theoretical example of increasing a sensor's MP not increasing system MTF. Practically speaking it doesn't jive- for example an AA filter hurts MTF, but camera makers have been weakening them or getting rid of them altogether with increases in MPs. Anecdotally I have seen increases in detail and sharpness with mediocre glass on increasingly high MP sensors.

--
Sometimes I take pictures with my gear- https://www.flickr.com/photos/41601371@N00/
 
There is no such thing as "resolution MFT". MFT is the product of resolution and contrast.
No, in layman's terms, it is the contrast at a given resolution.
JACS, I acknowledge that you are a layman. Mathematically, "at a" means "multiplied by" (actually, convolved with, but never mind).
 
Last edited:

Keyboard shortcuts

Back
Top