The utility of lens (system) tests.

Started 8 months ago | Discussions thread
ForumParentFirstPreviousNextNext unread
Flat view
Great Bustard
Forum ProPosts: 22,610
The utility of lens (system) tests.
8 months ago

The purpose of this thread is to discuss the utility of the results of lens tests, specifically, MTF-50 scores and DxOMark's PMP measure.

Ideally, the results of the lens tests should reflect differences we see in the displayed photo, keeping in mind that the display dimensions, viewing distance, and visual acuity of the viewer may all put an upper limit on meaningful resolution. For example, there is likely zero difference between MTF-50 scores of 3000 lw/ph and 2000 lw/ph, or PMP scores of 20 MP and 12 MP, for 4x6 inch print viewed at arm's length with 20-40 vision.

Whether or not whatever differences in resolution there are have any impact on the "success" of the photo is, of course, another matter all together.

OK, let's get to it!

The first issue is the processing of the photo. We're all aware that different RAW converters, as well as the cameras' jpg engines, can result in very different results. Thus, if we are comparing processed photos, how can we be reasonably sure that the photos for a resolution test were optimally processed, and what does "optimally processed" even mean? On the other hand, if we are comparing unprocessed photos, as DxOMark does, where is the utility in this as we never view unprocessed photos? Do measurements of unprocessed photos accurately reflect the differences we might expect with "optimally processed" photos?

Also, resolution tests are done with monochrome test scenes. Is this a useful method of measuring resolution? Might it not be better to measure resolution in each of the three color channels? That is, an MTF-50 or PMP score for red, green, and blue? Or is monochrome "close enough", in the same way that MTF-50 is "close enough" that we need not also supplement the MTF-50 measurement with MTF-10 and MTF-90 scores, for example? Or is the MTF-50 score, like the PMP measurement, by itself, also severely lacking, as a single measure for resolution is simply so simplistic as to not really be useful in terms of the resolution differences in actual photos we can expect (under ideal conditions) from the different systems we are comparing?

Lastly, what do y'all think of DPR's new test scene as a target by which to display (as opposed to measure) resolution? Would not a series of photos throughout the aperture range of the lens at a fixed shutter speed and various light levels (since noise affects resolution) be a good indicator of what you could expect from the particular system that was tested?

Or is this just way, way, way too much work (e.g. testing a lens on different bodies with a few different "standard candle" RAW converters, throughout the aperture range and light levels) just be too much work when single measures that are already performed on processed or unprocessed files (i.e. MTF-50 and PMP) are sufficient, and that variations between different copies of the lens far outweigh variations between processing and other resolution measures (e.g. MTF-10 and MTF-90)?

Lastly, I just want to say that, like any other measure of IQ, the importance of resolution in terms of the "success" of a photo depends on a great number of factors, from not making any difference whatsoever to a critical component. That said, resolution and noise are typically the two most important measures in the IQ of a photo (keeping in mind that image quality is, at best, merely a component of a "successful" photo), so I was thinking a discussion on how resolution is measured, and how that measure corresponds to the visual properties of the resulting photo, would be worthwhile.

ForumParentFirstPreviousNextNext unread
Flat view
Post (hide subjects)Posted by
ForumParentFirstPreviousNextNext unread
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark post MMy threads
Color scheme? Blue / Yellow