Reading resolution charts comparing MFT lenses to FX lenses

Started Mar 29, 2013 | Questions thread
Great Bustard Forum Pro • Posts: 39,900

Anders W wrote:

Great Bustard wrote:

Prairie Pal wrote:

I'm at photozone review website browsing through lenses for both FX and MFT.  I see that the highest resolving FX lenses can be near the 4000 lw/ph range.  The better MFT lenses rarely even get to 2300.  Is the difference between the 2 types of lenses relative?  Surely even the best MFT glass isn't inferior to FX.  I admit I can't technically explain what lw/ph is measureing, but to a certain extent I look at those graphs as a means of measuring one lens against another ie, when comparing the Sigma 35 1.4 to Nikon 35, or the various MFT primes between each other.

...and Anders will hopefully chime in so you can watch us argue about it. 

First of all, what PZ measures is the MTF-50, which is low many line widths per picture height (lw/ph) are resolved at 50% contrast.

Now, Klaus clearly states:

Please note that the tests results are not comparable across the different systems!

but he does not go into detail about why, so let's discuss that point for the rest of this post.

First of all, the resolution in the final photo, which is what PZ measures, is a function of many different factors:

  • Lens sharpness
  • Pixel count
  • Sensor size
  • AA filter
  • RAW conversion
  • Additional processing

In other words, you aren't looking at lens tests, you are looking at system tests (this is true for all other testing sites as well).

I discuss the relationship between lens sharpness and sensor size here, and the relationship between pixel count and resolution here, so I'll move on to other factors in this post.

As we all know, a weaker AA filter will result in what appears to be more detailed files, but much of this additional "detail" may well be false detail.  Depending on the processing, this false detail can look very pleasing and also result in higher MTF-50 resolution scores.  Indeed, it's not unlike the idea behind upsampling a photo for more "resolution".  That is, you cannot get more resolution than the initial capture, but with good enough software, you can make enough "lucky guesses" that the distinction between real and made up is inconsequential.  Of course, there are limits, however.

In addition, we all know that different demosaicking algorithms can have a marked impact on the look of a photo, and that includes resolution.  Thus, using different RAW converters for different files, or even the same RAW converter that treats the files from different systems differently, and not necessarily differently in an optimum way for each system, can cause further problems still.

Lastly, and this is where Anders and I bump heads, is on the matter of sharpening.  We normally don't think of noise at base ISO in good light to be an issue.  However, the noise differential exists even if it's not visible.

I think we actually agree on most of the facts involved but differ in the conclusions we draw from them. I think that as far as tests are concerned, it is useful to keep MTF performance apart from noise performance as much as possible. There are multiple reasons for this.

First, the way Photozone does it means that we mix the two in unknown ways. There is no way for me as a potential user of Photozone data to know the mix they have chosen for a particular camera/sensor and how that mix compares to others. This means that the results lack transparency.

Second, as I am sure you agree, no mix is appropriate for all conditions. As you point out below, there are cases where a D800 beats an E-M5 for noise but also situations where it is the other way around.

Third, it makes a difference to me whether a certain MTF result has been achieved with or without sharpening. Sharpening in PP is to a considerable extent a way to fool the eye and based on the visual evidence I have looked at, I'll take the unsharpened output from a good lens over the sharpened output from an inferior one any day, even though they might both reach the same MTF values at the same noise level.

My preferred solution to these problems is to present the results of MTF tests in such a way that an informed reader can draw his/her own conclusions with all relevant information on the table. At the present time, the two sources of information that I find preferable in this regard are Lenstip and LensRentals. Both present results based on RAWs converted by the same known algoritms without any additional sharpening. The information from LensRentals is presented in a form that simplifies cross-format comparisons since it is reported as line pairs per image height (lp/ih). The results from Lenstip are reported as line pairs per mm (lp/mm) and therefore have to be converted before they can be compared. But this is not difficult to do. To compare between FF and MFT, for example, you merely have to multiply the FF results by two or, alternatively, divide the MFT results by two.

You might ask, if it's not visible, then what does it matter?  Well, the reason it matters is that the lower noise photo can take more sharpening.  Thus, by comparing photos from, say, a D800 and EM5, with no sharpening applied, well, that artificially handicaps the D800 file which has lower noise and can take more sharpening than the EM5 file (for a given exposure).

Of course, then we have equivalent photos which are photos at the same DOF and shutter speed.  For these types of photos, no format is favored over the other -- the noise differential is due to differences in sensor efficiency, and that difference can vary throughout the ISO range.  For example, the D800 may be a stop more efficient than the EM5 sensor at base ISO, but a stop less efficient at ISO 3200 (just throwing numbers out there -- I haven't actually worked that particular comparison out).

So, in the end, what we have to ask is how much variation all these factors cause.  The answer is:  I don't have a freakin' clue.  But, to compound the issue further still, there's the issue of copy variation among lenses, for example, this article, or this article. or this article.

Lastly, we have to ask how the MTF-50 lw/ph measurements correspond to the visual properties of the final photo.  Can you notice a 50% difference?  One would think so, but at what display size would you not notice?  How about 40%?  30%?  20%?  10%?

Fact of the matter is that anyone telling you f/11 is a perfectly good aperture to shoot at is already losing massive amounts of resolution due to diffraction softening (the sharper the lens, or the greater the pixel count, the more resolution they are losing, although they'll always be left with more resolution than if they started off with a softer lens or a sensor with fewer pixels), so MTF-50 scores, and thus resolution, probably aren't much of a concern to them at all, as they have more than enough, anyway.

So, does that at least help in answering your question, or just confuse the hell out of you? 

...what we want is not numbers at all, but photos of various scenes with various systems (body + lens), converted with various RAW converters as well as OOC jpgs, and taken at various light levels.  Then, as photographers, we could just look at the resulting photos, and decide from there.

However, we all know this is impractical to do times a thousand.  So, what to do?

Well, it would be nice if we knew what effect various RAW converters had on the MTF-50 scores.  If all RAW converters resulted in MTF-50s within 10%, I'd say it's not a big deal, and just go with the converter than best matches all systems as a whole.  If not, then use the RAW converter that gives the "best" result for each system, but now we have to define "best", as "best" doesn't only mean detail.

As for sharpening and noise, well, I don't think we're ever going to convince each other.  As you know, my emphasis is always the visual properties of the final photo, and hence, sharpening and noise always go hand-in-hand.

However, here's what would be awesome:  take the make a bench that mounts the lens in front of the highest pixel density sensor out there.  Take some photos from various portions of the image circle, and use the same RAW converter with the exact same settings each time.  Now you're really testing the lens, and not the system.

Of course, this now opens a huge can o' worms, since the lenses for smaller sensor systems will, on average, resolve better than lenses for larger sensor systems, and we'll have the ignorant masses chanting "lw/ph = lw/ph" to go along with "f/2 = f/2", and any attempts to educate will be met with "entertainment".

But, now that I think about it, I'm good with that. 

Post (hide subjects) Posted by
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark MMy threads
Color scheme? Blue / Yellow