Sony FE lenses sample variation summary

[JohnNEX]

A lot of threads mention sample variation with the Sony FE lenses. I thought I would gather together all of the links to the excellent tests done by Lensrentals which cover sample variation.

The summary is simple:

1. Some of the Sony FE lenses released first, had bad sample variation. The main culprits were the 35 f/1.4 ZA and the 90 f/2.8 Macro. The 55 f/1.8 ZA was ok

2. Since then no Sony FE lens tested by Lensrentals has shown any more sample variation than similar lenses from other manufacturers.

All of the ‘reputation’ of bad sample variation in FE lenses seems to be based on a couple of the early cases. Lensrentals has actually been very positive about sample variation for Sony lenses.

Below are links to all of the tests, with direct quotes on sample variation which make instructive reading. I’ve included the “bad” lenses first, but the rest are in the random order of how I looked them up. I could not locate the tests for the 35 f/2.8, the 70-200 f/4 or the 70-200 f/2.8. If someone can find them then please provide the link and quotes.

I have not covered the APS-C lenses, which have their own history of sample variation, particularly for the 16-70 f/4 Zeiss.

35 f/1.4 ZA

https://www.lensrentals.com/blog/2015/10/sony-e-mount-lens-sharpness-bench-tests/

The variation graphs show what our photographic experience with this lens has suggested. The FE 35mm f/1.4 ZA lenses are all over the place. It actually is a bit worse than the graphs look because a lot of the variance is WITHIN a copy, not just copy-to-copy. None of the 10 copies we tested had even corners. And I’ll editorialize and say that none of the dozens we’ve tested on Imatest had even corners either. If you use this lens for centered objects, you’ll be happy. If you want 4 sharp corners, it’s not likely to happen unless your standards for equal sharpness are pretty low.

90 f/2.8 Macro

https://www.lensrentals.com/blog/2015/10/sony-e-mount-lens-sharpness-bench-tests/

Sony has a lot of variation compared to the others. The type of variation we’re seeing is also quite different from the Sony FE 35mm f/1.4. That lens has a lot of corner variation in every copy, but the center remains is consistently sharp comparing one lens to another. With the Macro, there’s a lot of center variation and it’s not particularly worse in the corners. This suggests there is significant copy-to-copy variation in overall sharpness, rather than individual lenses have a bad corner.

55 f/1.8 ZA

https://www.lensrentals.com/blog/2015/10/sony-e-mount-lens-sharpness-bench-tests/

The copy-to-copy variation graphs for the 55mm lens show it has a lot better consistency than the 35mm, too. There’s some variation, but it’s similar to most of the other 50ish lenses we’ve tested.

50 f/1.4 ZA

https://www.lensrentals.com/blog/2016/07/sony-fe-planar-t-50mm-f1-4-za-mtf-and-variance-testing/

The new 50mm f/1.4 ZA Planar clearly has less variation than the 55mm f/1.8 did. It’s really nice and consistent … Overall, the Sony FE 50mm f/1.4 Planar has very reasonable sample variation, about what we see in most other high-quality 50mm prime lenses from other manufacturers … On the basis of these [sample variation charts], the Sony 50mm f/1.4 Planar is as good as anything available.

24-105 f/4 G

https://www.lensrentals.com/blog/2018/02/mtf-tests-of-the-sony-fe-24-105mm-f4-oss/

The Canon 24-105mm f/4 IS II has been the standard for consistency in this range zoom, but the Sony is a bit better (as was the Sigma we tested last week). So, let me repeat, because in the past I’ve been pretty harsh about Sony sample variation. While 24-105mm zooms tend to have a high variation, the Sony does not; it’s very consistent in copy-to-copy.

24-70 f/2.8 GM

https://www.lensrentals.com/blog/20...s-the-24-70mm-f2-8-gm-mtf-and-variance-tests/

Like the Canon and Nikon, the Sony did have more variation at 70mm. But the Sony doesn’t worsen more than the others, they all end up fairly similar. As an aside, we also checked variance in the middle of the zoom range and the Sony was really quite good there. In other words, if your copy has problems, it will almost certainly be at 70mm, but if you buy a Sony 24-70mm f/2.8 you are no more likely to have an issue than you would with a Canon or Nikon.

16-35 f/2.8 GM

https://www.lensrentals.com/blog/2017/08/sony-fe-16-35mm-f2-8-gm-sharpness-tests/

I’m not posting the 16mm and 24mm variations trying to keep this reasonably brief, but at those two focal lengths there wasn’t much variation. We had one copy that was truly bad out of 11 tested at 24mm, but that copy was excellent at 16mm and 35mm. All of the copies we tested were very similar at 16mm ... But if you plan on using it at 35mm, which I generally don’t recommend for wide zooms if you have alternatives, well, you’re probably going to go through several copies before you find the one you want. Personally, I think it’s a really good 16-24mm f/2.8 zoom that can be used at 35mm if you have to.

16-35 f/4

https://www.lensrentals.com/blog/2016/06/sony-fe-16-35-f4-sony-fe-24-70mm-f4-zoom-mtf-and-variance/

We know from previous testing that variation for the Canon and Nikon 16-35 f/4s are pretty good as zoom lenses go; not great, but good. The Sony 16-35 f/4 OSS has very similar to a bit less variation at the wide end and very similar to a bit worse variation on the long end. Overall, both from an MTF and a copy-to-copy variation standpoint, the Sony FE lens is as good as, and sometimes better than, the Canon and Nikon offerings.

24-70 f/4 Zeiss

https://www.lensrentals.com/blog/2016/06/sony-fe-16-35-f4-sony-fe-24-70mm-f4-zoom-mtf-and-variance/

It really is pretty good [sample variation] at 24mm and 50mm, but there is a bit of worrisome center variation at 70mm. (Center variation is associated with an overall sharpness difference, where off-axis variance often indicates more of a tilt.) For a zoom, though, this isn’t a bad performance … So the bottom line is while there is definitely some copy-to-copy variation among the Sony f/4 zooms, it doesn’t really appear worse than most zoom lenses.

12-24 f/4 G

https://www.lensrentals.com/blog/2017/06/sony-fe-12-24-f4-g-mtf-tests/

[Sample variation is] better than average. Sony seems to be making progress with their sample variation in the newer designs … Especially at 12mm, there is almost no variation in the center. As with most wide-angle zooms, if your copy isn’t quite like the others it will probably be at the long end, but this is really a good performance.

50 f/1.8

https://www.lensrentals.com/blog/2016/05/sony-fe-50mm-f1-8-mtf-and-consistency/

The copy-to-copy variation on the FE 50mm f/1.8 is superbly low. Remember, the Canon 50mm f/1.8 STM is the standard against which all other lenses are measured. It’s amazingly consistent. The Sony is very close to that. In some ways (center sharpness variation) it’s actually a bit better. So hat’s off to Sony, they’ve made an FE lens that from a copy-to-copy variation standpoint is as good as anything on the market.

85 f/1.8

https://www.lensrentals.com/blog/2017/06/sharpness-tests-of-the-sony-fe-85mm-f1-8-g/

[Sample variation] was always a question with early Sony primes, but the range here is good, one of the more consistent Sony lenses.

85 f/1.4 GM

https://www.lensrentals.com/blog/2016/04/sony-fe-85mm-f1-4-g-master-lens-mtf-and-variance/

The Sony FE 85mm f/1.4 G Master lens compares very well with these other top lenses. There’s none of the severe copy-to-copy variation we saw with the Sony FE 35mm f/1.4 lens.

--
https://www.facebook.com/John-Clark-Photography-1035965476487072/

 

[C47]

Thank you for this summary!

 

[JohnNEX]

OK, I found the 70-200 f/4 - it was in a discussion of other similar lenses from Canon and Nikon: Also found the Batis 18mm and 135mm, but the commentary is sparse.

70-200 f/4 G

https://www.lensrentals.com/blog/2016/03/just-the-lenses-the-70-200mm-f4-comparison/

The variation for all these lenses is pretty reasonable – we expect zooms to vary more than primes and to have different amounts of variation at different focal lengths. The Nikon has a bit less variation than the Canon and the Sony at 70mm and 135mm, particularly in center sharpness. At 200mm, there is really no difference between the three. I don’t want to make any major generalizations here. I think it’s clear that the Sony lens is very similar to the Canon and Nikon as far as sample variation.

18 f/2.8 Batis

Loading…

www.lensrentals.com

The 10-copy variation curves are also very good for a wide angle lens ... a nice, tight pattern.

135 f/2.8 Batis

Loading…

www.lensrentals.com

The variance is also good, although we expect that with a 135mm lens.

 

[IamJF]

Wow - that's a great summary!

Thanks for all the work!

 

[DenImage]

Nice summary, thank you

Den

 

[fjbyrne]

[No message]

 

[Jeff2013]

Great summary!

I think that the copy variation of the GM 70-200mm f2.8 might beat the 35mm f1.4. When Roger Cicala tested this lens, it was so bad that he didn’t publish the results. My copy of the lens was quite soft at f2.8 compared to my Canon 70-200mm f2.8 II. Many months later, I purchased another GM lens and it is excellent... either on par or sharper than the Canon lens.

After four copies, I gave up on the FE 35mm f1.4.

 

[ZodiacPhoto]

Thanks for the great summary!

I had an opportunity to choose the 70-200mm from 3 samples. While I did select one lens, the difference between the 3 was minimal. I would be happy with any of them.

I also find my Batis 18, 24-105 and 90 macro sufficiently sharp for me on a 42Mp sensor.


http://www.zodiacphoto.com

 

[Rol Lei Nut]

A lot of threads mention sample variation with the Sony FE lenses. I thought I would gather together all of the links to the excellent tests done by Lensrentals which cover sample variation.

Are those summaries made before or after any badly decentered, etc., lenses were returned?

That is, did Lensrentals test lenses which they are actually renting, meaning any obviously sub-par lenses were previously weeded out?

 

[JohnNEX]

A lot of threads mention sample variation with the Sony FE lenses. I thought I would gather together all of the links to the excellent tests done by Lensrentals which cover sample variation.

Are those summaries made before or after any badly decentered, etc., lenses were returned?

That is, did Lensrentals test lenses which they are actually renting, meaning any obviously sub-par lenses were previously weeded out?

Good question. I don't know the answer but it is a question that has probably been asked before in the comments on the LensRentals blog - it would just take a lot of effort to trawl through it all.

My impression is that the answer is most likely "no", since: (a) Roger does mention some very bad copies in his testing which you would have thought would have been weeded out already; (b) its hard to see how that pre-testing would have been done - I think what we are seeing in the tests actually is Roger's checks to find bad samples; (c) I would have thought Roger would have mentioned any pre-weeding somewhere in the text and I have read a lot of his stuff and he seems like a pretty straight shooter. I don't recall him saying anything like "we sent back seven copies before we got usable ones."

I expect that they weed out lenses which very obviously don't work - e.g. won't focus at all or the zoom ring won't turn. But not the decentred lenses or those with unusually weak corners.

But that is just musings - I don't have the answer.

 

[RCicala]

A lot of threads mention sample variation with the Sony FE lenses. I thought I would gather together all of the links to the excellent tests done by Lensrentals which cover sample variation.

Are those summaries made before or after any badly decentered, etc., lenses were returned?

That is, did Lensrentals test lenses which they are actually renting, meaning any obviously sub-par lenses were previously weeded out?

Our protocol has always been as follows: all lenses are unpacked, checked (do they work properly) and a quick test chart shot made to show they are apparently decent. Depending on the lens in question (and please note I said lens, not brand of lens) 0.5% and 2% of lenses are sent straight back. The are obvious failures, and generally that's a gross optical failure (one test chart image shows the lens is massively soft or one side quite different than the other).

Then rest are sent over for true optical testing and that's what my statistics are taken from. This is a very old graph, from back when we used Imatest for testing, but it illustrates the point nicely. The failed lens is so different it's easily apparent. Variance is measured among the other lenses.

 

[TN Args]

A lot of threads mention sample variation with the Sony FE lenses. I thought I would gather together all of the links to the excellent tests done by Lensrentals which cover sample variation.

Are those summaries made before or after any badly decentered, etc., lenses were returned?

That is, did Lensrentals test lenses which they are actually renting, meaning any obviously sub-par lenses were previously weeded out?

Our protocol has always been as follows: all lenses are unpacked, checked (do they work properly) and a quick test chart shot made to show they are apparently decent. Depending on the lens in question (and please note I said lens, not brand of lens) 0.5% and 2% of lenses are sent straight back. The are obvious failures, and generally that's a gross optical failure (one test chart image shows the lens is massively soft or one side quite different than the other).

Then rest are sent over for true optical testing and that's what my statistics are taken from. This is a very old graph, from back when we used Imatest for testing, but it illustrates the point nicely. The failed lens is so different it's easily apparent. Variance is measured among the other lenses.

Excellent. And worth pointing out that, because the practice is common to all brands, the resutls remain valid. I'm sure you would point it out if the failure rate was strongly biased by brand.

 

[DiffractionLtd]

The only downside is that no camera lens is optimized to perform at infinity, where the test unit is. He even makes mention of it, but only in regard to macro lenses. How it impacts the lens test I don't know.

 

[JohnNEX]

A lot of threads mention sample variation with the Sony FE lenses. I thought I would gather together all of the links to the excellent tests done by Lensrentals which cover sample variation.

Are those summaries made before or after any badly decentered, etc., lenses were returned?

That is, did Lensrentals test lenses which they are actually renting, meaning any obviously sub-par lenses were previously weeded out?

Our protocol has always been as follows: all lenses are unpacked, checked (do they work properly) and a quick test chart shot made to show they are apparently decent. Depending on the lens in question (and please note I said lens, not brand of lens) 0.5% and 2% of lenses are sent straight back. The are obvious failures, and generally that's a gross optical failure (one test chart image shows the lens is massively soft or one side quite different than the other).

Then rest are sent over for true optical testing and that's what my statistics are taken from. This is a very old graph, from back when we used Imatest for testing, but it illustrates the point nicely. The failed lens is so different it's easily apparent. Variance is measured among the other lenses.

Thanks for jumping in Roger! Great to get an answer from you directly.

--

Log into Facebook

Log into Facebook to start sharing and connecting with your friends, family, and people you know.

www.facebook.com

 

[JimKasson]

The only downside is that no camera lens is optimized to perform at infinity

That's a very strong assertion. I don't know that that statement is true. How do you know that it is?

Jim

--
http://blog.kasson.com

 

[TRIODEROB]

A lot of threads mention sample variation with the Sony FE lenses. I thought I would gather together all of the links to the excellent tests done by Lensrentals which cover sample variation.

Are those summaries made before or after any badly decentered, etc., lenses were returned?

That is, did Lensrentals test lenses which they are actually renting, meaning any obviously sub-par lenses were previously weeded out?

Our protocol has always been as follows: all lenses are unpacked, checked (do they work properly) and a quick test chart shot made to show they are apparently decent. Depending on the lens in question (and please note I said lens, not brand of lens) 0.5% and 2% of lenses are sent straight back. The are obvious failures, and generally that's a gross optical failure (one test chart image shows the lens is massively soft or one side quite different than the other).

Then rest are sent over for true optical testing and that's what my statistics are taken from. This is a very old graph, from back when we used Imatest for testing, but it illustrates the point nicely. The failed lens is so different it's easily apparent. Variance is measured among the other lenses.

will you test someone's lens if they send it to you - pay postal and a test fee ?

 

[RCicala]

A lot of threads mention sample variation with the Sony FE lenses. I thought I would gather together all of the links to the excellent tests done by Lensrentals which cover sample variation.

Are those summaries made before or after any badly decentered, etc., lenses were returned?

That is, did Lensrentals test lenses which they are actually renting, meaning any obviously sub-par lenses were previously weeded out?

Our protocol has always been as follows: all lenses are unpacked, checked (do they work properly) and a quick test chart shot made to show they are apparently decent. Depending on the lens in question (and please note I said lens, not brand of lens) 0.5% and 2% of lenses are sent straight back. The are obvious failures, and generally that's a gross optical failure (one test chart image shows the lens is massively soft or one side quite different than the other).

Then rest are sent over for true optical testing and that's what my statistics are taken from. This is a very old graph, from back when we used Imatest for testing, but it illustrates the point nicely. The failed lens is so different it's easily apparent. Variance is measured among the other lenses.

will you test someone's lens if they send it to you - pay postal and a test fee ?

I don't have the set up to do a high volume of lenses at this time. I do accept an occasional lens for testing when schedule allows - if I have existing standards for that lens. That being said, if someone wants me to test a lens where I know the variance is high, I won't. When 90% of a given lens have a soft corner, I'm not testing a copy for soft corners.

 

[RCicala]

A lot of threads mention sample variation with the Sony FE lenses. I thought I would gather together all of the links to the excellent tests done by Lensrentals which cover sample variation.

Are those summaries made before or after any badly decentered, etc., lenses were returned?

That is, did Lensrentals test lenses which they are actually renting, meaning any obviously sub-par lenses were previously weeded out?

Our protocol has always been as follows: all lenses are unpacked, checked (do they work properly) and a quick test chart shot made to show they are apparently decent. Depending on the lens in question (and please note I said lens, not brand of lens) 0.5% and 2% of lenses are sent straight back. The are obvious failures, and generally that's a gross optical failure (one test chart image shows the lens is massively soft or one side quite different than the other).

Then rest are sent over for true optical testing and that's what my statistics are taken from. This is a very old graph, from back when we used Imatest for testing, but it illustrates the point nicely. The failed lens is so different it's easily apparent. Variance is measured among the other lenses.

Excellent. And worth pointing out that, because the practice is common to all brands, the resutls remain valid. I'm sure you would point it out if the failure rate was strongly biased by brand.

I definitely would, and do, but that tends to come later. When I'm first testing a lens, we have maybe 20 copies. If 0, 1 or even 2 are bad, that's probably a meaningless difference from small sample size. Months later, when we've had 150 copies I start to notice differences.

 

[Rol Lei Nut]

The only downside is that no camera lens is optimized to perform at infinity, where the test unit is. He even makes mention of it, but only in regard to macro lenses. How it impacts the lens test I don't know.

The Leica R Apo-Telyt 180 3.4 is the first lens which comes to mind (optimized for infinity).

 

[Jerry R]

John,

What impact will these variation have on real world photography?

Thanks,

Jerry