Lens variations

Started 3 months ago | Discussions
Joe Moche Regular Member • Posts: 188
Lens variations
2

A common theme on many of these forums involves folks who argue over the sharpness of a particular lens.  The argument often ends with "You got a bad sample.  Send it back."  Honestly, there seem to be an awful lot of "bad samples" floating around out there.  Is the lens manufacturing process really so loose that a high number of bad samples would be produced?  Are bad samples just an aberration, or is there a range of sharpness (from great to poor) that affects all lenses?  Do lens manufacturers test a lens before boxing it up, and is there some minimum level of performance that renders the lens "good enough?"  Finally, how can one know if a newly-purchased lens is a bad sample, other than by making a totally subjective assessment by comparing results to results claimed by others?

I have neither the time, skill, or desire to run lab tests on every lens I buy, and if a lens doesn't meet my expectations, I have no problem sending it back.  That said, I see no reason to settle for "bad copy" when a better copy can be had for the same price.  Thoughts?

-- hide signature --

Joe

 Joe Moche's gear list:Joe Moche's gear list
Canon EOS 5D Mark II Fujifilm GFX 50R Fujifilm GFX 100S +6 more
ipopov Regular Member • Posts: 227
Re: Lens variations
8

Honestly, there seem to be an awful lot of "bad samples" floating around out there. Is the lens manufacturing process really so loose that a high number of bad samples would be produced?

Judging number of bad samples from forums will definitely be overestimating. People who have unexpectedly bad results are much more likely to complain about it on a forum than people who are happy with their lenses.

Are bad samples just an aberration, or is there a range of sharpness (from great to poor) that affects all lenses?

There is definitely a range of performance. Something like a bell curve. Also, different optical designs have different sensitivity to manufacturing tolerances.

Do lens manufacturers test a lens before boxing it up, and is there some minimum level of performance that renders the lens "good enough?"

I don't think manufacturers share any info on that. I believe some high-end manufacturers may test all the copies (may be Leica, etc), but budget lenses are most likely only sample a few from a batch.

All the manufacturers have some performance thresholds there a copy passes as "within spec", but these thresholds are not public, and according to Roger Cicala, are very lax. Also, manufacturer's own testing equipment and methods may be not quite state of the art.

Finally, how can one know if a newly-purchased lens is a bad sample, other than by making a totally subjective assessment by comparing results to results claimed by others?

You can test for some obvious defects, such as tilt/decentering (see "clock tower test" or Jim Kasson's "acceptance test"). Other than that, the best thing is to compare to performance of other users or published reviews. Only places like aforementioned lensrentals have enough data to make an educated conclusion.

I have neither the time, skill, or desire to run lab tests on every lens I buy, and if a lens doesn't meet my expectations, I have no problem sending it back.

Roger reports that manufacturers quite often reject lenses sent for return or calibration as "within spec" even if they are obviously not.

That said, I see no reason to settle for "bad copy" when a better copy can be had for the same price.

No sample will be absolutely perfect. You'll have to stop somewhere and declare it "good enough".

In general, if you are interested in this topic I recommend reading an Lensrental's blog and Roger Cicala's articles on dpreview

https://www.dpreview.com/opinion/6856813208/roger-cicala-the-difference-between-sample-variation-and-bad-copies-part-1

https://www.dpreview.com/opinion/9596141908/roger-cicala-the-difference-between-sample-variation-and-bad-copies-part-2

https://www.dpreview.com/opinion/4042117089/roger-cicala-why-i-dont-use-an-mtf-bench-to-test-my-own-lenses

https://www.lensrentals.com/blog/2018/11/using-rapid-mtf-testing-how-we-test-monitor-our-lenses/

https://www.lensrentals.com/blog/2015/06/measuring-lens-variance/

ipopov Regular Member • Posts: 227
Cameras have variation too
2
JimKasson
JimKasson Forum Pro • Posts: 39,770
Asymmetry
5

Joe Moche wrote:

A common theme on many of these forums involves folks who argue over the sharpness of a particular lens. The argument often ends with "You got a bad sample. Send it back." Honestly, there seem to be an awful lot of "bad samples" floating around out there. Is the lens manufacturing process really so loose that a high number of bad samples would be produced? Are bad samples just an aberration, or is there a range of sharpness (from great to poor) that affects all lenses? Do lens manufacturers test a lens before boxing it up, and is there some minimum level of performance that renders the lens "good enough?" Finally, how can one know if a newly-purchased lens is a bad sample, other than by making a totally subjective assessment by comparing results to results claimed by others?

I have neither the time, skill, or desire to run lab tests on every lens I buy,

It's not that hard to do. I can do it in ten or fifteen minutes. The first time you do it, it'll take you longer, but soon you'll be as fast as I am.

and if a lens doesn't meet my expectations, I have no problem sending it back. That said, I see no reason to settle for "bad copy" when a better copy can be had for the same price. Thoughts?

The most common sample variations seem to result in asymmetrical sharpness variation. A well assembled lens may have aberrations, but those aberrations will be radially symmetric.

I have devised a test for that kind of asymmetry:

https://blog.kasson.com/lens-screening-testing/

-- hide signature --
 JimKasson's gear list:JimKasson's gear list
Leica Q2 Monochrom Nikon Z7 Fujifilm GFX 100 Nikon Z9 Hasselblad X2D 100c +1 more
Bernard Delley Senior Member • Posts: 1,914
how do you know for sure: DIY MTF measurements
6

Joe Moche wrote:

A common theme on many of these forums involves folks who argue over the sharpness of a particular lens. The argument often ends with "You got a bad sample. Send it back." Honestly, there seem to be an awful lot of "bad samples" floating around out there. Is the lens manufacturing process really so loose that a high number of bad samples would be produced? Are bad samples just an aberration, or is there a range of sharpness (from great to poor) that affects all lenses? Do lens manufacturers test a lens before boxing it up, and is there some minimum level of performance that renders the lens "good enough?" Finally, how can one know if a newly-purchased lens is a bad sample, other than by making a totally subjective assessment by comparing results to results claimed by others?

I have neither the time, skill, or desire to run lab tests on every lens I buy, and if a lens doesn't meet my expectations, I have no problem sending it back. That said, I see no reason to settle for "bad copy" when a better copy can be had for the same price. Thoughts?

In this article "Fast full-field modulation transfer function analysis for photographic lens quality assessment " published in  Applied Optics , I describe how to do this and what the results are like. The first link here is to the open access version of the article.

Needless to say that I use this type of incorruptible inspection routinely as part of my checking. This type of work quickly adjusts your perspective of what can be expected and what you easily note in the field when not hinted by the measurement.

 Bernard Delley's gear list:Bernard Delley's gear list
Olympus TG-6 Nikon D7200 Nikon D500 Nikon D850 Nikon AF-S Nikkor 14-24mm f/2.8G ED +13 more
QuietOC
QuietOC Veteran Member • Posts: 6,251
Re: Lens variations

Joe Moche wrote:

A common theme on many of these forums involves folks who argue over the sharpness of a particular lens. The argument often ends with "You got a bad sample. Send it back." Honestly, there seem to be an awful lot of "bad samples" floating around out there.

If they are offering you their particular lens, then that could be useful information.

I find discounted lenses are often poorer samples.

Is the lens manufacturing process really so loose that a high number of bad samples would be produced?

Sometimes.

I've tried multiple copies of several lenses. Some are more consistent, some vary a lot.

Are bad samples just an aberration, or is there a range of sharpness (from great to poor) that affects all lenses?

Some manufacturer's seem to have become much more consistent recently. But every lens is somewhat imperfect. But just like there are bad copies, there are also exceptionally good copies, and some variations offer performance tradeoffs for specific uses.

Do lens manufacturers test a lens before boxing it up, and is there some minimum level of performance that renders the lens "good enough?"

Sometimes. Probably.

Finally, how can one know if a newly-purchased lens is a bad sample, other than by making a totally subjective assessment by comparing results to results claimed by others?

I test every lens when I get them.

I have neither the time, skill, or desire to run lab tests on every lens I buy, and if a lens doesn't meet my expectations, I have no problem sending it back. That said, I see no reason to settle for "bad copy" when a better copy can be had for the same price. Thoughts?

One could spend an unlimited amount of time seeking better copies.

A bought a used Sony E 16mm F2.8 and wasn't particularly impressed with it, so I bought two more copies of it. Both were worse than the first one.

I bought a Samyang AF 24mm F2.8 FE. Three subsequent copies were all a little better than the previous one. The last one was significantly better. About the same time I sent a misaligned Tamron 24mm F2.8 in under warranty. It came back the same. Marc Alhadeff rates the Tamron as outstanding for sharpness and the Samyang as average.

 QuietOC's gear list:QuietOC's gear list
Pentax Q7 Sony a7R IV NEX-5T Sony a77 II Sony DT 16-105mm F3.5-5.6 +105 more
OpticsEngineer Veteran Member • Posts: 7,588
Re: Lens variations

i buy a lot of lenses where I work. My experiences closely match what Roger Cicala writes about about over at lens rentals.

 OpticsEngineer's gear list:OpticsEngineer's gear list
Fujifilm XF1 Olympus XZ-2 iHS Canon PowerShot G7 X Olympus Tough TG-4 Pentax *ist D +27 more
OP Joe Moche Regular Member • Posts: 188
Re: Lens variations

Thanks, everyone, for the opinions and links.  I don't know how many more lenses (except for vintage glass) I'm likely to buy, but I'll probably just have to invoke the "is this good enough for me" rule.  I've done things in the past (lens testing, fixing front-focus/back-focus, etc.) but found that when it all came down to it, they had very little effect on my end results, and never had an impact on the artistic value of my final edited photos.  I guess my question is really about getting less of a lens that someone else got for the same price.

Thanks again!

-- hide signature --

Joe

 Joe Moche's gear list:Joe Moche's gear list
Canon EOS 5D Mark II Fujifilm GFX 50R Fujifilm GFX 100S +6 more
D Cox Forum Pro • Posts: 31,932
Re: Lens variations

QuietOC wrote:

Joe Moche wrote:

A common theme on many of these forums involves folks who argue over the sharpness of a particular lens. The argument often ends with "You got a bad sample. Send it back." Honestly, there seem to be an awful lot of "bad samples" floating around out there.

If they are offering you their particular lens, then that could be useful information.

I find discounted lenses are often poorer samples.

Is the lens manufacturing process really so loose that a high number of bad samples would be produced?

Sometimes.

I've tried multiple copies of several lenses. Some are more consistent, some vary a lot.

Are bad samples just an aberration, or is there a range of sharpness (from great to poor) that affects all lenses?

Some manufacturer's seem to have become much more consistent recently. But every lens is somewhat imperfect. But just like there are bad copies, there are also exceptionally good copies, and some variations offer performance tradeoffs for specific uses.

Do lens manufacturers test a lens before boxing it up, and is there some minimum level of performance that renders the lens "good enough?"

Sometimes. Probably.

Sigma say they test every lens. Makers of budget lenses are more likely to test one in ten -- that would show up a problem in the production line.

Finally, how can one know if a newly-purchased lens is a bad sample, other than by making a totally subjective assessment by comparing results to results claimed by others?

I test every lens when I get them.

I have neither the time, skill, or desire to run lab tests on every lens I buy, and if a lens doesn't meet my expectations, I have no problem sending it back. That said, I see no reason to settle for "bad copy" when a better copy can be had for the same price. Thoughts?

One could spend an unlimited amount of time seeking better copies.

A bought a used Sony E 16mm F2.8 and wasn't particularly impressed with it, so I bought two more copies of it. Both were worse than the first one.

I bought one of those for use on the NEX-5N. As you say, not impressive, but it was good enough at the time. Topaz Sharpen cleans up the images, but one could argue that the detail is then false.

I bought a Samyang AF 24mm F2.8 FE. Three subsequent copies were all a little better than the previous one. The last one was significantly better. About the same time I sent a misaligned Tamron 24mm F2.8 in under warranty. It came back the same. Marc Alhadeff rates the Tamron as outstanding for sharpness and the Samyang as average.

 D Cox's gear list:D Cox's gear list
Sigma fp
bastibe
bastibe Senior Member • Posts: 1,073
Re: Lens variations
1

Joe Moche wrote:

A common theme on many of these forums involves folks who argue over the sharpness of a particular lens. The argument often ends with "You got a bad sample. Send it back." Honestly, there seem to be an awful lot of "bad samples" floating around out there.

I would guess that sample variations between forum members are far larger than in their lenses.

Which complicates things a bit, since you usually can't just send them back for a refund...

 bastibe's gear list:bastibe's gear list
Ricoh GR III Fujifilm X-Pro2 Fujifilm X-T3 Fujifilm XF 35mm F1.4 R Fujifilm XF 60mm F2.4 R Macro +3 more
Keyboard shortcuts:
FForum MMy threads