Variation Facts and Fallacies

Lens sharpness is widely discussed on the internet, so it shouldn’t come as any surprise that lens softness is also a major topic. However, my experience both as a photographer and as the owner of a lens rental business has shown me that much of this discussion is based on a fallacy. So just what is it reasonable to expect from a lens and how doggedly should you pursue the ‘best copy?’

Fallacy:

  • Noun - "an error in reasoning resulting in a misconception or presumption"

A bit of background

When I started in photography most of the forums I learned from had at least one thread a day about someone’s 'soft lens.' They knew the lens was soft because their camera worked 'fine' with all their other lenses. After a few years of running a rental business, though, I found myself in the following situation three or four times a week:

'The lens you sent me front-focuses, it's not good.'
'OK, we’ll overnight you a replacement.'

Only to find, when the first lens came back, that all our tests suggested it was perfectly fine. And the customer is very happy with the replacement lens - apparently, this one is 'fine.'

So what causes this problem? It's rather simple, actually. The fallacy here is the definition of 'fine.' Most people assume that 'fine' means, 'perfectly calibrated.' It became apparent to me that cameras and lenses are not perfectly calibrated, but rather, they all have some variation. That realisation shouldn’t have been shocking; every manufactured product has variation, why should cameras and lenses be any different?

For the next several years I continued to investigate this issue, writing articles about it as I went. Reading all of them now is an overly-long exercise and I’ve learned a lot since I began writing on this subject. So it seems time to put a summary article together.

Manufacturing variation affects all lenses from all brands – the only difference is the tolerance levels considered acceptable.

Some Semantics

In most online discussions, things derail when different people understand the same terminology differently. Discussions of lens quality are particularly prone to that, so let me define the terms as I intend to use them.

A soft or bad copy is a copy of lens X that clearly does not perform as well as other copies of lens X.  There are bad lenses out there. They happen.  

A design choice affects all the copies of Lens X. Every lens has some design choices made that we may not like.

Copy/sample variation is a slight difference that can be detected between different copies of lens X, or different copies of Camera B. Copy variation is detectable, but usually not significant (compared to a bad copy which is always significant).

The Search for the Holy Lens Grail

Before we go further, there are some people that should probably stop reading here. If you ever think 'for $1,500 I demand perfection,' this is not the article for you, it will just get you upset. The laws of physics are not suspended, nor are techniques of manufacturing altered, just because you demand it be so. If you enjoy The Quest for the Perfect Lens and other fantasy games, just move along and save yourself the aggravation of being administered this particular dose of reality.

Why Variation is Inevitable

First, I should mention that the problem has received more attention in recent years, oddly enough, because our equipment has improved so much. A 6MP crop-sensor camera didn’t expose the flaws in a lens that now, a 24MP full-frame camera makes painfully obvious. Better lenses contribute too: when a lens had four really soft corners it was hard to tell if one side was worse than the other. On a newer lens with sharp corners the difference may be instantly obvious.

It’s probably not a coincidence that photographer and blogger Lloyd Chambers brought attention to the fact that camera mounts aren’t always perfectly parallel to the sensor when he was testing Zeiss ZF 21mm lenses on a 24MP Nikon D3x. There's every chance it would not have been apparent if he had been shooting a lesser lens on a lesser camera.

The key thing to realize is that the problem is not limited to one brand, one type of lens, or even just SLR gear. Landscape photographer Joseph Holmes found significant variation in medium format lenses and focusing. Testing website SLR Gear found a batch of 50mm lenses that were all softer on the right side than the left. At Lensrentals we found a group of 300mm f/4 lenses from the same serial number run almost all suffered electrical failures. Why is this so?

Manufacturing Tolerances

Manufacturing tolerances are just that: a range of acceptable values, not an exact point. In other words, what is specified as a 1/4 inch diameter screw may be anything between 0.247 inches and 0.253 inches in diameter (see this chart from an internationally-renowned industrial fastener manufacturer). The machines that make them can’t be more accurate than that at reasonable cost.

Did you know that every time a glass manufacturer makes a run of a given optical glass, the refraction index and dispersion vary a tiny bit? The glass manufacturer furnishes a melt sheet to the lens manufacturer so they can make tiny adjustments in thickness or curvature of that element to compensate for the differences.

The glass is probably a tiny source of variation compared to the other components that make up a lens or camera. In addition to the multiple glass elements, there are clips, shims, and grips that hold them in place within the lens. There are helicoids, barrels, gears, and rings move them around to focus. Electrical motors and circuit boards tell them what to do, and, as Chambers has shown us, even the lens mount that connects the lens to the camera is a source of variation. 

Cross-section of a Zeiss 21mm lens. Note all of the clips, shims, and barrels holding the elements in position.

Many, if not most, of those components are outsourced to other companies or factories. A change in subcontractor could result in a slightly different part being supplied. Something as simple as a set of ribbon cables more likely to crack or a solder that is slightly less conductive being used on a circuit board could result in significant changes to a camera or lens.

If you delve into the manufacturer’s parts lists (when you can get them) you’ll find they take these variations into account and plan for them within a given tolerance range. For example, if you need to replace the front element of your Nikon 14-24mm lens (part # 1K104-xxx) you need to replace the adjustment washer behind it with one of five specific thicknesses - each copy of the lens requires a slightly different thickness for proper spacing of the front element to focus sharply.

There are numerous other variable-sized spacers and parts for every lens (most lenses have variable thickness spacers in at least three different places). In many lenses even the lens mount comes in several thicknesses or with shims so that the image focuses properly on the sensor. As someone who has to shim lenses fairly frequently, I can assure you it can be done close to perfection, but not perfectly. When a 0.06mm shim is indicated, we often have the choice of 0.05mm or 0.07mm shims - close, but not perfect.

Similarly, the lens must be electronically calibrated. The circuit boards inside each lens with its own AF motor contain adjustment screws to calibrate the frequency and current of the electronic pulses sent to the motor used to move the lens during focusing. Manufacturers don’t build in this adjustment because every lens is exactly the same; they build it in because every lens is slightly different and adjustments are necessary. And by the way: the factory manual gives an acceptable range for the adjustment, something like 150 +/- 2 KHz.

Frequency and voltage adjustment variable resistors on the printed circuit board inside a Canon EF lens.

It’s not my purpose to list every source of variation in a given copy of a lens or camera body. They are far too numerous. But just to give an idea of some major ones:

  • A lens has eight to 23 elements, each of which may vary slightly in its spacing from the other elements, centering along the axis of the lens, and tilting from right angles to the axis of the lens.
  • The focusing and zooming elements must move certain distances front to back within the lens and during their travel their tilt or centering may vary slightly.
  • The barrels and helicoids, and various slots in them, must machined so that the elements are not only aligned properly within them, but they are aligned properly with other barrels.
  • The lens mount may not be perfectly parallel to the camera’s sensor.

For everything on the list there is a slight variation within an allowable tolerance. You can demonstrate this for yourself if you have some friends (or camera club members) and can get several copies of the same lens. Mount a camera to a tripod and focus on a target that is very visible in live view at 10X magnification. Then, just change from one lens to another and watch the target be a bit off center in different directions with each lens. Just a bit for most lenses, but you might find one that's off target by half of the screen or so. 

Camera bodies are no different than lenses with regard to variation, and in fact could be even more problematic. Is the sensor perfectly parallel to the lens mount? Is the AF sensor properly calibrated to the imaging sensor? Is the AF mirror exactly aligned and angled in relation to the AF sensor? The list goes on and on. And in every case the camera is made nearly perfect, but not exactly perfect.

Lens-Camera Matching

During, and at the end of, the assembly process, the lenses and cameras are tested to make sure they are within the manufacturer’s specifications. If they are out-of-spec and get by quality control, then someone, somewhere gets a truly bad camera or lens. It doesn’t happen frequently, but it happens. When it does, it isn't a subtle call; it's very obvious the lens is bad. In mass-produced elements, quality control is likely to be conducted on a 'sample' basis - only one in every ten or every hundred units will be checked.

But even the cameras and lenses that meet specifications are still going to vary slightly. Many people think, 'I’ll try eight lenses and take the best one.' The reality though is that the only sensible definition of 'best' is 'best with the camera body you are using,'. This is because there is plenty of room for variance in the behavior of bodies, too.

Let’s consider just the lens mount as a theoretical example. First, we say the lens mount of a camera must be parallel to the sensor with a range of +/ - 0.05 degrees (I have no idea what an acceptable range is, I only know that they cannot consistently be made perfectly parallel). Then, let’s say camera A's lens mount has a tilt of 0.04 degrees to the right. Lens #32 has a tilt of 0.04 degrees to the left. The two tilts would cancel each other out, all would be magical, and the owner would write sonnets on the various forums praising his lens.

But let's say he sells his lens to someone whose camera's lens mount has a tilt of 0.03 degrees to the left. The lens and camera now both tilt to the left and the new owner may say, 'the lens you sold me is a bad copy, it’s horribly soft on the sides.'

And this is just tilt. The mounts may also vary in thickness. There is variation, too, in autofocus systems. Certainly more than there is in lens mounts. Through-the-viewfinder manual focus will also vary (you might be surprised to know that viewfinder focusing screens are shimmed by hand). Even exposure metering varies slightly from camera to camera, and requires recalibration if it’s out of specification.

The bottom line is every lens varies slightly, in several respects. And every camera varies slightly too, in a number of different ways. A given lens on four different cameras will behave slightly differently on each of them, and four different copies of the same lens will each perform slightly differently on a given camera body. How differently? Well the short version is you probably will notice the differences if you’re a 'pixel-peeper'. If you use cameras and lenses to take pictures, though, it would be very unlikely you’d notice normal variation, even with large prints. The one exception might be high quality wide aperture lenses because the narrow depth of field may make subtle differences apparent.

So how much of a difference does this make? Click here to read page 2

The views and opinions expressed in this article are those of the author and do not necessarily reflect the views and opinions held by dpreview.com or any affiliated companies.

Comments

Total comments: 231
123
CMDPHOTOS
By CMDPHOTOS (Oct 19, 2012)

Great article! However, I do have one problem. At what point did we decide that a $1500-$30000 lens shouldn't be "almost" perfect?I am sure there are those out there that think even $1500 is not a lot of money, but I'm not one of them! What ever happened to pride in workmanship or QC? Not to mention when did it become acceptable to spend this kind of money for a crap shoot (pardon the pun)? What if you can't afford to buy 3 different $5000 lenses to even try to hit a good lens? Are the tolerances displayed here something that should be acceptable? Yes, I do think I should be able to feel comfortable that my "only" $1500 lens will come to me in usable condition. If they can make one, they should be able to make sure that the odds are better than they apparently have. Buy the way, I would be willing to pay a reasonable price to have a new lens shipped to you first for a bench test before coming to me. No offense, but it is sad that I would have to go to this extreme to begin with.

1 upvote
ManVan123
By ManVan123 (Sep 29, 2012)

I Loved this articole !!!! Nice one.

0 upvotes
mikeon
By mikeon (Oct 18, 2012)

article

0 upvotes
Borek Lupomesky
By Borek Lupomesky (Jan 14, 2012)

I have EF 200/2.8L lens that's visibly softer on one side (but only at 100% pixels maginification). I have 5DMarkII that has slight magenta cast on the right side of image (but you need to shoot something really uniform and white to notice it). Well, nothing is perfect it seems, and I really decided not to worry about this too much.

0 upvotes
shahid11235
By shahid11235 (Jan 13, 2012)

"Tollerance value" is certainly an issue.. and NOTHING is "absolutely perfect", I agree. And if the manufacturers set a high precision level to minimize these types of flaw, then production cost must will increase which'll ultimately reflect an impact on price. This article might make some "perfectionists" understand why they can't get a "perfect" body or lens.

Another issue, one sample out of ten or a hundred is tested (I assume one is chosen randomly from a batch) for QC.. That's acceptable for a 150$ point-n-shoot camera, but for an 1200$ DSLR body or 1k$ lens? That doesn't make any sense.

Why don't you guys on dpreview run a test on various brands about their quality control and publish a detailed review on that? I think that'll help lots of photographers save both their time and money as well as energy?

Thanx :)

2 upvotes
Droppingin
By Droppingin (Dec 12, 2011)

My laptop ran out batteries before I finished the end

I made three posts: the first is (Dec 12, 2011 at 21:48:37 GMT) two posts down should be read first.

I just wanted to say that I am not attacking the author but my goal was to fill in the blanks or information that completed the picture of goes into the process control. Maybe because of background in the mfg'ing business the article, to me, painted a picture where things were not shown as controlled as they truly are. So I hope this balances it.

I did leave out in-process qc and final qc. It is one Hell of a lot more than testing samples these days. Specially when it comes to electronic equipment including things like lenses.

Crap, I just checked and one of my posts was lost and cannot remember what I said; probably said enough already.

Comment edited 3 times, last edit 10 minutes after posting
2 upvotes
Droppingin
By Droppingin (Dec 12, 2011)

If their is a change in supplier, the whole process is done again.

In the mfg' process, highly accurate and constrained jigs and fixtures are used to assemble parts such that the absolute minimum operator error impact is allowable. Knowing Canon, Nikon and others use robotic and dedicated automated machines to the critical assembly work.

With respect to screws, these are not critical items because if joining two parts are critical, an indexing tool pin is used which can be accurate to +/-0.001" or better depending on what is required. Screws simply help hold things together, not to locate two critical items. I am not saying the author made this claim but some folks may interpret his statements as such.

The modern machining processes are very accurate. Modern mfg'ing uses something called True Position analysis which when used properly and throughout an assembly and its subassemblies can produce amzaing results that surpass the traditional bias tolerancing. More in next post.

0 upvotes
Droppingin
By Droppingin (Dec 12, 2011)

I know I am not the only one that is/was a Mechanical engineer working electronics manufacturing or that we have the market corner on opinions on this topic. But here comes the however.

Generally speaking everything in the article is reasonably accurate but it does give rather bleak picture on the state of manufacturing and tolerance analysis.

I assume Canon, Nikon, and others have state of the art mfg'ing capability. They most likely use computer aided design software capable of sophisticated tolerance analysis before any material is cut or formed. That is, they have a good projection where there the problem areas will be.

All suppliers parts have to meet new part/supplier qualification programs that include: source(this means inspecting the supplier's plant) inspections, source quality control inspections, incoming part qualification programs, and ongoing parts-receivable inspections. I will finish in next post.

0 upvotes
Reactive
By Reactive (Dec 6, 2011)

What a superb article! I wonder if a photgrapher inspects and adjusts his masterpiece in Photoshop (i.e. pixel-peeping), then does the sharpening or automatic lens compensation made in Photoshop immediately outweigh and override any small errors made visible by a particular lens/camera combination?

It might be ideal to capture a perfectly focused shot 'in camera', but does software sharpening make the issue of tolerances largely irrelvant for most practical purposes (i.e. the perceived visual sharpness)?

Comment edited 59 seconds after posting
2 upvotes
fvdbergh2501
By fvdbergh2501 (Dec 6, 2011)

In theory you can undo blurring perfectly, but the conditions required for this do not occur in actual images (you need the PSF of the imaging system, zero noise, no quantization error, etc.). In practice this means that software sharpening introduces artifacts, which may or may not bother you.

A related issue is that the tolerance problems may introduce systematic bias (like Roger described above), which shows up as front- or back-focus. This means that the sharpest point of focus in your image may not be where you want it to be (imagine a portrait with the sharpest focus on the tip of the nose). You can only repair such a defect in software by selectively sharpening and blurring just the right areas of the photo, something that would become tiresome very quickly.

0 upvotes
S Hendrik Seelen
By S Hendrik Seelen (Dec 1, 2011)

@Roger
As you report live-view AF to be more precise, i.e. with less variation to it, I wonder what your data is on contrast- versus phase-based AF cameras--anecdotal and statistical. Do you get less complaints from four-thirds users than from dslr users? Did you do any experiments on 4/3's lens focussing? Are there any general "truths" to be gleaned on the difference between the systems?

0 upvotes
fvdbergh2501
By fvdbergh2501 (Dec 1, 2011)

If you want to evaluate the sharpness of your own lenses, or you wish to calibrate the AF fine tuning of your DSLR body, you should try MTF Mapper

http://sourceforge.net/projects/mtfmapper/

This utility is totally free, and source code is provided. Or you could buy Imatest for $300-$5000 :)

Please read the user documentation thoroughly, though.

I would appreciate any feedback, and welcome all discussion!

2 upvotes
Irakly Shanidze
By Irakly Shanidze (Dec 1, 2011)

Excellent write-up.

0 upvotes
ennemkay
By ennemkay (Dec 1, 2011)

very interesting analysis and presentation of data.

0 upvotes
Frank C.
By Frank C. (Nov 30, 2011)

silly pointless article, this read isn't going to change people's behavior one iota, his time would have been better spent reviewing some photo gear

Comment edited 4 times, last edit 4 minutes after posting
0 upvotes
Mr  Roygbiv rainbow
By Mr Roygbiv rainbow (Nov 30, 2011)

Great ! I am now paranoid about the nano mearsurements of machined surfaces that can effect optical alignment of lens/camera fit of my next purchase !

0 upvotes
Jim Lowell
By Jim Lowell (Dec 1, 2011)

Hey Frank C..
The article is interesting and does explain some things about why every lens is not perfect.

2 upvotes
PaulBearer
By PaulBearer (Dec 1, 2011)

Less pointless than at least one comment...

9 upvotes
HeezDeadJim
By HeezDeadJim (Dec 6, 2011)

I hope you don't watch movies, then. Watching them and then discussing what problems you had with friends and family over said movie isn't going to change one iota of why family member B thinks it's still a great movie. Instead of watching pointless movies (any movie) could be better spent taking photos. Because we all know that addressing/discussing certain topics is not worth ANY time than it being spent on being productive. Sleep is overrated.

2 upvotes
Jim Lowell
By Jim Lowell (Nov 30, 2011)

Good eye-opener article. I guess metal lens mounts on lenses are better than plastic ones to lessen wear and tear and maintain some accuracy over usage and time. Live and learn!

0 upvotes
AJs Dad
By AJs Dad (Nov 30, 2011)

Really good article and just about what I would expect from Roger (I've been a LensRentals customer for quite a few years). Lots of straight talk and unbiased opinion based in real world experience.

4 upvotes
AbrasiveReducer
By AbrasiveReducer (Nov 29, 2011)

Hi Roger - Since we're talking about pixel peeping and focus fine tuning, can you explain why you say the 28-75 Tamron is "...the best lens nobody knows about, and possibly the best bang-for-the-buck of any camera lens, period. Its nearly as sharp as the Nikon 24-70 , particularly on DX cameras, but much smaller, lighter, and cheaper...." I realize your remark about DX hints that the Tamron has soft corners but this seems like reading between the lines. Variations? I've had 4 of these lenses, with and without the motor and none has been particularly good. Granted, the alternative costs a lot more but the difference is equally dramatic; the 24-70 has sharp corners on FX.

0 upvotes
RCicala
By RCicala (Nov 30, 2011)

My take on the Tammie is several years old, written when the vast majority of photogs were shooting crop bodies (as opposed to less of a vast majority today, I guess). It is weak in the corners on an FX but on a DX I feel it's a bargain: 80% of the performance for 30% of the price. But more than that I was coming from the other direction: in the under $500 lens category there's this and the Tamron 17-50 as the f/2.8 zoom entries. Both are decent lenses at an amazing price, but neither is going to compete with a $1,500 lens wide open.

Roger

Comment edited 1 minute after posting
0 upvotes
taotoo
By taotoo (Nov 29, 2011)

Very informative article (as were your previous ones), but a couple of queries:

"If they are out-of-spec and get by quality control, then someone, somewhere gets a truly bad camera or lens. It doesn’t happen frequently, but it happens. When it does, it isn't a subtle call; it's very obvious the lens is bad."

This doesn't stand up to reason - it suggests cameras/lenses SLIGHTLY out of tolerance always get caught by quality control.

"The reality though is that the only sensible definition of 'best' is 'best with the camera body you are using,'."

And best at your chosen focal length, aperture, focus distance etc...

0 upvotes
RCicala
By RCicala (Nov 30, 2011)

I didn't mean to infer that slightly out of tolerance lenses get caught. Poor wording on my part. Rather I was suggesting that an occasionally awful lens does get by. Not often, but sometimes.

And good point about focal length, aperture, etc.

Roger

0 upvotes
taotoo
By taotoo (Dec 1, 2011)

Then I wonder why your data doesn't show any slightly out of tolerance lenses?

Assuming your interpretation of your data is correct, then it backs up your statement - that when there's an error the error is obvious.

But common sense suggests that it is more likely that a highly out of tolerance lens would be caught by QC than a slightly out of tolerance lens, or at least that there would be a significant number of slightly out of tolerance lenses appearing.

Maybe there is some explanation e.g perhaps when a lens receives a knock during delivery the alignment inevitably moves out by a large amount (and never a small amount).

0 upvotes
Summit
By Summit (Nov 29, 2011)

Thank you for providing data! Perhaps consider statistical analysis? (for fun)

0 upvotes
ljgude
By ljgude (Nov 29, 2011)

Great article that confirms much of what I have picked up over the years. I own a Kodak Rapid Rectilinear from about 1900 and one of its 'perfected' descendants - a Zeiss Tessar from the 30s. Yet Stieglitz, Adams, Weston all shot with lenses of that general quality. I shot Nikons in the 60s and 70s and had real need of a short zoom, but was plagued by the softness of Nikons 43-86 zoom until one day a student showed me a sharp 8 by10 he claimed he shot with the 43-86. I tested it; it was sharp! Here is a reverse example of the thrust of the article and clear evidence that things have improved. I recently reentered photography with a 600D and was wowed by the sharpness of the 18-55 kit. Next I bought a Canon IS 100mm macro on reputation. With my rheumy old eyes I really am not sure what sharpness is anymore but I can see a difference between the two lenses. Objects look solider, more real with the macro...more like ,well, Weston's 8 x 10 contacts.

0 upvotes
Hiram P Bumblesniger
By Hiram P Bumblesniger (Nov 29, 2011)

What are you measuring on these graphs? There are no labeled axes. Without telling us what the graphs are really showing, how can we assess anything? Please add x and y axis labels. The graphs are pretty meaningless without them. Thanks.

0 upvotes
RCicala
By RCicala (Nov 29, 2011)

Hiram,

Trying to keep graphs in the allowed image size made the labels too small so I identified the axis in the text: Both axis are MTF50 in Line Pairs / Image height. The horizontal axis is center point resolution, the vertical is average resolution of the entire lens.

0 upvotes
hansaus32
By hansaus32 (Nov 29, 2011)

How do I search on dpreview.com for cameras with the front/back focus (aka microfocus adjustment) feature?

0 upvotes
Richard Butler
By Richard Butler (Nov 30, 2011)

At present you can't, but I'll put in on the list of things to consider adding to the database.

0 upvotes
SteveHageman
By SteveHageman (Nov 29, 2011)

Well done article - I would love to have a blow up of that cut away lens hanging on my wall. How do they fit all that stuff in there? Can you imagine designing that in the days before computers?

0 upvotes
jimkahnw
By jimkahnw (Nov 28, 2011)

Roger confirms my intuition on equipment build quality--a "broad" range of tolerance. I went through a quest with two copies of a Nikkor 16-85 zoom. I told tech support that images were "soft" at short focal lengths and with distant subjects the lens never hit the infinity mark. They looked at my samples and said they were within tolerance. I disagreed and they serviced the lenses without improvement. I sent them back with the camera body, a D300, again failure. So, I took equipment to the Nikon repair station in Melville, NY and pleaded my case to the techie. I even shot a series of images of the Nikon headquarters to demonstrate the problem. When the equipment was returned, lo-and-behold, the lenses were sharp. I don't know what repair was performed. As I understand it, a lens calibration with the camera is only good for one focal length; useless on a zoom. Oh, and the lenses still do not hit the infinity mark on distant subjects and short focal lengths. So much for zone focus.

0 upvotes
jack1a
By jack1a (Dec 5, 2011)

I find the main article interesting however I feel quality should never be an issue it never was with film SLR's you knew what you were buying, you could also focus and set DOF and subject position within DOF from the lens barrell markings. I have never been happy with either of my crop sensor DSLR's (30D&40D&50D) with my L lens even the "kit" lens were "Soft" no pixel peeping required. I also was told by canon that there was nothing wrong twice with all bodies and 3 different L lens. So I experimented because I needed a work around and happened on a set up where i could consistantly repeat the annomallies and which helped me figure out the workaround. I foolishly believed it would improove with next model body but I believe it is a fundamental mismatch of 2.8 L series lenes to crop sensor bodies that canon can't admit.
No 60D for me I have enough paperwieghts I have resurected the F5 and some fuju 100 slide film

BTW Jim I can't get anything sharp in infinty either

Comment edited 2 minutes after posting
0 upvotes
AbrasiveReducer
By AbrasiveReducer (Nov 28, 2011)

This is good to know and not surprising. But I still think it's unintentionally misleading. I've never used a 100 macro of any brand or price that wasn't at least very good. However, I've had many dreadful wide angle zooms that no amount of variation/calibratrion could explain away. Since the consumer can't control the variation, the first question has to be "Are ALL copies of a certain lens, dogs?" It's helpful to know that certain lenses, by design are lousy--and this includes some very expensive ones. My mother would make a great lens reviewer because she would say "If you can't say anything nice, don't say anything at all".

0 upvotes
Marek Rucinski
By Marek Rucinski (Nov 28, 2011)

Fantastic article, thanks a lot!

I don't even dare to ask if you're willing to disclose all your lens testing data ;)

0 upvotes
DrugaRunda
By DrugaRunda (Nov 28, 2011)

the next stage is for someone like DPR to take 10 lens copies of each 70-200 f2.8 lenses

Canon L IS
Nikon VR
Sigma OS (Nikon + Canon version)
Tamron (Nikon + Canon version)

and test on 10 bodies Nikon 3Dx and 10 Canon 1Ds MKIII

So we have 30 lenses x10 bodies for Canon and Nikon = 600 tests

Only after that we would have it all settled, not.

The article should be named "There can be only one!".

2 upvotes
cbsva
By cbsva (Nov 29, 2011)

Not to forget Sony, Olympus, Pentax and all variations between FF and APS-C bodies... ;-)

0 upvotes
Antony John
By Antony John (Nov 29, 2011)

No actually it would seem that many camera users would like the camera manufacturers to come knocking on their door to check if the particular copy of a lens they have will suit the particular camera the photographer has before putting it on a shop shelf somewhere in the world.

0 upvotes
MalbikEndar
By MalbikEndar (Nov 28, 2011)

> something like 150 +/- 2 MHz

Are you sure you don't mean KHz? 150 MHz is very implausible for an ultrasonic motor.

0 upvotes
RCicala
By RCicala (Nov 28, 2011)

You are absolutely correct - I typoed that. Roger

0 upvotes
Musiclady
By Musiclady (Nov 28, 2011)

Interesting reading, but also filled with some hot air. Solder in a circuit that doesn't conduct as well as some other brand or type of solder? The amounts of solder used in these boards is so small as to have no significant or measurable effect on conductivity.

With lenses as with cameras, you usually do get what pay for, including tighter tolerances. Plus or minus 20% is probably way beyond the tolerance range for Leica glass and probably wider than Canon's L lenses. Also, the better lenses are made of materials that have a similar co-efficient of expansion and contraction so that the critical alignments are held in position regardless of whether it's cold or warm/hot weather.

For me, the bottom line is that the "soft" or inaccurate lens is the rarity compared to the lenses that perform just the way they're supposed to. I also think out-of-focus or front/rear focus issues have more to do with human error and/or a maladjusted camera AF system than they do with the lens itself.

1 upvote
mlackey
By mlackey (Nov 28, 2011)

It is HOW the solder is applied. Having 10 years experience In the automotive electronics world, a "small" quantity production line may produce around 200K modules per year, a "large" quantity line in the 1M range. Any manufacturing defect, no matter how small, is bound to surface.

Consider flow soldering: minor flaws show up sooner or later as manufacturing defects, no ifs, ands or buts about it. The defect can surface as intermittent, variable resistance, etc. It may not even surface unless the printed circuit board (PCB) is flexed (or mounted) in a certain way at a certain temperature. Solder itself is subject to manufacturing tolerance, and some brands may not perform (adhere) as well as a (chemically) different brands. PCBs have manufacturing tolerance, ICs have manufacturing tolerances... the effects of all this are multiplicative, and can add up to staggering odds AGAINST producing a perfect product each and every time.

Hot air... I disagree.

Mike

4 upvotes
veroman
By veroman (Nov 29, 2011)

You're both right ... except, as always, the proof is in the pudding. The number of lenses or cameras that are out of spec due to variations in solder application or solder type or anything else related to solder, including solder along the PCBs, is so small as to make it a moot issue. In fact, the amount or error that can be attributed to solder application is so small as to be immeasurable ... at least, on a lens or camera PCB. There was far, far, far more error when solder and components were applied by hand and there is far, far, far more error (or tolerance swing) with integrated circuits than with solder application. I believe the mention of the solder is part of the "hot air" of the article.

0 upvotes
mlackey
By mlackey (Nov 30, 2011)

"...so small as to be immeasurable ..."

Depends on scale. My guess is you've never seen an automobile dashboard that's been melted by a "thermal event".

If I use this same logic, atoms do not exist because they are "...so small as to be immeasurable ..."

Hot air... I disagree.

Mike

0 upvotes
HeezDeadJim
By HeezDeadJim (Dec 6, 2011)

It's not questioning if errors exist or not, it's about which errors are going to factor considerably the end result. A sniper shooting a soda can from 300ft is probably going to notice scope error first over trigger wobble. Trigger wobble error exists and will incur shake, but not noticeable at 300ft. And atoms do exist. Without Atom and Eve, we wouldn't be here...

0 upvotes
mlackey
By mlackey (Dec 7, 2011)

"It's not questioning if errors exist or not, it's about which errors are going to factor considerably the end result."

So you are saying that if the probability of an error is small it can be ignored? Sorry, but I can't accept that answer. Any error, no matter how small, is too large if you are the one that happens to experience it.

Mike

0 upvotes
mike16
By mike16 (Nov 28, 2011)

Could someone comment on 'random' softness that sometimes occurs with zoom lens. Most pictures with lens fine but perhaps once in about 1-200 times an area towards middle righthand side is noticably soft, no pixel peeping required. Has happened with a couple of mid range zoom lenses I've had, once in era of film and one digital. I take thousands of shots per year so this is a significant amount of shots lost and it seems unpredictable as to when it will happen, just at that particular zoom/focus combination which you can't find again to reproduce the effect. Do those computers they use to simulate how the lens will perform cover every possibility of focus/zoom and if they find a particular very small combination of focus/zoom that happens to be soft does it still get through into production?

0 upvotes
tbcass
By tbcass (Nov 28, 2011)

Leonard Shepherd; screw diameter tolerances can certainly effect IQ variances because it allows variations in lens placement from lens to lens. For example one lens element may be slightly to the right while the next may be slightly to the left.

0 upvotes
tbcass
By tbcass (Nov 28, 2011)

TORN, that is the reality pf economics. If you expect manufacturers to hold lenses to the tolerances you want be prepared to remortgage your house to pay for just one. The biggest problem I have with these gear forums is the ease with which people can zoom in on photographs to find and complain about minute IQ problems that have nothing to do with the quality of the photograph.

1 upvote
TORN
By TORN (Nov 28, 2011)

First I follow and appreciate Mr. Cicalas articles for quite some time now.

But with this evaluation of a 20% difference between best and worst acceptable copy being just fine the manufacturers should be very happy.

Follow the argument with the quality rating being barely within the 5-limit then what about enlargements? I prefer to often cut my shots sometimes giving away half of the pixels and thereby increasing the error by a large amount.

In my book a sample variation of 20% is not acceptable to me. At least I want to reach the middle ground and this can usually (!) be archieved by testing 2-3 samples (well knowing that by really bad luck I test three mediocre samples, but hopefully my experience helps me out in this case). Since I am not looking for the top of the best my goal can be reached with a very limited extra effort per lens buy. Ok, the risk of having a really off camera is still there but can be mitigrated by testing my other cams as well.

0 upvotes
julieng
By julieng (Nov 28, 2011)

20% difference may not be as big or significant as the number suggest.

0 upvotes
Leonard Shepherd
By Leonard Shepherd (Nov 28, 2011)

It is an interesting article - but how reliable is it?
Roger uses an example of a screw diameter - screw diameter does not affect assembly tolerance, though it might cause tightening issues.
In "whats it all about" point 5 there seems to be a misunderstanding of how AF works. If starting AF front infinity and then minimum focus gives different results the size of the parallel part of the target is not big enough relative to the length of the AF detection line.
What is happening is the AF is detecting detail further away coming from infinity and closer from minimum focus that would not occur with a bigger target. This is not "tolerance variation".
I agree lens and body variations likely to show in a 20 inch wide print are rare. If you have fast broadband this link I got from dpreview indicates the various electronic test stages in assembling a Nikon 24-70 - and includes a Nikon AF test target.
http://lens-club.ru/public/files/pdfs/4b4fb97547c5d54819491c07c715f2c2.pdf

0 upvotes
RCicala
By RCicala (Nov 28, 2011)

Leonard, for the example I use the same result occurs focusing on a star chart on a flat wall: there's nothing in front of or behind the target for the autofocus system to detect.

Roger

0 upvotes
paqman
By paqman (Nov 28, 2011)

EXCLLENT article - thank you! have suspected major take aways for years but you've given all the backup.

1 upvote
singveld
By singveld (Nov 28, 2011)

what about lapping? Just like people who lap the heatsink so that there is better contact with processor. Therefore, if there is a way to accurately measure the tolerance. The mount can be lap by sand paper.

0 upvotes
ulfie
By ulfie (Nov 28, 2011)

There is also thermal expansion and contraction of parts to consider as well as centering and the optical effects of multiple lens coatings. As Suzuki-Roshi said, "Seek perfection in imperfection."

0 upvotes
harold1968
By harold1968 (Nov 28, 2011)

quite correct

however "manufacturer tolerance" is key

I presume Canon L lenses have lower tolerance then normal ones ?

Now I am also using Leica. Leica's lenses on my M9 are bank on razor sharp from the factory.
Lower "tolerances" ? probably .......

1 upvote
Tord S Eriksson
By Tord S Eriksson (Nov 27, 2011)

A few months ago I bought a very old Pentax-M 400/5.6, which didn't work too well with my Pentax K-x, nor the K-5, but excellently with my new Sony NEX-5N, and its K Mount adapter!

Somehow that mix turned up far better, but you have explained it excellently ;-)!

Even with my wife's E-PL1 things seemed better with the 400 (Novoflex K Mount adapter).

Yours,

Tord

1 upvote
kimvette
By kimvette (Nov 27, 2011)

Somehow, I think even with the stats and analysis posted here in this article, we will still have daily "my lens is soft" and "I exchanged five copies of my foomatic f/2.8 lens before I got a sharp one" threads.

My used 18-55mm was sharp out of the box but the others were a bit soft - but sharp in liveview with contrast AF. A few minutes with a printed target and microadjust cleared it right up - no complaining and exchanging lenses nonsense because obviously given microns' worth of tolerance, even a "perfect" lens would no longer be perfect after being handled by UPS.

I'd love to see an article by Cicala covering high-end DSLR image "softness" compared to P&S and even entry-level DSLRs, along with an analysis of the underlying causes and reasons for it. And yet, that still would not stop the nonsense.

1 upvote
mick232
By mick232 (Nov 27, 2011)

The "nonsense" is there because many manufacturers don't allow AF adjustment except in high-end bodies.

3 upvotes
kimvette
By kimvette (Nov 28, 2011)

But even high-end body owners post that they keep exchanging lenses until they get a good one. It is a topic that comes up from time to time and explaining what the impact of even tighter tolerances when it's already in the micron range would have on the price of glass, they still insist that they should not have to adjust for front/back focus (aka microfocus adjustment).

The tolerance on these things is so tight that a lens that is "perfect" on your camera may not be perfect after being subjected to UPS's mishandling of packages, and one that is not "perfect" from the factory very well may be "perfect" for your body when it arrives.

IMHO If it is within the +/- range MFA allows for, it's a keeper (and it's very rare it won't be within that range).

1 upvote
photo nuts
By photo nuts (Nov 28, 2011)

Mick232:
a) MA does NOT correct misaligned elements in lenses
b) In their current implementation, Nikon and Canon MA do NOT work well for zoom lenses.

0 upvotes
mick232
By mick232 (Nov 28, 2011)

Yes, but we are not talking about misalignment here (which would be a defect) but about lenses that are within specifications.

I agree that it doesn't work so well for zoom lenses. But current MFA is far from perfect, isn't it? At least I'd want to have different correction values for the long and short end instead of just one.

0 upvotes
mbrobich
By mbrobich (Nov 27, 2011)

Excellent article. Just went thru all of my lenses and tuned them to my liking !!

0 upvotes
sherwoodpete
By sherwoodpete (Nov 27, 2011)

Why not use the correct word "sample" rather than "copy" which implies plagiarism or worse?

0 upvotes
CaseyComo
By CaseyComo (Nov 27, 2011)

Pixel-peeping, meet semantic quibbbling. You'll get along great.

11 upvotes
SteveHageman
By SteveHageman (Nov 29, 2011)

Dictionary.com lists as a definition for "Copy" as,

"one of the various examples or specimens of the same book, engraving, or the like."

So the authors use of the word "Copy" is within the definition of the word.

3 upvotes
em_dee_aitch
By em_dee_aitch (Nov 27, 2011)

.... Continued from last post: So when you say that your technicians can easily detect a "bad lens" but not distinguish among the acceptable "variations" in your chart, I think you are conveniently omitting cases in which the lens may be within the acceptable variation at the test point within the frame (or even on average) and yet be yielding images which are to some degree easily detectable as of asymmetrical quality to the naked eye. I bring up this omission because it is the primary mode of failure that I believe reasonable people can easily spot without going so far as Imatest or even charts.

Comment edited 1 minute after posting
0 upvotes
RCicala
By RCicala (Nov 27, 2011)

Actually I completely agree (and a great example of the same definition meaning different things to different people). But I consider theses one of the easily detectable "bad lenses". Optically it's readily apparent as a portion of the image not nearly as sharp as the rest. Imatest shows it as acceptable peak sharpness with bad weighted average sharpness.

But I think you've given me a nice lead for a new article: I can definitely do some examples of this.

Best,

Roger

0 upvotes
em_dee_aitch
By em_dee_aitch (Nov 27, 2011)

Dear Roger - Thanks for this new piece in your series. I enjoyed the new information contained here. However, I have one bit of constructive criticism. You go briefly into the aspect of visible decentering (i.e. you mention the phenomenon of some lenses being off target), but you did not go into the obvious effect this has on focus. I am concerned that you downplay this a bit in that this is one of the focus effects that can make a huge visible difference to a working photographer, and it happens more often than any of us would like in lenses like the Nikon 70-200 VR2. The issue is that when an element is notably off center you will get extremely sharp focus in one part of the intended focal zone and dull focus in another part of intended focal zone. Another way of putting this is that the focus zone is skewed rather than perpendicular. Often this reproduces with the lens on multiple camera bodies, i.e. is a true lens issue and can't simply be blamed on lens/body "combination." Cont...

0 upvotes
dopravopat
By dopravopat (Nov 27, 2011)

Maybe lenses should have MFA themselves. That would be great.

1 upvote
RCicala
By RCicala (Nov 27, 2011)

The interesting thing is lens MFA wouldn't be hard: each lens contains electronic hook-ups inside that the manufacturer uses with proprietary software to adjust focus. It wouldn't fix a decentered or tilted element, but it would make autofocus more accurate if we had the option of accessing those electronics.

0 upvotes
WIMorrison
By WIMorrison (Nov 27, 2011)

An excellent article which is entirely objective, and proves fairly succinctly that the subjective comments about lenses posted in the forums are essentially bunkum.

6 upvotes
kombizz0
By kombizz0 (Nov 27, 2011)

No wonder time to time I found soft images.
Not sure what did you mean by microfocus adjustment. How it works and how can I do it.
Thanks

0 upvotes
MarcusGR
By MarcusGR (Nov 27, 2011)

In real-life situations there is at least another element which - in theory - might affect lens performance: temperature. When using a camera outdoors the temperatures' range may easily exceed 40 degrees (Celsius), and both plastics and metal will be heavily affected ...
Then, of course, there will be slight accidents: most of us will occasionally end up shocking our lenses and cameras both inside or outside our bags ...
The only solution I can foresee is regular calibration : a truly professional camera should be equipped with several calibration options, both a the camera-body level and at the lens level. And the whole system should be regularly re-calibrated, after exposure to shocks or temperature extremes expecially.
Alternatively, I think software might be developed to "virtually" re-calibrate a system by adjusting the electronic manipulation of the collected image ?

1 upvote
Desert Cruiser
By Desert Cruiser (Nov 27, 2011)

Best article yet! I didn't realize all that was involved and now I know. Thanks, very informative article. All the years we've used D-SLR's we've never found one that was unacceptable, this explains why, we were lucky with bodies and lenses. And like you said people are too critical in many cases. With all that said, we've only found one lense (on a micro four thirds) that has an anomaly that we decided was unacceptable, to us anyway. We could never get anyone to tell us if this was the case or if we were maybe expecting too much? Please take a look at this video and tell me if this something that you'd consider as acceptable? I would love to know......
http://vimeo.com/25066664
We've had other cameras with video capabilities but none that had this problem, we believe it to be in the lens, as it clicks when mildly shook off the camera like something is loose in the lens. I'd really appreciate an expert's opinion.
Thanks in advance,
Don....

Comment edited 2 minutes after posting
0 upvotes
RCicala
By RCicala (Nov 27, 2011)

Don, that clicking is definitely not normal. Something is loose in that lens. Every time we've opened up a lens like that we've found a loose screw, or even a loose element.

0 upvotes
Desert Cruiser
By Desert Cruiser (Nov 27, 2011)

Thank you, Panasonic said it was normal, and so did a couple of people here on the forum. I'm still going to use the camera but the lens is a cheap one. I really think they should have fixed it, but they didn't, so we'll have to look for another better quality lens. Thanks now I know I was correct in my assumption, thanks again for a great article.

0 upvotes
jj74e
By jj74e (Nov 27, 2011)

At the end, what does it mean by microfocus adjust it? How exactly does one do that?

0 upvotes
SteveHageman
By SteveHageman (Nov 29, 2011)

Some middle to upper range changeable lens cameras now allow you to "tweak" the nominal focus point for an attached lens. Plus or minus some amount. The camera then remembers this setting with this specific lens (usually by lens model, not by lens serial number). The Canon 7D has this adjustment while the Canon 60D does not.

0 upvotes
Antony John
By Antony John (Nov 27, 2011)

For those who think that perfection can be reached by throwing money and QA at something think again.
The most obvious optical example can be summed up in one word - Hubble.

1 upvote
tagomilonga
By tagomilonga (Nov 27, 2011)

You are wrong!
Hubble main lens was made by Perkins Elmer.
NASA awarded contract to PE because their bid was $1m less than Kodak's. PE cheaper bid did not include testing resulting in $1m saving. NASA was fully aware of this but took the risk and paid for it $1.5b in subsequent repairs in space.

3 upvotes
SteveHageman
By SteveHageman (Nov 29, 2011)

NASA also didn't spend the $$$ to roll the Hubble outside and focus it on anything. This was recommended by the engineers, but alas... Pay me now or pay me later. At least it gave some Astronuts something to do.

0 upvotes
Antony John
By Antony John (Nov 29, 2011)

Tagomilonga you have justified my argument, not nullified it. No matter how much testing is done there are still possibilities that something could go wrong. Even if Kodak had manufactured the lens there is no certainty it would have performed any better - even if 'properly tested' - I cannot imagine PE would have done zero testing.
Ultimately even for a meg$ project there were restrictions on cost so how do you expect a commodity lens manufacturer to do 100% check on a 'low cost' lens?
No matter how much one spends on QA there will always be a possibility of an error. There is no such thing as 'zero tolerance'.

0 upvotes
jack1a
By jack1a (Dec 5, 2011)

Quality will never improve if we dont have "zero tolerance" for crapy equipment

0 upvotes
Total comments: 231
123