Autofocus System Design

Started Aug 15, 2014 | Discussions
Marianne Oelund Veteran Member • Posts: 7,274
Autofocus System Design
103

This thread will present the stepwise development of a phase-detect autofocus system, using basic optical concepts and ray diagrams. The intent is to lay a solid foundation for the reader, to understand concepts critical to autofocus optics and operation, at a level which is visual, intuitive and readily understandable. There will be some mention of mathematical concepts that apply, but a working knowledge of them is not required in order to follow the discussion and diagrams.

See the following posts for presentations of each step in the development. More posts will be added later, as I have time, and/or in response to questions. The initial posts cover the fundamental optics for the AF system, starting with a single lens, then adding more optics to complete the AF system optical model.

There are many misconceptions associated with AF system behavior, as it is not always intuitive. Some readers may have difficulty accepting the system characteristics described, and ask for supporting references. The best reference I can give, is an optical system that I have sitting on my table right now, configured as detailed in the first few posts: It functions exactly as specified in this thread. I will post some details of that system, and photos of its operation, at a later time (taking photos can be easier than constructing theoretical diagrams, anyhow).

Suffice it to say that this thread will present more than purely theoretical concepts. It is my hope that this will be both fun and educational.

-- hide signature --

Qualities possessed by God in infinite proportion: Love, Grace, Power, Righteousness, Wisdom, . . .
Qualities possessed by humans in infinite proportion: Ignorance.
- Marianne

OP Marianne Oelund Veteran Member • Posts: 7,274
Step 1: The Phase Plane
18

All optical systems need to start somewhere, so let's begin with a single lens. In addition to the usual considerations, though, I'd like to discuss another aspect that is important in autofocus optics: What I refer to as the "phase plane," also known as the aperture plane.

Here we have the familiar double-convex lens, its object plane, and image plane where focus is achieved:

The object planes and image planes are completely interchangeable. You may place a subject at either one, and an image will be formed at the other. Hereafter, I will often refer to these planes simply as "image planes" regardless of whether an object, or an image, is placed there. The lens works by providing a straight one-to-one correspondence between points on its two image planes, and does so in a well-behaved, linear fashion with (ideally) no scale variation.

Now let's think a bit about the light rays at the plane of the lens itself. The first fundamental concept is that you can take any small area on the lens, and the light rays passing through just that tiny area can form a complete image. (Anyone who has worked with holograms will be very familiar with this concept. Holograms use wave phase information - interference patterns - to encode an entire image at every elemental area on the film.)

All of the rays required to form a complete image, are passing through the small area on the lens plane. How is the image information represented? Each point on the image (or subject) has a unique ray angle associated with it, that is, the ray angles "encode" the image. In many applications, angles constitute "phase" thus my term for this plane is "phase plane." A mathematical operation known as the two-dimensional Fourier Transform can convert phase-plane information to image-plane information, and vice versa.

The only difference between the image projected by the entire lens, and the image projected by any small area of the lens, is the image brightness. In both cases, the image will still be complete, even if we chose a small area that is off-center.

Now, if we consider all of the small areas on the lens plane together, we see that there is a large collection of rays, constituting a somewhat complex light field. Think about this: What if we had a way to precisely duplicate such a light field artificially? If we placed a lens at that light field, it would be able to project an image from it.

When you're comfortable with that idea, go on to Step 2.

-- hide signature --

Qualities possessed by God in infinite proportion: Love, Grace, Power, Righteousness, Wisdom, . . .
Qualities possessed by humans in infinite proportion: Ignorance.
- Marianne

OP Marianne Oelund Veteran Member • Posts: 7,274
Step 2: And Then There Were Three
14

Now we get to do something apparently destructive: Take a thin diamond saw, and cut the lens into two sections, along its central plane, and polish the surfaces nicely. That makes two plano-convex lenses with the same diameters as the original lens.

Line the two lenses up on the same axis, with a wide space between them. To restore our optical system to its former operation, what could we do? Think about that phase plane again: If we could transfer the phase plane produced by the left lens, to the surface of the right lens, then it could project the same image that it did before, when the two lenses were still one.

If that sounds too difficult, rest assured that it's not. In fact, it's a perfect job for another convex lens, and we then end up with a system of 3 lenses. As it turns out, a lens is not only capable of projecting an object to an image, but it's also perfectly suitable for projecting one phase plane, to another phase plane. Let's inspect a few of the rays to see how this works.

Choose a lens with a focal length that is 1/4th of the distance between the two plano-convex lenses that we made with the diamond saw, i.e., the distance between the plano-convex lenses is 4f, where f is the focal length of the third lens we add, placed exactly between the other two lenses. Call this third lens the "field lens" since its job is to transfer the light field from the left lens, to the right lens:

By inspection and symmetry, we see it's possible to select any point on the left lens, and a pair of rays coming from that point, through the field lens equidistant from the optical axis, which arrive at a corresponding point on the right lens (same point, vertically inverted). Additionally, the angles of the rays have been precisely duplicated, since the rays form a parallelogram around the field lens. Since we now have rays arriving at the right lens, at the same (vertically inverted) point, and at the same angles that they had leaving the left lens, the light field has been duplicated. The right lens must be projecting the same image that it did before - except that it's inverted. (The inversion won't cause a problem for our AF system, as long as we allow for it.)

Our system of 3 lenses has five planes which are significant to us, but which have slightly different meanings to the three lenses:

1. At the far left, the left lens object plane.
2. At left lens center, its phase plane - which is also the field lens object plane.
3. At field lens center, its phase plane.
4. At right lens center, its phase plane - which is also the field lens image plane.
5. At the far right, the right lens image plane.

There is a relationship between the field lens focal length, and the left/right lens focal lengths: For an AF system, we want the field lens phase plane to coincide with the left lens image plane, in other words the left lens is projecting its image onto the field lens. We also have the right lens object plane coinciding with the field lens phase plane. (Note the horizontal scale of this diagram is compressed relative to the original single-lens diagram.)

I should probably comment that with the three lenses spaced this way (left-lens image plane coincides with right-lens object plane), we could remove the field lens and the two remaining lenses would form the same image. However, the field lens still serves an important purpose, as we will see in the next step.

Note: If you are familiar with relay lens systems, you will notice some similarity. Relay systems differ in that they align image and object planes only, thus are much greater in length (it is their purpose to lengthen optical systems).

The next step will show the advantages we can obtain from the image/phase plane coincidences.

-- hide signature --

Qualities possessed by God in infinite proportion: Love, Grace, Power, Righteousness, Wisdom, . . .
Qualities possessed by humans in infinite proportion: Ignorance.
- Marianne

OP Marianne Oelund Veteran Member • Posts: 7,274
Step 3: Virtual Apertures
14

Here is where the field lens becomes especially important. Recall that its image planes are at the left/right lens phase planes. This means that it will project any object at one of those planes, onto the other. For example, if we take a Sharpie pen and write a letter on the left lens surface, then shine some light through it, the field lens will project that letter onto the right lens surface.

This will also work for an aperture. Let's add an aperture diaphragm to the left lens. The field lens then projects that diaphragm onto the right lens. In other words, a covering on any part of the left lens that will not pass light, will deny light reaching the corresponding part of the right lens. We say that the right lens has acquired a virtual aperture, which is identical in size and shape to the real aperture on the left lens.

This works in reverse, as well. Placing an aperture on the right lens, produces a matching virtual aperture on the left lens. Any light passing through the left lens, in the covered virtual-aperture area, will hit the aperture diaphragm on the right lens and thus will not reach the right lens. Conversely, light passing through the open area of the virtual aperture on the left lens, will hit the open area of the real aperture on the right lens, and thus pass through:

Since our system presently has left and right lenses that are the same size (let's suppose they're both 50mm diameter), the effect of a given aperture diaphragm on either lens will be the same: Stopping it down will darken the final image at the far right, projected by the right lens.

Now consider the effect of having real aperture diaphragms on both the left and right lenses, independently adjustable. If we stop the right lens down to only 10mm diameter, for example, all of the light from the left lens outside of its central 10mm will be rejected. In this situation, placing a real aperture on the left lens that is larger than 10mm will have no effect, as it's just blocking light that was already blocked at the right lens diaphragm. Thus we will see no effect from the left-lens diaphragm until it is reduced to less than 10mm diameter. Any larger diameters will not change the brightness of the image projected by the right lens.

Making a Couple Adjustments

In practical AF systems, the right lens - known as a separator lens - is quite small. Let's make our system more representative by changing the right lens to 6mm diameter. This is the same as placing a 6mm-wide aperture diaphragm in front of the former large lens, so the separator lens will only receive light from the central 6mm of the left lens. Now the brightness of the image projected by the 6mm separator lens will be much less than from the previous 50mm lens - but it will not be darkened further unless the diaphragm on the left lens is reduced to less than 6mm. The small separator lens also has a much shorter focal length, projecting a smaller image.

Another change we need to make, is to offset the separator lens from the optical axis, and add a second separator lens diametrically opposite to it. Let's offset these separators 9mm from the optical axis; then their circles will span from 6-12mm away from the optical axis. We also add a mask in front of the separators, to eliminate flare problems from rays that do not enter the separators.

Each separator will receive light from the left lens, across corresponding 6mm circles, also offset 9mm from the optical axis (since our system currently has the field lens centered). This means that setting any aperture diameter on the left lens that is 24mm or more, will not block any of the light reaching the separators. If we stop the left lens down to less than 24mm, the images projected by the separators will start to darken, and when the left-lens aperture reaches 12mm or less, the separators will receive no light at all.

In this diagram, the images of the separator lenses on the left lens (shown in gray) are the only areas that rays can pass through, and reach the separator lenses (rays shown solid). Other rays (dotted lines), not passing through the separator lens images, will miss the separator lenses at the right. The aperture diaphragm on the left lens is shown at nearly the narrowest setting that will not block light rays to the separators; if it is opened up more, it will admit more rays, but they will miss the separators:

In Nikon's AF systems, the separator-lens images are set just inside the f/5.6 circle.  This diagram shows why lenses with maximum apertures larger than f/5.6, are not able to send more light through the separator lenses, to the AF detector, than an f/5.6 lens can.

-- hide signature --

Qualities possessed by God in infinite proportion: Love, Grace, Power, Righteousness, Wisdom, . . .
Qualities possessed by humans in infinite proportion: Ignorance.
- Marianne

Carerra
Carerra Regular Member • Posts: 361
Re: Step 3: Virtual Apertures
1

This is really nice of you to have taken then time to do this for us!!

-- hide signature --

www.louisamore.com

 Carerra's gear list:Carerra's gear list
Nikon D850 Nikon AF-S Nikkor 50mm f/1.4G Nikon 85mm F1.8G Nikon AF-S Nikkor 18-35mm f/3.5-4.5G ED Tamron SP 150-600mm F/5-6.3 Di VC USD +1 more
digital ed
digital ed Veteran Member • Posts: 3,539
Thank You
1

I am looking forward to future episodes.

-- hide signature --

"There is a little of not done yet in all of us."
John Madden, football coach

 digital ed's gear list:digital ed's gear list
Olympus E-1 Olympus E-3 Nikon 1 V2 Nikon D810 Nikon AF-S Nikkor 14-24mm f/2.8G ED +15 more
Osvaldo Cristo
Osvaldo Cristo Veteran Member • Posts: 4,346
Brava, maestrina!
4

Marianne, you are a great teacher. Thanks for your time preparing that material. Cristal clear.

Regards,

-- hide signature --

O.Cristo - An Amateur Photographer
Opinions of men are almost as various as their faces - so many men so many minds. B. Franklin

 Osvaldo Cristo's gear list:Osvaldo Cristo's gear list
Olympus Stylus 1030 SW Nikon D200 Nikon D300S Nikon D810 Nikon AF DX Fisheye-Nikkor 10.5mm f/2.8G ED +32 more
OP Marianne Oelund Veteran Member • Posts: 7,274
Lessons from the Real Optics Model
11

As mentioned in the opening post, I have been using a real model alongside the theoretical analyses, primarily as a means of confirmation. It is also valuable as a demonstrator, and since I'm rather tired of producing diagrams, I thought I'd use some photos instead.

Here is the real optics model:

AF optics model: Three lenses plus projection screen

At the left is the AIs 105mm f/2.5, serving as the main imaging lens (aka "left lens" in the theoretical diagrams). The field lens in the middle is the AIs 50mm f/1.8, and the separator lens is the AIs 28mm f/2.8. As described earlier, the spacing between the main and separator lenses is 4f or 200mm (f is the field lens focal length, 50mm). The separator lens rests on a wooden cradle attached to a lateral micro-adjust slide, so I can set it to precise lateral displacements.

Virtual Apertures, Revisited: Subject Masking

We again apply the concept of virtual apertures, this time to the image planes of the left and right lenses instead of the image planes of the field lens.  An aperture or mask at any one of these locations effectively "crops" the subject down, which helps to keep the images projected by the separator lenses from overlapping or producing flare.

Placing an aperture mask at any one of those 3 planes will effectively mask the other two as well, as discussed previously for the field lens object and image planes. It is usually most convenient to place this mask in front of the field lens. In the real model, we simply need to stop the field lens down. In practical AF systems with regular arrays of AF points, a rectangular mask is often desirable.

It's important to understand that adding this subject mask to the system does not darken the images projected by the separators; it just eclipses (crops) those images so that they cover a smaller area.

In this series of examples from the real model, we see the reducing aperture of the field lens cropping the AF detector's view of the subject (and you can clearly see the shape of the AIs 50mm's diaphragm opening).  These are photos of the projection screen on the model, which simulates the surface of the AF detector:

Cropping of detector image by field-lens aperture - sequence covers f/1.8 to f/5.6 settings

Apologies for the comical subject figure (no, it is not intended as a self-portrait).  You can see why I went to engineering school rather than art school.  Ideally, the screen would be black instead of gray, but the image is not very bright, and there is enough light leakage in the room for the camera to pick it up easily.

In the next post, we will use the real model to study the effect of reducing the main lens aperture, and look at how adjusting focus of the main lens shifts the AF-detector image.

-- hide signature --

Qualities possessed by God in infinite proportion: Love, Grace, Power, Righteousness, Wisdom, . . .
Qualities possessed by humans in infinite proportion: Ignorance.
- Marianne

MisterHairy Senior Member • Posts: 2,255
Re: Lessons from the Real Optics Model
4

Excellent demonstration. I hope that you are getting your work peer reviewed by Leonard before posting, just to check for errors of logic or understanding.

 MisterHairy's gear list:MisterHairy's gear list
Fujifilm X100F Sony Alpha a7R III Sony a7 III Sony FE 35mm F2.8 Sony FE 55mm F1.8 +7 more
OP Marianne Oelund Veteran Member • Posts: 7,274
Real Optics Model part 2
7

For this demonstration, the model has been set up with the separator lens aperture diameter at 2.6mm (set f/11 on aperture ring), and the lateral shift has been set to 5mm. This places the separator-lens image just inside the f/5.6 circle at the main lens, as is standard in Nikon AF systems.

By taking a series of photos of the image projected on the screen (AF detector), as the main lens aperture is adjusted, we see when light starts to be reduced for the AF detector. This sequence starts at f/2.8:

We see clearly, that there is no change in detector-image brightness until the main lens is stopped down past f/5.6. At f/8, it is noticeably dimmer, and at f/11 it is no longer visible at all since the main lens aperture has completely covered the separator-lens circle in the main lens exit pupil.

AF System Effective Aperture

Since the AF detectors are receiving light through a fairly small circle on the main lens exit pupil, the effective aperture of the AF system is quite narrow, producing a high value for the focal ratio (f-stop). Although this makes the AF-detector image rather dim, it also has the benefit of yielding a high DOF or depth of focus for the AF sensors, which helps in determining focus errors when the main lens is far out of focus.

When a lens is focused at infinity, its focal ratio is given by f/d, where f is the focal length and d is the physical diameter of the lens entrance pupil. More generally, the focal ratio is effectively the lens-to-image distance divided by the lens entrance pupil diameter. Referring to the "Virtual Separator Lens" diagram posted earlier, we see that the lens-to-image distance for the separator lens can be taken as the distance from main lens to field lens, and its entrance pupil diameter is the diameter of the separator lens image circle on the main lens. For example, for the real optics model, the lens-to-image distance is 100mm and the separator-lens diameter is 2.6mm, giving a focal ratio of 38 - very high!

For commercial AF modules, we find that effective focal ratios run from about f/22 to f/32, for AF sensors that are set to the f/5.6 circle of the main lens (AF sensors set up for f/4 or f/2.8 can have "faster," or brighter, focal ratios).

Measuring Focus Error

We now turn to the ultimate goal of all of this optics discussion: How the optics give us a measure of focus error.

Forcing the separator lenses to view the subject from two different points on the main lens, which are offset from the optical axis, gives them an angled view of the subject - just as human binocular vision has. Because of the angled pathways, a change in the main lens focus setting produces a shift in the position of the two AF-detector images, towards or away from each other. For additional description of this, and experiments you can perform yourself, see the series of posts starting with

http://www.dpreview.com/forums/post/29789486

As the main lens is focused closer, the two AF-detector images move slightly closer together, or conversely as the main lens is focused towards infinity, the AF-detector images move further apart.  The real optics model only has one separator lens due to its large size, so we can only observe a single AF-detector image at a time.

The shift produced by changing main lens focus is surprisingly small.  Fortunately, it helps that the subject masking (field lens aperture) gives us a reference position that does not move; we can compare the image to its fixed boundary.  One additional complication with the model, is that there is also a noticeable change in magnification as the lens focus is changed from infinity to closest-focus, so it's best to look at the central point (nose) to see the movement.  To help make the change easier to see, I have set the separator lens lateral position to use the main lens f/2.8 circle:

As the projected image in this example is only around 4mm wide, we see that the image shift is even less than 1mm - not much for a 105mm lens changing from infinity to 1m focus.  One can imagine how small the shifts are for f/5.6 AF sensors when wide-angle lenses are used.

As a final point, note that in spite of the large focus change for the main lens, the AF-detector image does not go very far out of focus.  This is a good demonstration of the advantage of the high focal ratio for the AF optics.

-- hide signature --

Qualities possessed by God in infinite proportion: Love, Grace, Power, Righteousness, Wisdom, . . .
Qualities possessed by humans in infinite proportion: Ignorance.
- Marianne

M Lammerse
M Lammerse Forum Pro • Posts: 11,425
Re: Autofocus System Design

Thanks Marianne for this learningful post!

Michel

OP Marianne Oelund Veteran Member • Posts: 7,274
Re: Lessons from the Real Optics Model
3

MisterHairy wrote:

Excellent demonstration. I hope that you are getting your work peer reviewed by Leonard before posting, just to check for errors of logic or understanding.

Leonard =  Peer?

No need to check with Leonard, in any case.  I already know what he would say - unless he is somehow able to approach the discussion with a more open mind.

-- hide signature --

Qualities possessed by God in infinite proportion: Love, Grace, Power, Righteousness, Wisdom, . . .
Qualities possessed by humans in infinite proportion: Ignorance.
- Marianne

MisterHairy Senior Member • Posts: 2,255
Re: Lessons from the Real Optics Model
4

It was a joke. Particularly the "peer" part.

 MisterHairy's gear list:MisterHairy's gear list
Fujifilm X100F Sony Alpha a7R III Sony a7 III Sony FE 35mm F2.8 Sony FE 55mm F1.8 +7 more
OP Marianne Oelund Veteran Member • Posts: 7,274
I should give Leonard a little credit
7

MisterHairy wrote:

It was a joke. Particularly the "peer" part.

Of course - my response would have been rather more passionate, had I thought otherwise. 

We do need to recognize, though, that indirectly, Leonard has helped to bring this thread into existence.  If it were not for his incessant bumbling and drawing of my attention to AF system details, I wouldn't have come up with the ideas that went into the presentations here.

We can also give Leonard credit for offering sound advice on a range of other photographic topics; I just wish he would leave AF topics alone!

There are still more posts to come, and I am also trying to obtain an AF module from a camera to set up and study (if nothing else, I can have my oldest D2Hs camera, which has some internal contamination/corrosion, dismantled to provide a module).

-- hide signature --

Qualities possessed by God in infinite proportion: Love, Grace, Power, Righteousness, Wisdom, . . .
Qualities possessed by humans in infinite proportion: Ignorance.
- Marianne

MisterHairy Senior Member • Posts: 2,255
Re: I should give Leonard a little credit
9

I know. He has been a great help. Much the same as cows have helped the development of fences in the UK.

 MisterHairy's gear list:MisterHairy's gear list
Fujifilm X100F Sony Alpha a7R III Sony a7 III Sony FE 35mm F2.8 Sony FE 55mm F1.8 +7 more
Brandon birder Veteran Member • Posts: 4,135
Re: Step 3: Virtual Apertures
1

So well put. I have learned a lot and looking forward to more.

Thanks,

Brandon

fromero Contributing Member • Posts: 918
Re: Autofocus System Design
1

Thanks so much for your series of explanations on AF in such clear pedagogic style.

I think they deserve to be compiled in an article in order to get an easy access in the future.

It has brought memories of informative discussions held in years past in these fora with which I learnt about technical aspects of photography and equipment. Now they are rare.

Rgds

-- hide signature --

Francisco Romero

Bernard Delley Senior Member • Posts: 1,032
thanks for this stimulating thread !

I feel the urge to jot some back of envelope estimates about the noise to be expected in the AF module data and its consequences on PAF accuracy.

At quick intuition, the limits might come out similarly for a Nikon 1 style, on sensor, AF system, where the virtual apertures are implemented via the micro prisms over the AF pixels.

Leo360 Senior Member • Posts: 1,141
Re: Real Optics Model part 2

Marianne Oelund wrote:

As a final point, note that in spite of the large focus change for the main lens, the AF-detector image does not go very far out of focus. This is a good demonstration of the advantage of the high focal ratio for the AF optics.
- Marianne

Marianne,

thanks a lot for an illuminating series of posts on AF inner-workings.

Please, correct me if I am wrong, but I think that the smaller aperture of the AF optics is a disadvantage. If AF operates at f/5.6 while the actual shot is taken wide open at f/2.8 the depth of field as seen by AF is greater then DOF seen by the main sensor. Because of this AF optics is less sensitive to focusing error than the main sensor. For 36MP sensors in D800 and D810 it might cause some loss of sharpness.

-- hide signature --

Leo--

 Leo360's gear list:Leo360's gear list
Nikon D5100 Nikon D750 Nikon AF-S DX Nikkor 18-200mm f/3.5-5.6G ED VR II Nikon AF-S Nikkor 24-120mm f/4G ED VR Nikon AF-S Nikkor 50mm f/1.8G
nikonuserinfo Regular Member • Posts: 278
Thanks. Probably best Dpreview topic ever.

nt

Kindest regards,

Stany
http://www.nikonuser.info
My former Dpreview profiles ( For which I do not have the Original email address anymore)
Stany Buyle
Photoforfun

 nikonuserinfo's gear list:nikonuserinfo's gear list
Nikon D1 Nikon D1H Nikon D2H Nikon D200 Nikon D810 +1 more
Keyboard shortcuts:
FForum MMy threads