Nikon Zf AF not state of the art

Hi Marcel,

the golden middle has been used ("3"). To be honest, at least according to the decsription and basic algorithm design principles it should not matter, as there is nothing moving in between subject & camera and actual subject detection has never been lost.

Quote from the manual:
Choose how quickly focus responds if something passes between the subject and the camera when AF‑C is selected for the focus mode.

Furthermore (I didn't mention this), I use Focus priority for single shots and Focus + Release for burst shots, as the "Focus" settings for burst can bring the cameras down to a crawl (at least the Z6/Z7, I have not enough experience with burst shots on the Zf yet).
I just checked the timestamps and all of my example are single shots and not bursts (typically something like 1-3fps was being used by manual activation of the shutter) (whereas my burst setting is 5fps). All of the samples shown should have used "Focus Priority" accordingly.

Laqup said:

the golden middle has been used ("3"). To be honest, at least according to the decsription and basic algorithm design principles it should not matter, as there is nothing moving in between subject & camera and actual subject detection has never been lost.

Click to expand...

I'm going to have to go to the replay booth on this one ;~). One thing I believe I noticed in your examples—hard to tell for sure because of the way you presented them—is that the camera is indeed moving. I've written this before—and to some degree it's one of the things that is a bit different between brands—but handling discipline informs AF on the Nikons, for sure.

The position of the focus sensor boxes is the first sign that is telling me that. Locked down on a tripod on a static subject, the box won't move. Handle the camera loosely, and particularly with a subject that might move some, and the box will drift off. Worse still, the box is not where focus was achieved. The reason for that has to do with the timing of recording the box position. If I put the camera on something like a paint shaker, the box is never on the actual focus position, because the focus continues during the movement while the camera is trying to record the focus position. In other words, Nikon has chosen to put "focus box position" with the lowest priority in the chain of things it's doing while focusing.
But again, I'd need to see the actual replay of your photography session to make a more informed comment.

Finally, as someone else noted, the Zf lags the Z8/Z9 in autofocus performance some. I'm pretty sure that has to do with the way the viewfinder stream is done on the two different systems, and the speed at which the system is getting data. There's no lag in focus data stream on a Z8/Z9, while there can be on the Zf.

Thom Hogan said:

Laqup said:

the golden middle has been used ("3"). To be honest, at least according to the decsription and basic algorithm design principles it should not matter, as there is nothing moving in between subject & camera and actual subject detection has never been lost.

Click to expand...

I'm going to have to go to the replay booth on this one ;~). One thing I believe I noticed in your examples—hard to tell for sure because of the way you presented them—is that the camera is indeed moving. I've written this before—and to some degree it's one of the things that is a bit different between brands—but handling discipline informs AF on the Nikons, for sure.

Click to expand...

Maybe I phrased that wrong, with moving I actually meant "passing". The camera has definitively moved and the subject as well. I mostly was sitting / kneeing somewhat stationary, but there is always some movement. This is what I hoped would be tackeled by AF-C and subject detection (aka constant refocusing or at least focusing on shutter half press). With Canon it works that way.

But: As far as I understand the option "A3", blocked shot is not related to camera or subject movement but rather to objects that PASS in the line of sight between camera and subject. In my examples there was nothign passing in between but I always had a "unobstructed" line of sight. Please correct me if I understand the block shot option wrong.
My interpretation is, that in case of "5" (aka delayed), a passing object would not lead to an immediate switch of the subject, the camera will rather try to "predict" / "estimate" / "guess" where the previous target was, where a setting of "1" (quick) would immediately lead to a switch of the subject from original to the one passing in between.
I am an algorithm developer in the field of computer vision and this is a typical parameter for subject tracking, where the implemented filter ("predict / update" cycle) in case of occlusion / missing measurement would stay longer in the "prediction" mode w/o increasing the underlying variances that allow measurement association or alternatively w/o deleting the track.

In my world to allow a focus switch from the eye to the hand (if the hand is being interpreted as a passing object) would first have to be triggered by a loss of the actual subject/eye detection, which at least was not indicated in the GUI.
If the camera is performing a full area auto AF point selection parallel to subject detection and then even grants priority to this "hidden auto AF point selection" even if the original subject is still present and has a valid measurement this is a rather weird algorithm implementation. Maybe I should apply at Nikon to get things finally right then. ;-)

Here a typical example where I assumed "blocked shot" would be relevant:



In image #14 the person is detected and properly tracked, in image #72 the object is occluded by a "pseudo passing object". If blocked shot is set to quick, the track would be deleted very fast and either the lamp post or the second person would be selected as new subject. With blocked shot set to "delayed" the algo would predict were the person went and "wait" a few frames before changing focus distance / deleting the object or selecting a new subject.
I am not sure if the subject / eye detection is actually being tracked at all though, to me it looks more like a "single cycle" event, maybe with a very short prediction window. 3D tracking is a different beast though.

Laqup said:

But: As far as I understand the option "A3", blocked shot is not related to camera or subject movement but rather to objects that PASS in the line of sight between camera and subject.

Click to expand...

That's basically correct for most situations. There is a scenario when your subject is so small (e.g. distant, small bird) whereby the camera may decide that the background is "the subject" and the "small object" is the thing that it should consider is blocking. Z8/Z9 firmware updates tended to "fix" that. But it doesn't apply to the situation you illustrated.

Laqup said:

My interpretation is, that in case of "5" (aka delayed), a passing object would not lead to an immediate switch of the subject, the camera will rather try to "predict" / "estimate" / "guess" where the previous target was, where a setting of "1" (quick) would immediately lead to a switch of the subject from original to the one passing in between.

Click to expand...

Let me correct the wording here some. Nikon's wording isn't exactly clear, nor are some of their descriptions of what it does. Moreover, CSM #A3 doesn't apply in some situations. The set value is basically ignored in 3D-tracking, and in Auto-area AF the values of 1 and 2 are ignored.

An object that passes in front of the recognized focus subject in Subject Detection will trigger the blocked shot response on the delay value you set (outside of what I just mentioned). But it's not about predicting, it's about "instantly change subject and thus change focus." If #A3 is delaying a change of focus, focus is kept where it was last made.

Laqup said:

Maybe I should apply at Nikon to get things finally right then. ;-)

Click to expand...

Nikon is trying to be consistent with what they've done for over a decade. One simplification on the Zf is that it doesn't have the Steady/Erratic subject motion option, and this could be an issue if your camera handling is shaky.

Thom Hogan said:

Nikon is trying to be consistent with what they've done for over a decade. One simplification on the Zf is that it doesn't have the Steady/Erratic subject motion option, and this could be an issue if your camera handling is shaky.

Click to expand...

Probably a naive question here but doesn’t IBIS at least help in this situation? I suppose though when not using tracking the focus point has to move if the camera moves.

CliveTJ said:

Thom Hogan said:

Nikon is trying to be consistent with what they've done for over a decade. One simplification on the Zf is that it doesn't have the Steady/Erratic subject motion option, and this could be an issue if your camera handling is shaky.

Click to expand...

Probably a naive question here but doesn’t IBIS at least help in this situation? I suppose though when not using tracking the focus point has to move if the camera moves.

Click to expand...

It does and it doesn't. Remember, with VR set to Normal the VR system recenters, which can mean that if the camera is moving much the viewfinder (and thus the focus stream) jumps.

Thom makes some very good points. Nikon AF also has a strong closest subject and high contrast bias.

• I would have used AF-C Wide (L People) release priority and burst 3-5fps. Bursting improved my ML results, and I wish I had done more bursting with my D500 as well.
  
  
• I also would use 3D AF-C with the subject selected "marked" as it is intended to be used.
  
  
• I found Z8/ZF face detection to be less detective than the II's, but better when it did detect. You can experiment with it using photos, TV, and other videos. This way you know what to expect.
  
  
• Your subjects were backlit and the eyes were in shadows. That usually means spot focus and metering, and possibly even EVF exposure off, as recommended for flash and studio photography.
  
  
• With F4 glass I don't need eye-level detection, and I wish Nikon offered a simpler face only detection option as the Z5/6/7 offered. This would also make it easier to switch between subjects.

As you're experience it is better than the Expeed6 bodies, but not quite up to the Z8/Z9 experience. Competitors $2,000 systems are less restricted. I feel like the ZF is more of what the Z5 should have been. I also feel that the ZF/Z8 are a bit over priced, but the competition's flagships are still more expensive with other drawbacks.

For enthusiast/family shooters there is still up for grabs. The right DX body could grab these users, and the Sony/Canon APS-C releases didn't have updated sensors.

Laqup said:

Leonard Shepherd said:

Laqup said:

forgot to mention: I did not use zones, but my default AF mode is full frame subject detection AF.

Click to expand...

AF benefits from input from the photographer - with any brand of camera.

Full frame AF is about as inappropriate an AF starting point as can be selected when wanting AF to detect a subject occupying a very small percentage of the image area, similar to your examples.

Click to expand...

And here ladies and gentleman we have him. ("I will not go into all details of the settings, as I know that someone always has "a better idea"). Congrats to you, sir.

So how would you explain subject detection working flawlessly if this is the "most inappropiate mode ever"? Subject has always been detected and eye was selected (opposite to what you actually stated, you made it sound like subject detection did not work), it's only the AF system that failed to achieve focus on the selected eye.
Just for double clarification: You can see the selected focus point in my examples, it's the little red box in the vicinity of the eye.. Why do you think that "additional input from the photographer" is needed in that case?

And why do you think a clearly defined motive that covers like 30-40% pixels in the frame can be described as "very small percentage"? Care to elaborate?

Maybe you didn't understand what I was showing, this is a single, uncropped photo, I marked the subject for you:

Click to expand...

What I would like to see is comparison between Zf's AF and say comparably priced Sony's and/or Canon's AF in similar as possible situations. I know, hard to do without a robotic kid inside a building with consistent lighting.

Always frustrating to read where someone comes up with "you should have used this setting" or "you are not holding camera steady" or "shutter speed is too slow" as a reason for poor AF hit/miss rate; when other brands manage to have better AF hit/miss rate.

However, proof is needed.

Laqup said:

Leonard Shepherd said:

Laqup said:

forgot to mention: I did not use zones, but my default AF mode is full frame subject detection AF.

Click to expand...

AF benefits from input from the photographer - with any brand of camera.

Full frame AF is about as inappropriate an AF starting point as can be selected when wanting AF to detect a subject occupying a very small percentage of the image area, similar to your examples.

Click to expand...

And here ladies and gentleman we have him. ("I will not go into all details of the settings, as I know that someone always has "a better idea"). Congrats to you, sir.

Click to expand...

Here some go once again.

Used in a reasonable way camera AF in all cameras works well.

Some I find fail to recognise the ability of the person behind the viewfinder has a lot to do with how cameras performs.

Most get excellent results, some do not.

I have found over the years those who get poor results have a camera that can instantly take good photos - used reasonably in capable hands.

--
Leonard Shepherd
In lots of ways good photography is similar to learning to play a piano - it takes practice to develop skill in either activity.

I would say that for those who really care more than anything else about completely effortless, as close to always perfect intelligent focus as possible, Nikon may be the wrong brand. Just from reading comments by posters in this forum who shoot multiple brands, it sounds as if Nikon remains a bit behind in this regard. My Nikon bodies work great for me and how I shoot, including under what I consider challenging circumstances. They are fantastic cameras and I make more mistakes than they do. But others have different expectations.

j_photo said:

But others have different expectations.

Click to expand...

+1.

In some ways Nikon Expeed 7 body AF is ahead of most of the competition - though I find it benefits from reasonable consideration of the various AF options available to get the best performance from it.

Leonard Shepherd said:

j_photo said:

But others have different expectations.

Click to expand...

+1.

In some ways Nikon Expeed 7 body AF is ahead of most of the competition - though I find it benefits from reasonable consideration of the various AF options available to get the best performance from it.

Click to expand...

+1

I find the Zf both enjoyable and reliable.

I now primarily use Sony Alpha full-frame cameras after many active years in Nikon D/SLR universe. Although I have some comparative observations based on practical experience, I use the Zf as it is.

My personal guideline: the first 500+ images are for learning about and getting familiar with a new camera or lens, in various scenarios and light conditions. Any “weird” results during that period are most likely due to my insufficient understanding and experience.

Also, I am conscious of “photo bombs” of all kinds, not only in front of the camera but also behind it. So I always remind myself: one capture means zero choices.

Leonard Shepherd said:

j_photo said:

But others have different expectations.

Click to expand...

+1.

In some ways Nikon Expeed 7 body AF is ahead of most of the competition - though I find it benefits from reasonable consideration of the various AF options available to get the best performance from it.

Click to expand...

I’d hope the software would stay latched onto an eye though typically I’d be realistic if an eye is in dim light. Personally I’d never want to use a very shallow DoF which only has an eye in focus and is grainy to boot. I’m therefore unlikely to hit these issues as I’d want a whole face in focus. I know I’m not that demanding a photographer.

Also though, I’m not denying the OP is finding issues vs other brands.

The best demo of the Zf's Eye AF is shown in a recent Jared Polin YouTube video.

As they say, he "brings the receipts." He shows an extensive Zf Eye AF session with very good results, but then the typical Nikon M/L behavior rears its head--the viewfinder shows the eye in-focus but the image shows focus somewhere else.

Leaps and bounds better than the Z6 II (I speak from extensive experience with that body) but still not up to the competition.

--

I don't feel the need to "defend" Nikon when genuine problems are credibly demonstrated. I don't reflexively claim that "user error" or lack of "skill" are to blame.

Leonard Shepherd said:

Here some go once again.

Used in a reasonable way camera AF in all cameras works well.

Some I find fail to recognise the ability of the person behind the viewfinder has a lot to do with how cameras performs.

Most get excellent results, some do not.

I have found over the years those who get poor results have a camera that can instantly take good photos - used reasonably in capable hands.

Click to expand...

You don't have to imply so much. If you want you can call me outright incompetent. I don't care, won't report anyone, especially not the master of all Nikon AF modi.

As I have mentioned before I am a Canon shooter as well and I have some Canon sequences from the same fella in the same location as well. I was not aware of the Zf AF issues during the time of catching those photos, therefore I don't have exactly the same setup and thus it will remain comparing apples to pears (which I am fully aware of).
The Canon shots were done with the R8 and the RF 50mm 1.2 (fully open) and slightly slower shutter speeds, which leads to much more light on the sensor and lower ISO values. Neverthless focusing at f1.2 is a diffrent challenge as well. Overall apples to pears but not apples to bananas.

Example 1: Only slow movements, this is actually a burst (I think it was 10fps):


Example 1, I included the hand and the playdoll to show how focus falloff is with the 1.2 lens.

And here just the 100% crop of the eye (similar focus setup, complete frame auto subject detection, servo AF (AF-C).


Example 1, 100% crops

In the whole sequence only two images are slightly out of focus and as I have seen no real sharp areas anywhere else this is probably not a focus issue but actual motion blur. In the last photos the AF jumped from the right to the left eye. During the sequence he actually lifted his hand and pointed at me (foreground object).



Example 2: More subject movement involved and especially a lot of foreground where the AF could potentially latch on.


Example 2: He is grabbing his doll, this is the last image of the sequence shown below

And here the 100% crops, again RF 50mm 1.2 at f/1.2:



Again, out of those images none is on the ear, none is on the nose, none is on the foreground. One is slightly out of focus and 1-2 are not critically sharp. Canon just can.

And this is what slightly shocked me when comparing the results from the Zf to the R8 as the scenarios are even more demanding in the Canon case (face only partially visible as you can see in the Example 2 overview shot).

And btw I have other sequences with other kids as well that exhibit the exact same behavior, but feel free to pick your conclusion:

1. I am fully incompetent
2. I am not compatible with Nikon
3. Nikon is not for convenience shooting
4. Nikon has some catching up to do with the Zf and Z6 III and expeed 7 alone is not the solution (a fast sensor is probably needed)

I really wanted the Zf to be the breakthrough for Nikon with their "lower end" bodies, this is why I bought one even though I hate the ergonomics. I just wanted to be able to use the fantastic glas with a capable body (the RF 50mm 1.8 is a joke for example) + I really don't want to bring the better RF glas with me during vacation -D).
As MJ_79 already mentioned there is a lot hype going on surrounding the Zf but so far I can't see this hype reflected in the images I actually get. The R6II/R8 plattform on the other hand is fantastic.

And one additional disclaimer: The actual RAW images / PNG exports when directly opened from my local filesystem showcase the differences between "critically sharp" / "sharp" and "out of focus" better than the uploads. The compression applied on this plattform degrades image quality quite a bit and evens out some of the differences. The same is of course true for both Nikon and Canon examples.

j_photo said:

I would say that for those who really care more than anything else about completely effortless, as close to always perfect intelligent focus as possible, Nikon may be the wrong brand. Just from reading comments by posters in this forum who shoot multiple brands, it sounds as if Nikon remains a bit behind in this regard. My Nikon bodies work great for me and how I shoot, including under what I consider challenging circumstances. They are fantastic cameras and I make more mistakes than they do. But others have different expectations.

Click to expand...

Yes, at least presently Nikon's competitors seem to be a bit better, at least in remembered experience, not direct comparison, because nobody but a well-funded corporate test lab is going to have an AF test bench for erratic targets tracked hand-held...if any do.
Nikon is time-wise 5 years behind Sony and Canon in the development of its large-format AF/targeting systems. It doesn't seem to be technically that lagging.

EDIT: the Canon shots just posted are pretty amazing.

I guess my question to the OP is: given your technical expertise in this area (computer vision and subject recognition), do you have any insight into why Nikon's architecture lags the competition except in their flagship level cameras?
Thom has already mentioned that the Zf and nonstacked sensor Zs' data stream is not as dense and consistent as that of the Z8 and Z9. Is there something about Nikon's algorithms that is fussier about data consistency than Sony's? Or is just that there's still not enough data density than Sony and Canon might provide for the Expeed 7 chip to chew on?
Canon has had DPAF for over a decade now - the AF data stream can be very dense, with every pixel potentially an AF pixel. Both Sony and Nikon use sparse AF arrays. But Sony seems to stick harder.
I find it a bit hard to believe that Nikon's architecture is inherently compromised. But both Nikon's sensors and algorithms may be a bit behind some competitor's products. This is a good sign, it means that improvements are still possible. Hopefully, with the new sensors coming it means another period of frequent firmware updates that finally will let Nikon equal and perhaps pull ahead.
But still...AF is very good across the board. Is the complete Z experience now good enough for most photographers, or must it be the best across the board, especially for AF, before we are happy?

mosswings said:

I guess my question to the OP is: given your technical expertise in this area (computer vision and subject recognition), do you have any insight into why Nikon's architecture lags the competition except in their flagship level cameras?

Click to expand...

Unfortunately I am not an expert for the actual focusing algorithms and I have no insight into the Nikon implementation at all.

Just some observations:
The subject detection is very solid given the processing / energy / heat constraints, no complaints here, well done. Furthermore I outright adore the "generic object tracking mode" of the Z6 / Z7 (II) (this small yellow rectangle you can activate by pressing FN1 (if bound correctly)). Here Nikon definitevly has implemented some kind of template matching algorithm that actually stores some of the features of the selected image region, as it is possible to completely leave the FoV and nevertheless get a track after entering back again. This is was probably the coolest thing I had seen so far from any of the manufacturers. 3D tracking is nice as well of course.

In terms of subject detection the Z6 and Z7 were well ahead of Canon with their EOS R (which I also owned and even with the latest Canon firmware). Actual AF speed was on the same level. But of course both were trailing behind Sony in both aspects.

Compared to Canon, even today Nikon is miles ahead when it comes to "generic object tracking" (aka tracking an object that is not covered by subject detection). Canon absolutely fails at this.
But if an object can be detected Canon has a better coupling of their subject detection with the actual AF module and a slightly better subject detection overall too (it seems to me that Canon can pick up slightly "smaller" subjects (eyes, birds, faces, vehicles, ..).

But to at least try to answer your question (with some guesses):

- Coupling the subject detection module to the AF module and overall focus prioritization needs a major overhaul (even if this is not in line with how the AF has behaved in the past as mentioned by Thom)
- Potentially a fast sensor ("readout speed") and a fast data pipeline with short processing cycle times is necessary to get the most out of the current AF implementation
- And of course the points you already mentioned (how many focus points, how distributed, how to actually achieve focus..) play a certain role too
- Maybe the actual communication protocol between body and lens is a limiting factor as well (I have no insights here, but it would actually suprise me), but it would be good to know if e.g. "update rates" of lens commands are faster in the RF / E protocol than in Z (or at least let's say between R5/R6 II and Zf). If I would have to guess this is probably coupled to the internal processing cycle of the AF module and somewhat "dynamic" (or available in certain discrete steps, e.g. 20Hz, 40Hz, 50Hz, 100Hz) and not really a fixed frequency, but who knows.. (not me )

HRS said:

The best demo of the Zf's Eye AF is shown in a recent Jared Polin YouTube video.

As they say, he "brings the receipts." He shows an extensive Zf Eye AF session with very good results, but then the typical Nikon M/L behavior rears its head--the viewfinder shows the eye in-focus but the image shows focus somewhere else.

Click to expand...

The usual reason that happens is down to the VR system used in "normal" mode. The usual fix: switch the VR to "Sport" mode. I don't have a ZF but that works on any of my Nikons (Z50, Z9 etc).

Laqup said:

mosswings said:

I guess my question to the OP is: given your technical expertise in this area (computer vision and subject recognition), do you have any insight into why Nikon's architecture lags the competition except in their flagship level cameras?

Click to expand...

Unfortunately I am not an expert for the actual focusing algorithms and I have no insight into the Nikon implementation at all.

Just some observations:
The subject detection is very solid given the processing / energy / heat constraints, no complaints here, well done. Furthermore I outright adore the "generic object tracking mode" of the Z6 / Z7 (II) (this small yellow rectangle you can activate by pressing FN1 (if bound correctly)). Here Nikon definitevly has implemented some kind of template matching algorithm that actually stores some of the features of the selected image region, as it is possible to completely leave the FoV and nevertheless get a track after entering back again. This is was probably the coolest thing I had seen so far from any of the manufacturers. 3D tracking is nice as well of course.
In terms of subject detection the Z6 and Z7 were well ahead of Canon with their EOS R (which I also owned and even with the latest Canon firmware). Actual AF speed was on the same level. But of course both were trailing behind Sony in both aspects.
Compared to Canon, even today Nikon is miles ahead when it comes to "generic object tracking" (aka tracking an object that is not covered by subject detection). Canon absolutely fails at this.
But if an object can be detected Canon has a better coupling of their subject detection with the actual AF module and a slightly better subject detection overall too (it seems to me that Canon can pick up slightly "smaller" subjects (eyes, birds, faces, vehicles, ..).

But to at least try to answer your question (with some guesses):

- Coupling the subject detection module to the AF module and overall focus prioritization needs a major overhaul (even if this is not in line with how the AF has behaved in the past as mentioned by Thom)
- Potentially a fast sensor ("readout speed") and a fast data pipeline with short processing cycle times is necessary to get the most out of the current AF implementation
- And of course the points you already mentioned (how many focus points, how distributed, how to actually achieve focus..) play a certain role too
- Maybe the actual communication protocol between body and lens is a limiting factor as well (I have no insights here, but it would actually suprise me), but it would be good to know if e.g. "update rates" of lens commands are faster in the RF / E protocol than in Z (or at least let's say between R5/R6 II and Zf). If I would have to guess this is probably coupled to the internal processing cycle of the AF module and somewhat "dynamic" (or available in certain discrete steps, e.g. 20Hz, 40Hz, 50Hz, 100Hz) and not really a fixed frequency, but who knows.. (not me )

Click to expand...

The little yellow box I call the "subject designator". I use it all the time because I can't stand the dancing boxes of Auto-Area-AF but want a designate-and-track functionality. This is one thing I really appreciate in the Nikon UI because it tells the camera unequivocally what you want it to track. You can then set the camera up to drop into the qualitative subject identification modes from there, but at least you are in charge of telling the camera what you're interested in, not allowing the camera to guess at what you want. It's kind of the same flow as in a Nikon DSLR's 3D-tracking mode - designate and the camera then follows, but mirrorless 3D-tracking is much more responsive. Before the Z50 had a firmware update that allowed you to drop into and out of subject-designation quickly, I was deeply disappointed in the UI, coming from the D7K series.

I know nothing about state of the art or what that is supposed to indicate. But really like what I see in the photos as it really reinforces my long standing edict that my cameras are never allowed to choose what or where to focus on. I reserve the right to do that myself and that way this does not happen to me. I never trip the shutter without first being comfortable that I have told the camera where "I" want it to focus. If it doesn't, it is simply that I did not do my job properly. I can't fuss about my camera focusing on the wrong thing.