Not all that deep since I'm far from an expert on this niche of sensor theory but perhaps still useful to those who are curious.
Theory of Operation
Most image sensors are color blind and achieve color sampling by placing colored filtered in front of each pixel. The filters are typically arranged in a repeating 2x2 RGGB pattern, with green pixels representing 50% of the total pixels and red and green 25% each. This means each pixel in a raw file has only green, red, or blue data - not all three. The monochromatic values between these are interpolated during raw processing to simulate full color images, with some inherent downsides including lower chroma sampling and demosaicing artifacts.
Pixel Shift uses multiple exposures and the sensor's IBIS mechanism to shift the position of the sensor in increments of 1 pixel for each exposure so that every "scene pixel" will be sampled by all four of the bayer color filters in the 2x2 RGGB pattern. This yields a full RGB triplet for every "scene pixel" vs the single monochromatic value obtained in a single exposure. This produces three benefits: increases the chroma (color) resolution of the image, reduces the noise of each pixel (from mean/median averaging of the multiple exposures), and eliminates demosaicing artifacts because the resulting image contains linear RGB that doesn't need to be demosaiced by the raw processor.
Here's a diagram showing how the Df moves the sensor for a 4-shot pixel shift sequence:
Nikon Df sensor movement for a 4-shot pixel shift
The sequence is:
Increased Resolution
Another technique that can be optionally employed is to shift the sensor by a fractional amount, specifically 1/2 a pixel. This can simulate a sensor with double the linear pixel density in both the horizontal and vertical axis (shifting both up/down), resulting in an image with 4x the total number of pixels. There are lots of practical factors which limit the increase in effective resolution though.
Here's a diagram showing how the Df moves the sensor for a 16-shot pixel shift sequence. Note that it still performs the chroma-enhancing 4-shift operation on top of the 1/2 pixel shift:
Nikon Df sensor movement for a 16-shot pixel shift
I generated the above diagram by analyzing the 16 raw files created by a 16-shot shift sequence and noting which direction the image shifted for each exposure and by how much, ie 1 pixel vs 1/2 sub-pixel. Here's a visual animation showing a sample of the image data I used.
Animation: Crop section of image movement for a 16-shot pixel shift
Note that the direction of sensor movement is opposite that of the image movement. For example, shifting the sensor right by one pixel actually shifts the captured image left by one pixel. There is a significant chance I messed the process up but even so hopefully you'll get the gist
Pixel Shift options and their effect on resolution and noise
The Zf offers four possible shot counts for a pixel shift:
There are additional workflow options as well, including delay to start of the sequence and the inter-exposure delay. Here is a short menu walkthrough demonstrating the options:
Sample Results
I'll now present some samples showing the benefits of pixel shifting in action. First we'll look at the reduction of demosaicing artifacts (aliasing and moiré) as well as the increase in resolved detail:
Image composite: Zf single exposure vs 4-shot vs 16-shot shift vs Z7
The above is an enlarged crop from each of the respective images, all upscaled to the resolution of the 16-shot exposure via PS Preserve Details 2.0. Note I show both capture-sharpening only samples (ie, ACR raw sharpening, set to 45/0.7/35), as well as capture sharpening + USM sharpening in PS (using 300/0.5). USM is used to demonstrate how the 16-shot pixel shift can tolerate (and actually require) additional sharpening vs the single exposure and 4-shot shift. I hope to go more into this in a later post.
I chose to use the same USM values for all samples, even though it creates obvious over-sharpening artifacts for all samples other than the 16-shot shift. I did this because using right-sized sharpening for each sample runs the risk of becoming subjective. Still, I plan to do some more nuanced sharpening comparisons in a future post.
When comparing the images, try to find areas of text in the chart that are unreadable in the single exposure but readable in the pixel-shifted images. And of course compare that to the native higher resolution of the Z7 sample. Also note the reduction/elimination of aliasing/false detail.
Here are animations of the same crops to help in visualizing the differences. Unfortunately the Z7 sample is slightly rotated - I chose not to correct that in post for this comparison since doing so would affect its acuity.
Animation: Zf single exposure vs 4-shot vs 16-shot shift vs Z7, capture-sharpening only
Animation: Zf single exposure vs 4-shot vs 16-shot shift vs Z7, capture+USM sharpening
Now let's take a look at the noise improvement, with a scene shot at -5EV exposure and raised in post to make the noise differences more obvious:
Animation: Zf single exposure vs 4-shot vs 8-shot, -5EV exposure
There's a significant decrease in visible noise between the single exposure and 4-shot in the above animation. There's actually a similar noise reduction between the 4-shot and 16-shot but it's less perceptible. Special thanks to SrMi for tips on looking into the noise improvements.
I hope to shoot some additional test scenes, preferably some with actual color to better highlight the increase in chroma resolution.
Theory of Operation
Most image sensors are color blind and achieve color sampling by placing colored filtered in front of each pixel. The filters are typically arranged in a repeating 2x2 RGGB pattern, with green pixels representing 50% of the total pixels and red and green 25% each. This means each pixel in a raw file has only green, red, or blue data - not all three. The monochromatic values between these are interpolated during raw processing to simulate full color images, with some inherent downsides including lower chroma sampling and demosaicing artifacts.
Pixel Shift uses multiple exposures and the sensor's IBIS mechanism to shift the position of the sensor in increments of 1 pixel for each exposure so that every "scene pixel" will be sampled by all four of the bayer color filters in the 2x2 RGGB pattern. This yields a full RGB triplet for every "scene pixel" vs the single monochromatic value obtained in a single exposure. This produces three benefits: increases the chroma (color) resolution of the image, reduces the noise of each pixel (from mean/median averaging of the multiple exposures), and eliminates demosaicing artifacts because the resulting image contains linear RGB that doesn't need to be demosaiced by the raw processor.
Here's a diagram showing how the Df moves the sensor for a 4-shot pixel shift sequence:
Nikon Df sensor movement for a 4-shot pixel shift
The sequence is:
- Take first exposure at baseline sensor position
- Shift the sensor one pixel right and take second exposure
- Shift the sensor one pixel down and take third exposure
- Shift the sensor one pixel left and take fourth exposure
Increased Resolution
Another technique that can be optionally employed is to shift the sensor by a fractional amount, specifically 1/2 a pixel. This can simulate a sensor with double the linear pixel density in both the horizontal and vertical axis (shifting both up/down), resulting in an image with 4x the total number of pixels. There are lots of practical factors which limit the increase in effective resolution though.
Here's a diagram showing how the Df moves the sensor for a 16-shot pixel shift sequence. Note that it still performs the chroma-enhancing 4-shift operation on top of the 1/2 pixel shift:
Nikon Df sensor movement for a 16-shot pixel shift
I generated the above diagram by analyzing the 16 raw files created by a 16-shot shift sequence and noting which direction the image shifted for each exposure and by how much, ie 1 pixel vs 1/2 sub-pixel. Here's a visual animation showing a sample of the image data I used.
Animation: Crop section of image movement for a 16-shot pixel shift
Note that the direction of sensor movement is opposite that of the image movement. For example, shifting the sensor right by one pixel actually shifts the captured image left by one pixel. There is a significant chance I messed the process up but even so hopefully you'll get the gist
Pixel Shift options and their effect on resolution and noise
The Zf offers four possible shot counts for a pixel shift:
- 4-shot: Same resolution / pixel count as single exposure but with full chroma sampling and lower noise from mean/median averaging of the exposures
- 8-shot: Same resolution / pixel count as 4-shot mode but even lower noise from additional averaging of extra exposures.
- 16-shot: 4x the pixel count of a single exposure and with full chroma sampling. I'm being careful to not say 4x the resolution since the effective increase in resolution will be limited by factors that will be considered in a future post.
- 32-shot: Same pixel count / chroma as the 16-shot but even lower noise from additional averaging of extra exposures.
There are additional workflow options as well, including delay to start of the sequence and the inter-exposure delay. Here is a short menu walkthrough demonstrating the options:
Sample Results
I'll now present some samples showing the benefits of pixel shifting in action. First we'll look at the reduction of demosaicing artifacts (aliasing and moiré) as well as the increase in resolved detail:
Image composite: Zf single exposure vs 4-shot vs 16-shot shift vs Z7
The above is an enlarged crop from each of the respective images, all upscaled to the resolution of the 16-shot exposure via PS Preserve Details 2.0. Note I show both capture-sharpening only samples (ie, ACR raw sharpening, set to 45/0.7/35), as well as capture sharpening + USM sharpening in PS (using 300/0.5). USM is used to demonstrate how the 16-shot pixel shift can tolerate (and actually require) additional sharpening vs the single exposure and 4-shot shift. I hope to go more into this in a later post.
I chose to use the same USM values for all samples, even though it creates obvious over-sharpening artifacts for all samples other than the 16-shot shift. I did this because using right-sized sharpening for each sample runs the risk of becoming subjective. Still, I plan to do some more nuanced sharpening comparisons in a future post.
When comparing the images, try to find areas of text in the chart that are unreadable in the single exposure but readable in the pixel-shifted images. And of course compare that to the native higher resolution of the Z7 sample. Also note the reduction/elimination of aliasing/false detail.
Here are animations of the same crops to help in visualizing the differences. Unfortunately the Z7 sample is slightly rotated - I chose not to correct that in post for this comparison since doing so would affect its acuity.
Animation: Zf single exposure vs 4-shot vs 16-shot shift vs Z7, capture-sharpening only
Animation: Zf single exposure vs 4-shot vs 16-shot shift vs Z7, capture+USM sharpening
Now let's take a look at the noise improvement, with a scene shot at -5EV exposure and raised in post to make the noise differences more obvious:
Animation: Zf single exposure vs 4-shot vs 8-shot, -5EV exposure
There's a significant decrease in visible noise between the single exposure and 4-shot in the above animation. There's actually a similar noise reduction between the 4-shot and 16-shot but it's less perceptible. Special thanks to SrMi for tips on looking into the noise improvements.
I hope to shoot some additional test scenes, preferably some with actual color to better highlight the increase in chroma resolution.
Attachments
Last edited:
