a surprise with Z8 pixel shift

[Bernard Delley]

what 4 and 16 step pixels shifts do can be easily guessed. The wording on 8 and 32 step procedures was not obvious to me. It sounded like noise reduction.

So lets have a look. My full field MTF analysis does perspective analysis on the side to support accurate chart alignment. It is described in my publication and and supporting info. The perspective analysis describes the relation of positions on the chart to positions on the sensor.

The pixel shift steps were found as shown in the schemes below. For example in the 4 step procedure: if a landscape feature is projected onto the R pixel on the first step, then the first horizontal move puts the G1 pixel onto that image, the next vertical move puts the B pixel on it and the final horizontal move puts the G2 on it. Thus, full color information is sampled for each pixel and no de-mosaic interpolations are needed to fill it in.

The 16 step procedure is as expected, providing twice as fine spatial sampling in x and y direction. After the initial 4 step tour 3 further 4 step tours are done along a similar scheme of half step size.

But, what should the 8 and the 32 step procedures be? Is it providing finer sampling in one direction ? No ! My experiment shows that 8 and 32 step procedures repeat the 4 or 16 step procedures, however taking samples with the vertically neighboring Bayer patch ! This is not just doubling the takes for noise reduction.

My guess is, that is done to gather real image samples for the rows with many mapped image pixels, which are functionally dedicated to AF duty.

 

[Ryan Mack]

Yep, here's the offsets I use in my pixel-shift merging software.

case 4: {{0, 0}, {0, 1}, {1, 1}, {1, 0}};

case 8: {{0, 0}, {0, 1}, {1, 1}, {1, 0}, {2, 0}, {2, 1}, {3, 1}, {3, 0}};

case 16: {{0, 0}, {0, 2}, {2, 2}, {2, 0}, {0, 1}, {0, 3}, {2, 3}, {2, 1},

{1, 1}, {1, 3}, {3, 3}, {3, 1}, {1, 0}, {1, 2}, {3, 2}, {3, 0}};

case 32: {{0, 0}, {0, 2}, {2, 2}, {2, 0}, {0, 1}, {0, 3}, {2, 3}, {2, 1},

{1, 1}, {1, 3}, {3, 3}, {3, 1}, {1, 0}, {1, 2}, {3, 2}, {3, 0},

{4, 0}, {4, 2}, {6, 2}, {6, 0}, {4, 1}, {4, 3}, {6, 3}, {6, 1},

{5, 1}, {5, 3}, {7, 3}, {7, 1}, {5, 0}, {5, 2}, {7, 2}, {7, 0}};

In addition to pixel mapping and hot pixels, I also wonder if there's some row-to-row variation caused by the AF system. I know early on-sensor phase detect AF systems had some rows with slightly different optical characteristics. I haven't seen any mention of that with Nikon Z sensors but if they did have that problem then this would help mitigate it.

 

[Horshack]

But, what should the 8 and the 32 step procedures be? Is it providing finer sampling in one direction ? No ! My experiment shows that 8 and 32 step procedures repeat the 4 or 16 step procedures, however taking samples with the vertically neighboring Bayer patch ! This is not just doubling the takes for noise reduction.

My guess is, that is done to gather real image samples for the rows with many mapped image pixels, which are functionally dedicated to AF duty.

Per Nikon's own on-screen documentation in the Pixel Shift menu, the 8/32 modes are indeed for lower noise. The fact that is uses neighboring pixels rather than resampling the same pixels again is likely because an algorithmic preference to keep the sensor moving the same direction rather than reversing direction to resample.

My post two months ago on Nikon's Pixel Shift covered this:

https://www.dpreview.com/forums/post/67458022

 

[Bernard Delley]

But, what should the 8 and the 32 step procedures be? Is it providing finer sampling in one direction ? No ! My experiment shows that 8 and 32 step procedures repeat the 4 or 16 step procedures, however taking samples with the vertically neighboring Bayer patch ! This is not just doubling the takes for noise reduction.

My guess is, that is done to gather real image samples for the rows with many mapped image pixels, which are functionally dedicated to AF duty.

Per Nikon's own on-screen documentation in the Pixel Shift menu, the 8/32 modes are indeed for lower noise. The fact that is uses neighboring pixels rather than resampling the same pixels again is likely because an algorithmic preference to keep the sensor moving the same direction rather than reversing direction to resample.

My post two months ago on Nikon's Pixel Shift covered this:

https://www.dpreview.com/forums/post/67458022

sorry, I had forgotten about your "deep dive into Nikon pixel shift", which I read with interest at the time. It was evaluated with the Zf, and I had no prospect for access to pixel shift. Your examples seemed to be taken imaging a poorly printed page which left me unimpressed by the gain in IQ.

I found a demo with an IPA beer can impressive, shown near bottom of page. But, I disagree with the MTF analysis presented there: pixel shift changes the sampling and thus the Nyqvist, but the MTF of lens and pixels remains the same.

 

[Ryan Mack]

I found a demo with an IPA beer can impressive, shown near bottom of page. But, I disagree with the MTF analysis presented there: pixel shift changes the sampling and thus the Nyqvist, but the MTF of lens and pixels remains the same.

Does anybody know how to contact Ed Dozier from photoartfromscience.com? I tried googling him but it looks like a pseudonym. I'd like to work with the guy to evaluate the results of my own pixel shift software compared to NX Studio which has a few serious limitations.

 

[Horshack]

But, what should the 8 and the 32 step procedures be? Is it providing finer sampling in one direction ? No ! My experiment shows that 8 and 32 step procedures repeat the 4 or 16 step procedures, however taking samples with the vertically neighboring Bayer patch ! This is not just doubling the takes for noise reduction.

My guess is, that is done to gather real image samples for the rows with many mapped image pixels, which are functionally dedicated to AF duty.

Per Nikon's own on-screen documentation in the Pixel Shift menu, the 8/32 modes are indeed for lower noise. The fact that is uses neighboring pixels rather than resampling the same pixels again is likely because an algorithmic preference to keep the sensor moving the same direction rather than reversing direction to resample.

My post two months ago on Nikon's Pixel Shift covered this:

https://www.dpreview.com/forums/post/67458022

sorry, I had forgotten about your "deep dive into Nikon pixel shift", which I read with interest at the time. It was evaluated with the Zf, and I had no prospect for access to pixel shift. Your examples seemed to be taken imaging a poorly printed page which left me unimpressed by the gain in IQ.

No need to apologize - it's always good to get more eyes on an interesting subject like this. The printed page I chose could not be correctly resolved by a single exposure without obvious aliasing and demosaicing artifacts, which makes it an ideal sample to demonstrate the gain expected from pixel shifting.

 

[Bernard Delley]

But, what should the 8 and the 32 step procedures be? Is it providing finer sampling in one direction ? No ! My experiment shows that 8 and 32 step procedures repeat the 4 or 16 step procedures, however taking samples with the vertically neighboring Bayer patch ! This is not just doubling the takes for noise reduction.

My guess is, that is done to gather real image samples for the rows with many mapped image pixels, which are functionally dedicated to AF duty.

Per Nikon's own on-screen documentation in the Pixel Shift menu, the 8/32 modes are indeed for lower noise. The fact that is uses neighboring pixels rather than resampling the same pixels again is likely because an algorithmic preference to keep the sensor moving the same direction rather than reversing direction to resample.

My post two months ago on Nikon's Pixel Shift covered this:

https://www.dpreview.com/forums/post/67458022

sorry, I had forgotten about your "deep dive into Nikon pixel shift", which I read with interest at the time. It was evaluated with the Zf, and I had no prospect for access to pixel shift. Your examples seemed to be taken imaging a poorly printed page which left me unimpressed by the gain in IQ.

No need to apologize - it's always good to get more eyes on an interesting subject like this. The printed page I chose could not be correctly resolved by a single exposure without obvious aliasing and demosaicing artifacts, which makes it an ideal sample to demonstrate the gain expected from pixel shifting.

I designed once a chart pixel by pixel in pnm for the iMac retina screen 5120 x 2880 which remains a challenge for aliasing in its finest detail. I lost my program for generation it, but, I still have a png converted version somewhere. I have not used it in a long time. The png provides full pixel resolution: 'back lit', high contrast.

 

[Bernard Delley]

I found a demo with an IPA beer can impressive, shown near bottom of page. But, I disagree with the MTF analysis presented there: pixel shift changes the sampling and thus the Nyqvist, but the MTF of lens and pixels remains the same.

Does anybody know how to contact Ed Dozier from photoartfromscience.com? I tried googling him but it looks like a pseudonym. I'd like to work with the guy to evaluate the results of my own pixel shift software compared to NX Studio which has a few serious limitations.

I plan to write a simple program combining the pixel shifted raw files into a P3 or P6 type basic tricolor pnm pixel file. PS may do the fine tuning for color balance etc. I will not find time for this in the upcoming weeks though. This route will allow me to keep full control over the raw data and avoid sharpening and other processing in a foreign software, that may not give access to full control.

I guess with your skills you could easily produce such a program for reference purposes.

 

[Ryan Mack]

I found a demo with an IPA beer can impressive, shown near bottom of page. But, I disagree with the MTF analysis presented there: pixel shift changes the sampling and thus the Nyqvist, but the MTF of lens and pixels remains the same.

Does anybody know how to contact Ed Dozier from photoartfromscience.com? I tried googling him but it looks like a pseudonym. I'd like to work with the guy to evaluate the results of my own pixel shift software compared to NX Studio which has a few serious limitations.

I plan to write a simple program combining the pixel shifted raw files into a P3 or P6 type basic tricolor pnm pixel file. PS may do the fine tuning for color balance etc. I will not find time for this in the upcoming weeks though. This route will allow me to keep full control over the raw data and avoid sharpening and other processing in a foreign software, that may not give access to full control.

I guess with your skills you could easily produce such a program for reference purposes.

I have already done this, that's why I'm here doing research It's a LR plugin that outputs a DNG with no additional processing beyond combining the files (although you can turn on some additional options if you like). If you don't use Lightroom then you can use it via the command line (let me know if you need help with that, it isn't well documented). And if you don't want a DNG you *could* have it output a TIFF file with the 16 bit RAW data in it although that's really intended for debugging. Can provide instructions for that too.

You can download it at https://mackman.net/mergeraw