16-shot pixel shift shooting -- Unusable gimmic?

Started 7 months ago | Discussions
JimKasson
JimKasson Forum Pro • Posts: 34,829
Re: 16-shot pixel shift shooting -- Unusable gimmic?
2

fieldray wrote:

JimKasson wrote:

fieldray wrote:

Pixel sampling issues are some of the most frustrating (sometimes impossible!) things to remove for me. The truth for me is that the 4x pixel shift is genuinely useful for dealing with color aliasing but I have not encountered a situation in my photography where spatial sampling with a 60MP array limits the effectiveness of my photography, so the 16-shot pixel shift is fun to play with but will not likely help my photography. This reminds me of some of the f#*lamda/p (Q) arguments we used to have designing space optics! Q=2 was our holy grail as there was by definition no aliasing, but from a practical point of view Q=1 or Q=1.2 was a better optimization. On some of our FLIRs with bigger pixels compared to the optics cutoff, sub-pixel shifting analogous to the 16-shot function on the Sony camera made a big difference in the final product

Q = 2 means two samples per Sparrow Distance, right? To do that with a Bayer CFA will take about two binary order of magnitudes decrease in pitch at reasonable f-stops. Or we could just stop way down...

Q=2 for a Sony A7riv sensor is about f/13 if you figure on a center wavelength of .55 um. So based on our old arguments, optimum f/number for practical use might be about f/8. Turns out that non-linear image restoration ‘ super resolution algorithms like to use Nyquist sampling (Q=2) but for pixel peeping, image detail is really getting soft at this point. Bayer sampling doesn’t change the sampling cutoff but it puts a dent in the higher frequency MTF.

The green diagonal sampling pitch is the same, but the green horizontal and vertical pitch changes by a factor of two and all the red and blue pitches change by a factor of two. You could argue that, if you had a priori knowledge that the subject were monochromatic, that the pitch doesn't change.

As for two binary order of magnitudes, I think that one order of magnitude (factor of 2) would be plenty. My experience in the technical world is that depth of focus over the field of view becomes the dominant challenge if you are trying to do diffraction limited imagery much below f/4 or f/5. Thanks for the discussion!

I agree with a almost all of what you have to say, but here's a counterexample to the notion that f/8 in a 3.76 um Bayer CFA sensor will eliminate aliasing.

https://blog.kasson.com/gfx-100/a-visual-look-at-gfx-100-diffraction-blur/

Of course, many subjects will not produce visible aliasing at f/8. But fabric often will.

You are entirely correct about DOF. Then you have to stack, use deconvolution, or extreme measures like this:

This is shown with my GFX 50R, but works fine with an a7RIV.

Jim

-- hide signature --
 JimKasson's gear list:JimKasson's gear list
Nikon Z7 Fujifilm GFX 100 Sony a7R IV Sony a9 II +1 more
JimKasson
JimKasson Forum Pro • Posts: 34,829
Re: Wrong example?
2

blue_skies wrote:

I see plenty of reviewers and example online showing that it does work as advertised.

I think it works as designed. There's a subtle difference.

There are two places to avoid using it:

a) when there is movement in the image, e.g. in landscape shots

b) when there is extreme shallow DOF, e.g. in your example

Additionally, your shutter time of 1.3 sec is quite lengthy, and prone to vibrations (softening).

For (extreme) macro, ie. very shallow DOF, I would suggest to use focus stacking, this is unrelated to pixel shift, but it gives you what I think that you are after.

If you're a glutton for punishment, you can use both to create a single final image.

For basic pixel shift at 4x versus 16x, 4x shifts by a whole pixel, and 16x shifts by half a pixel. For images which have clear sharpness defined at this level of pixel-detail, the image will up-sample quite nicely.

Pixel shift does not change the MTF curve, but it does reduce aliasing:

https://blog.kasson.com/the-last-word/does-pixel-shift-increase-resolution/

Try out some other scenes first, and then decide if your example is just a bad example.
Particularly, start with an example which has a larger area of in-focus-plane details that show better when up-sampled (pixel-shifted).

-- hide signature --
 JimKasson's gear list:JimKasson's gear list
Nikon Z7 Fujifilm GFX 100 Sony a7R IV Sony a9 II +1 more
JimKasson
JimKasson Forum Pro • Posts: 34,829
Re: 16-shot pixel shift shooting -- Unusable gimmic?
2

fieldray wrote:

My experience in the technical world is that depth of focus over the field of view becomes the dominant challenge if you are trying to do diffraction limited imagery much below f/4 or f/5.

Some of my thoughts on what to do about that:

https://blog.kasson.com/the-last-word/real-world-sharpness-what-to-do/

There is also the issue of trading off DOF and diffraction depending on the scene:

https://blog.kasson.com/?s=optimizing+dof

-- hide signature --
 JimKasson's gear list:JimKasson's gear list
Nikon Z7 Fujifilm GFX 100 Sony a7R IV Sony a9 II +1 more
fieldray Regular Member • Posts: 147
Re: 16-shot pixel shift shooting -- Unusable gimmic?

JimKasson wrote:

fieldray wrote:

JimKasson wrote:

fieldray wrote:

Pixel sampling issues are some of the most frustrating (sometimes impossible!) things to remove for me. The truth for me is that the 4x pixel shift is genuinely useful for dealing with color aliasing but I have not encountered a situation in my photography where spatial sampling with a 60MP array limits the effectiveness of my photography, so the 16-shot pixel shift is fun to play with but will not likely help my photography. This reminds me of some of the f#*lamda/p (Q) arguments we used to have designing space optics! Q=2 was our holy grail as there was by definition no aliasing, but from a practical point of view Q=1 or Q=1.2 was a better optimization. On some of our FLIRs with bigger pixels compared to the optics cutoff, sub-pixel shifting analogous to the 16-shot function on the Sony camera made a big difference in the final product

Q = 2 means two samples per Sparrow Distance, right? To do that with a Bayer CFA will take about two binary order of magnitudes decrease in pitch at reasonable f-stops. Or we could just stop way down...

Q=2 for a Sony A7riv sensor is about f/13 if you figure on a center wavelength of .55 um. So based on our old arguments, optimum f/number for practical use might be about f/8. Turns out that non-linear image restoration ‘ super resolution algorithms like to use Nyquist sampling (Q=2) but for pixel peeping, image detail is really getting soft at this point. Bayer sampling doesn’t change the sampling cutoff but it puts a dent in the higher frequency MTF.

The green diagonal sampling pitch is the same, but the green horizontal and vertical pitch changes by a factor of two and all the red and blue pitches change by a factor of two. You could argue that, if you had a priori knowledge that the subject were monochromatic, that the pitch doesn't change.

Yes!  This is where the 4-shot pixel shift works!

As for two binary order of magnitudes, I think that one order of magnitude (factor of 2) would be plenty. My experience in the technical world is that depth of focus over the field of view becomes the dominant challenge if you are trying to do diffraction limited imagery much below f/4 or f/5. Thanks for the discussion!

I agree with a almost all of what you have to say, but here's a counterexample to the notion that f/8 in a 3.76 um Bayer CFA sensor will eliminate aliasing.

https://blog.kasson.com/gfx-100/a-visual-look-at-gfx-100-diffraction-blur/

Yes - this supports that you need to be at f/13 to insure elimination of all aliasing.  Just saying that in most cases f/8 is sufficient and looks a little sharper than f/11 or f/13.

Of course, many subjects will not produce visible aliasing at f/8. But fabric often will.

You are entirely correct about DOF. Then you have to stack, use deconvolution, or extreme measures like this:

This is shown with my GFX 50R, but works fine with an a7RIV.

Jim

 fieldray's gear list:fieldray's gear list
Nikon D5 Nikon D850 Sony a7R IV Sony a9 II Nikon Z6 II +14 more
Beatsy
Beatsy Senior Member • Posts: 1,166
Re: 16-shot pixel shift shooting -- Unusable gimmic?
12

I wouldn't say it's a gimmick, but pixel shift *is* only useful in very controlled (studio) settings IMO.

The key to success, (as in getting repeatably improved results with it), is stability and no subject movement. You need a very stable setup that's free from vibration, especially if working at high magnifications. My first test was done using a Sony 90mm G macro at near 1:1 on a rail and optical bench like this (archive shot because this setup changes near-daily). The rail is on the left.

Test subject was a silicon chip wafer placed face on to the lens. Full frame FoV was this (just a converted RAW here, not pixel shifted).

I shot a single ARW, a 4-shot pixel shift and 16-shot version and overlaid them to show the differences. A crop of one die is 42Mb - so here's a 100% crop from the centre. You should be able to figure out what's what from the partial labels. Top left one says "240MP..." and the other two images were upscaled to the same size as the 240Mpix one for compositing).

As you can see - definite improvements in sharpness, colour and resolution. In controlled conditions.

I rarely use the 16-shot shift, but the 4-shot version is fantastic for getting better colour resolution (and therefore better resolution in general) in a photomicrography setting. I use it very often there.

Edit: Forgot to say that I see no point using pixel-shift for landscape or handheld. Too many things to fail and they nearly always do. Much easier to take a nine-shot pano with a longer FL lens (on a pano head, preferably) and get the resolution that way.

 Beatsy's gear list:Beatsy's gear list
Sony a7R II Sony a9 Sony a7R IV Sony a1 Canon MP-E 65mm f/2.5 1-5x Macro +16 more
Tim O'Connor
Tim O'Connor Veteran Member • Posts: 5,641
Re: Interesting

Trollmannx wrote:

Tim O'Connor wrote:

fferreres wrote:

eques wrote:

AlephNull wrote:

I wouldn’t (yet) call it an unusable gimmick, but I do get the feeling that getting pixel-level sharp images will require preparation, organisation, and maybe a little luck.

It works handheld up to 1/60 s with an Olympus em1.3.
https://www.dpreview.com/forums/post/64061978

I keep hearing “oh, but it works perfectly on Panasonic” - I wonder if their merging of images does a bit of blurring, or smoothing, to eliminate problems in one shot of the sequence?

I am chiming in because coming from M43 I plan to move to FF. So this is an interesting topic, because I also consider the EM1.3 as an alternative to FF.

Peter

It’s iffy and poor. I think it’s as bad as it can get just short of being completely useless.

Only if one does not know how to use this tool.

As said above: great for product photography and reproduction work.

I've used it successfully - but it is still poorly implemented IMHO. Sony should at least offer in camera processing, rather than having to rely on their desktop tools, or free tools likeSonyPixelShift2DNG.

-- hide signature --
 Tim O'Connor's gear list:Tim O'Connor's gear list
Nikon D600 Nikon D500 Sony a7R IV Canon MP-E 65mm f/2.5 1-5x Macro Nikon AF-S DX Nikkor 17-55mm f/2.8G ED-IF +11 more
Avery Robins New Member • Posts: 8
No
3

ahaslett wrote:

eques wrote:

AlephNull wrote:

I wouldn’t (yet) call it an unusable gimmick, but I do get the feeling that getting pixel-level sharp images will require preparation, organisation, and maybe a little luck.

It works handheld up to 1/60 s with an Olympus em1.3.
https://www.dpreview.com/forums/post/64061978

I keep hearing “oh, but it works perfectly on Panasonic” - I wonder if their merging of images does a bit of blurring, or smoothing, to eliminate problems in one shot of the sequence?

I am chiming in because coming from M43 I plan to move to FF. So this is an interesting topic, because I also consider the EM1.3 as an alternative to FF.

Peter

That’s not done by pixel shift. HHRes has IBIS on while taking each shot and off in between. As Magnar says you can get a similar effect by taking a burst with IBIS set to recentre between individual shots. You then need to merge the shots with a suitable programme that understands super resolution.

Andrew

I am sorry but it is wrong to say any camera can create in camera high resolution photos. Actually no Sony can.

If you are saying any camera can be used to take multiple photos that are later combined on a PC to create a higher resolution photo, that is true.

Magnar W
Magnar W Veteran Member • Posts: 6,024
Re: No
1

Avery Robins wrote:

ahaslett wrote:

eques wrote:

AlephNull wrote:

I wouldn’t (yet) call it an unusable gimmick, but I do get the feeling that getting pixel-level sharp images will require preparation, organisation, and maybe a little luck.

It works handheld up to 1/60 s with an Olympus em1.3.
https://www.dpreview.com/forums/post/64061978

I keep hearing “oh, but it works perfectly on Panasonic” - I wonder if their merging of images does a bit of blurring, or smoothing, to eliminate problems in one shot of the sequence?

I am chiming in because coming from M43 I plan to move to FF. So this is an interesting topic, because I also consider the EM1.3 as an alternative to FF.

Peter

That’s not done by pixel shift. HHRes has IBIS on while taking each shot and off in between. As Magnar says you can get a similar effect by taking a burst with IBIS set to recentre between individual shots. You then need to merge the shots with a suitable programme that understands super resolution.

Andrew

I am sorry but it is wrong to say any camera can create in camera high resolution photos. Actually no Sony can.

No claims about in-camera processing here, as I can see. Ahaslett wrote that you need a suitable piece of software, and I said the same.

If you are saying any camera can be used to take multiple photos that are later combined on a PC to create a higher resolution photo, that is true.

That's exactly what was said!

 Magnar W's gear list:Magnar W's gear list
Sony a7 Sony a7R III Sony FE 90mm F2.8 macro Zeiss Loxia 21mm F2.8 Zeiss Loxia 35 +2 more
whumber
whumber Veteran Member • Posts: 3,633
Re: No

Avery Robins wrote:

I am sorry but it is wrong to say any camera can create in camera high resolution photos. Actually no Sony can.

If you are saying any camera can be used to take multiple photos that are later combined on a PC to create a higher resolution photo, that is true.

Olympus and Pentax both do that. How successfully they do it is another topic of conversation.

EDIT: Nevermind, I think I misinterpreted what you wrote.

 whumber's gear list:whumber's gear list
Fujifilm X-T1 Olympus OM-D E-M1X Olympus E-M1 III Olympus M.Zuiko Digital ED 60mm F2.8 Macro Olympus M.Zuiko Digital ED 75mm F1.8 +9 more
ahaslett
ahaslett Veteran Member • Posts: 8,707
Re: No

Avery Robins wrote:

ahaslett wrote:

eques wrote:

AlephNull wrote:

I wouldn’t (yet) call it an unusable gimmick, but I do get the feeling that getting pixel-level sharp images will require preparation, organisation, and maybe a little luck.

It works handheld up to 1/60 s with an Olympus em1.3.
https://www.dpreview.com/forums/post/64061978

I keep hearing “oh, but it works perfectly on Panasonic” - I wonder if their merging of images does a bit of blurring, or smoothing, to eliminate problems in one shot of the sequence?

I am chiming in because coming from M43 I plan to move to FF. So this is an interesting topic, because I also consider the EM1.3 as an alternative to FF.

Peter

That’s not done by pixel shift. HHRes has IBIS on while taking each shot and off in between. As Magnar says you can get a similar effect by taking a burst with IBIS set to recentre between individual shots. You then need to merge the shots with a suitable programme that understands super resolution.

Andrew

I am sorry but it is wrong to say any camera can create in camera high resolution photos. Actually no Sony can.

If you are saying any camera can be used to take multiple photos that are later combined on a PC to create a higher resolution photo, that is true.

The EM1.3 is an Olympus  camera.

Andrew

-- hide signature --

Infinite are the arguments of mages. Truth is a jewel with many facets. Ursula K LeGuin
Please feel free to edit any images that I post

 ahaslett's gear list:ahaslett's gear list
Sigma DP1 Merrill Sigma DP3 Merrill Olympus E-M1 Sony a7R Panasonic Lumix DMC-GM1 +28 more
OP smiba New Member • Posts: 2
Re: 16-shot pixel shift shooting -- Unusable gimmic?

Beatsy wrote:

As you can see - definite improvements in sharpness, colour and resolution. In controlled conditions.

This is cool to see!

Could you give me a comparison between the 4 shot and the 16 shot? As the piece in the middle (Is it a register?) may look a bit sharper on the 4 shot? But that might just be the downscaling of the 16-shot which makes it look a bit softer.

I will see if I can try to get my setup a bit more stable. I think the ballhead on the tripod does not allow it to be 100% stable which is why it might become moved. Although I'd assume this would also affect the 4-shot, but maybe the movement is slightly compensated for? (After all its not trying to get a higher resolution, just a more accurate colour and detail representation)

 smiba's gear list:smiba's gear list
Fujifilm X100V Sony a7R IV Sony FE 90mm F2.8 macro Sony FE 24-70mm F2.8 GM Sony FE 50mm F1.4 ZA
Qqphot Regular Member • Posts: 205
Re: 16-shot pixel shift shooting -- Unusable gimmic?
2

I've been able to make it work indoors with fixed lighting and subject and the camera on a heavy tripod, which I think is the only way it could really be practical.

I think getting 200+ MP at all requires that level of stability, it's just that with pixel shift any defect is much more visible since it manifests as weird color glitches instead of blur. This isn't even to mention stuff like mount alignment, optics, etc.

Ed Constable Regular Member • Posts: 396
Re: 16-shot pixel shift shooting -- Unusable gimmic?

But it is unusable for many Mac users. The Sony software needed does not run on Big Sur.

 Ed Constable's gear list:Ed Constable's gear list
Fujifilm GFX 100 Fujifilm GFX 100S Sony a1 Sony 50mm F2.8 Macro Sony FE 35mm F2.8 +13 more
Beatsy
Beatsy Senior Member • Posts: 1,166
Re: 16-shot pixel shift shooting -- Unusable gimmic?

smiba wrote:...

Could you give me a comparison between the 4 shot and the 16 shot? As the piece in the middle (Is it a register?) may look a bit sharper on the 4 shot? But that might just be the downscaling of the 16-shot which makes it look a bit softer.

The 4-shot and single shot were upsampled to 240 megapixels before compositing. There was no downsampling as that would have thrown away detail. This could make the single and 4-shot parts appear sharper, but the 16-shot bit is clearly *resolving* more.

Don't forget, we're pretty much up against diffraction limits at this resolution even at f/3.5 (< 2 micron effective pixel pitch) . I suspect that's the main cause of any softening you see here.

 Beatsy's gear list:Beatsy's gear list
Sony a7R II Sony a9 Sony a7R IV Sony a1 Canon MP-E 65mm f/2.5 1-5x Macro +16 more
JimKasson
JimKasson Forum Pro • Posts: 34,829
Re: 16-shot pixel shift shooting -- Unusable gimmic?

Beatsy wrote:

smiba wrote:...

Could you give me a comparison between the 4 shot and the 16 shot? As the piece in the middle (Is it a register?) may look a bit sharper on the 4 shot? But that might just be the downscaling of the 16-shot which makes it look a bit softer.

The 4-shot and single shot were upsampled to 240 megapixels before compositing.

This is very, uh, illuminating.

What was the resampling algorithm?

Did you try Adobe's Enhance Details on the unshifted raw?

There was no downsampling as that would have thrown away detail. This could make the single and 4-shot parts appear sharper, but the 16-shot bit is clearly *resolving* more.

Don't forget, we're pretty much up against diffraction limits at this resolution even at f/3.5 (< 2 micron effective pixel pitch) . I suspect that's the main cause of any softening you see here.

Sparrow distance for f/4 and 500 nm light is 2 um. 1 um sampling would provide a Q of 2. Q = 1 is probably reaching the point of diminishing returns, though. That's for the luminance component of a Bayer CFA sampler, with optimum reconstruction of a monochromatic subject.

-- hide signature --
 JimKasson's gear list:JimKasson's gear list
Nikon Z7 Fujifilm GFX 100 Sony a7R IV Sony a9 II +1 more
JimKasson
JimKasson Forum Pro • Posts: 34,829
Re: 16-shot pixel shift shooting -- Unusable gimmic?
3

Ed Constable wrote:

But it is unusable for many Mac users. The Sony software needed does not run on Big Sur.

Have you tried this?

https://www.fastrawviewer.com/PixelShift2DNG

Jim

-- hide signature --
 JimKasson's gear list:JimKasson's gear list
Nikon Z7 Fujifilm GFX 100 Sony a7R IV Sony a9 II +1 more
bluevellet Veteran Member • Posts: 3,646
Re: Interesting
2

eques wrote:

AlephNull wrote:

I wouldn’t (yet) call it an unusable gimmick, but I do get the feeling that getting pixel-level sharp images will require preparation, organisation, and maybe a little luck.

It works handheld up to 1/60 s with an Olympus em1.3.
https://www.dpreview.com/forums/post/64061978

I keep hearing “oh, but it works perfectly on Panasonic” - I wonder if their merging of images does a bit of blurring, or smoothing, to eliminate problems in one shot of the sequence?

I am chiming in because coming from M43 I plan to move to FF. So this is an interesting topic, because I also consider the EM1.3 as an alternative to FF.

Peter

I've managed to do it a full 1 second with HHHR with a 75mm (150mm equivalent).

 bluevellet's gear list:bluevellet's gear list
Nikon Z6 Olympus E-M1 III Panasonic Lumix G 20mm F1.7 ASPH Olympus 17mm F1.2 Pro Nikon Z 14-30mm F4 +13 more
Beatsy
Beatsy Senior Member • Posts: 1,166
Re: Interesting
1

bluevellet wrote:

I've managed to do it a full 1 second with HHHR with a 75mm (150mm equivalent).

Glad to see your subsequent resuscitation worked out OK

 Beatsy's gear list:Beatsy's gear list
Sony a7R II Sony a9 Sony a7R IV Sony a1 Canon MP-E 65mm f/2.5 1-5x Macro +16 more
fieldray Regular Member • Posts: 147
Re: 16-shot pixel shift shooting -- Unusable gimmic?

JimKasson wrote:

Ed Constable wrote:

But it is unusable for many Mac users. The Sony software needed does not run on Big Sur.

Have you tried this?

https://www.fastrawviewer.com/PixelShift2DNG

Jim

It works fine on my Mac

 fieldray's gear list:fieldray's gear list
Nikon D5 Nikon D850 Sony a7R IV Sony a9 II Nikon Z6 II +14 more
Ed Constable Regular Member • Posts: 396
Re: 16-shot pixel shift shooting -- Unusable gimmic?
2

Thanks Jim. I had only got version 1.0.0, which does not support the a1. I just updated to 1.1.0.94 and all is hunky dory.

Ed

 Ed Constable's gear list:Ed Constable's gear list
Fujifilm GFX 100 Fujifilm GFX 100S Sony a1 Sony 50mm F2.8 Macro Sony FE 35mm F2.8 +13 more
Keyboard shortcuts:
FForum MMy threads