Increasing Imaging Resolution by Non-Regular Sampling and Joint Sparse Deconvolution & Extrapolation

[TN Args]

What do the PST guys think of this 2022 paper by Seiler et al?

https://arxiv.org/pdf/2204.12867

Abstract

"Increasing the resolution of image sensors has been a never ending struggle since many years. In this paper, we propose a novel image sensor layout which allows for the acquisition of images at a higher resolution and improved quality. For this, the image sensor makes use of non-regular sampling which reduces the impact of aliasing. Therewith, it allows for capturing details which would not be possible with state-of-the-art sensors of the same number of pixels. The non-regular sampling is achieved by rotating prototype pixel cells in a non-regular fashion. As not the whole area of the pixel cell is sensitive to light, a non-regular spatial integration of the incident light is obtained. Based on the sensor output data, a high-resolution image can be reconstructed by performing a deconvolution with respect to the integration area and an extrapolation of the information to the insensitive regions of the pixels. To solve this challenging task, we introduce a novel joint sparse deconvolution and extrapolation algorithm. The union of non-regular sampling and the proposed reconstruction allows for achieving a higher resolution and therewith an improved imaging quality."

Sounds promising to me, but I don't have the background to critique it.

cheers

 

[Jack Hogan]

Interesting idea. It seems to me that their algorithm does a better job with aliasing, as expected, but introduces ringing. Also, most enthusiast level cameras these days are BSI.

Jack

 

[knickerhawk]

Interesting idea. It seems to me that their algorithm does a better job with aliasing, as expected, but introduces ringing. Also, most enthusiast level cameras these days are BSI.

Jack

Why not apply the method to quad bayer sensors? Wouldn't it work to just "randomly" ignore one of the four subpixels, apply the pixel reconstruction and demosaicing algorithms, then repeat the process only this time "randomly" dropping a different subpixel? Each reconstruction/demosaiced pass could then be averaged together. Wouldn't this preserve the anti-aliasing benefits of the strategy without the unwanted light gathering loss implicit with the proposal in the article? Or maybe do four passes so that all 4 subpixels get excluded once. Would that open up possibilities for reduction in noise in addition to the aliasing benefits?

Also would have the advantage of being easily applied to existing Sony quad bayer sensors. The downside, of course, is more processing time and energy required, but that should be insignificant, especially if this processing option is performed off-camera.

 

[Jack Hogan]

Interesting idea. It seems to me that their algorithm does a better job with aliasing, as expected, but introduces ringing. Also, most enthusiast level cameras these days are BSI.

Jack

Why not apply the method to quad bayer sensors? Wouldn't it work to just "randomly" ignore one of the four subpixels, apply the pixel reconstruction and demosaicing algorithms, then repeat the process only this time "randomly" dropping a different subpixel? Each reconstruction/demosaiced pass could then be averaged together. Wouldn't this preserve the anti-aliasing benefits of the strategy without the unwanted light gathering loss implicit with the proposal in the article? Or maybe do four passes so that all 4 subpixels get excluded once. Would that open up possibilities for reduction in noise in addition to the aliasing benefits?

Also would have the advantage of being easily applied to existing Sony quad bayer sensors. The downside, of course, is more processing time and energy required, but that should be insignificant, especially if this processing option is performed off-camera.

All good questions Nick.

 

[J A C S]

They are trying to do the impossible. Random sampling (and their sampling is not quite random) has been studied and used, and in principle can reduce the visibility of aliasing artifacts from repeated, mostly humane made patterns. "Reducing the visibility" is subjective. You do not see the typical random aliasing artifacts like moire, etc.; instead you see random aliasing. Is that better? Depends on who you are asking. I am very sensitive to it. They actually say all this in the introduction but mostly as a criticism to other algorithms.

The only interesting example there is the kitty image. It is too small and lacking detail even before the sampling to make conclusions. I do not see some big improvement with their method.