OmniVision launches the OV64B, a 64MP smartphone sensor with 0.7 micron pixels

The Samsung s20 already does 8K video....

@fan onlly clube:
I'm also not a fan of small sensors, but the DR can be increase with such sensor, as it is possible to have 4 different exposures at the same time. An instant HDR effect so to say...

grafli;

You are right, SONY has two kind of "In sensor HDR" sensor, one of them is released 4 years ago as:

DOL-HDR Sensors
http://image-sensors-world.blogspot.com/2016/10/sony-unveils-dol-hdr-sensors.html

The other "In sensor HDR" phone sensor is more for still images.

Noted:
In either case, OmniVision is not as SONY sensor's in sensor HDR which is executed in very low lever, but HDR function could be executed at higher lever by Qualcomm's chip set.

@Fungshui: please explain how we have to learn science before commenting?
0.7µ is equal to 700nm. 700nm is the wavelength of visible light (red). That is correct.
What happens to a pixel that is only the size of the wavelength?
It apears that this is not even a problem?

My pleasure.

Furthermore, on the Bayer filtered sensor, the resolution of red is just half of the pixel count in orthogonal direction. It means the limit of pixel size is just 175nm for red. The limit of blue pixel is down to 100nm.

But I would argue that you can still extract more information from even smaller pixels, like 50nm. For example, you can set give them multiple sensitivity an combine their readings. Da da, a single shot ultra high resolution HDR sensor is born! There are many other ways to utilize the extra transistors, like phase-detection autofocus as what EOS-R does, or better yet, ray sensor pixels similar to Light L16 which give you distance of objects at every pixel. You can post focus at any distance, with whatever depth of field or bokeh you want. All we need is better algorithm and processors.

That's why these breakthrough never came from traditional camera vendors. They are just dinosaurs in a digital age.

@ilza

Sorry that I wasn't clear on what the extra information. Let me restate it again.

First of all, let's establish a common understanding. We agree the resolution limit is ½λ. For red light, it is 350nm and it is 200nm for blue light. Correct?

With that cleared, we can continue on what benefit can be get from a pixel density beyond Nyquist frequency.

1) Bayer interpolation.
As we all know, most sensors are built with Bayer filters. For example, there are 50,331,648 (50mil) pixels in a sensor of 8192x6144 resolution. Only a quarter of those pixels are red and they are arranged in 4096x3072 grids. A pixel size of 175nm² does not violate the Nyquist limit of 350nm because each pixel is offset 350nm, both horizontally and vertically. The green and blue channels can resolve even finer details. A Bayer sensor only gets a quarter of the raw resolution to begin with. Hence we need quadruple pixel count to get back to the real physical limit.

to be cont...

cont'd

A vendor can state the honest resolution or most likely, state the raw resolution as all do with Bayer sensor anyway. Nonetheless, a sensor of 175nm pixel size can still out resolve a 350nm sensor if the lens is sharp enough.

2) phase detection
An implementation similar to the dual pixel implementation of Canon EOS-R but instead of splitting each pixel into half, only half pixels are double duty. Every other pixels is single duty that record the luminance level only. The other half are double duty that record both luminance and phase detection value as illustrated below:

RrGgRrGgRrGg
GgBbGgBbGgBb
RrGgRrGgRrGg
GgBbGgBbGgBb

where
R - Red channel luminance only pixel;
r - Red channel phase detection + luminance double duty pixel;
G - Green channel luminance only pixel;
g - Green channel phase detection + luminance double duty pixel;
B - Blue channel luminance only pixel;
b - Blue channel phase detection + luminance double duty pixel;

to be cont...

cont'd

The value of luminance only and double duty on the right are combined to form a final luminance value of the specific channel, red, green or blue.
The phase detection value from each double duty pixels are compared to the double duty pixels of the same channel in order to calculate the focus.
It will give a half the raw resolutions horizontally but full resolution vertically. Nonetheless, Bayer interpolation limited them to a quarter to both red and blue channels, a half to green channel. We lost little. See point 1.

The advantages of this arrangement over dualpixel implemented on Canon EOS-R sensor are:
1. ½ stop better sensitivity. Every double duty pixel lost about 1 stop sensitivity due to covered part. This implementation spares half pixels from losing 1 stop.

to be cont...

cont'd

2. It gets phase detection for both horizontal and vertical directions. Using green double duty pixel for horizontal. Red and blue double duty pixels for vertical. We have "cross-type" phase detection on every image recoding pixel.

3) Lytro

The purpose of Lytro implementation is not about increasing the resolution of the image plane. It is about adding the depth value to each pixel. Thank hologram. The pixels are oriented to various directions instead to all forward arrangement. It can be done with the micro lens in front of each pixel. Therefore, while multiple pixels below Nyquist limit does not increase resolution of on the image plane, they can add direction readings. The final image still cannot resolve better then Nyquist limit allowed but you can get an infinity depth of field from a single shot. Or any depth of field with any bokeh characteristics you want after shot.

to be cont...

cont'd

Lytro is not snake oil as you state. It is just limited by the current technology on resolution and processing power. In fact, you can see smartphone is already using some of the techniques employed in Lytro. Multiple sensors are more than giving smartphone zoom capability.

4) HDR

There are better way of combining pixel data than the way quad sensor of Sony IMX686 or Samsung Nonacell do. You can give each sensor a different sensitivity. For a 2x2 setup, you can offset each ½ stop or 1 stop. You don't get better resolution than but you get 2 to 4 stop exposure bracket HDR in a single shot.

---

In conclusion, a pixel density above Nyquist can still be useful. These are just a few examples. There are a lot more. You are so focused on the resolution alone but there are a lot more useful information can be gain other than resolution alone.

@Fungshui: I’m looking forward at a pleasure of reading what you wrote — but so far I managed to read only through ①…

Anyway, for ①: with <9 red sensels per μm², 25 blue, and <16 green, one does not get more than 50 sensels/μm².

And given the scarcity of S-cones, I do not think having more blue than green makes “any sense”.

Moreover, with any decent demosaicing algorithm, Bayer loses only about 30% of (linear) resolution, not 50% as you say. So having about 30 sensels/μm² is close to the maximum *meaningful* count.

So: 30 may be meaningful (with an ideal lens) as far as Bayering is concerned; up to 50 if one wants to have no Nyquists left behind!

@liza

Are you playing dumb?

For 1), what I mean is just a simple Bayer arrangement of sensor, as illustrated below:

RGRGRGRG
GBGBGBGB

When each pixel are 175nm², The distance between 2 red pixels is 350nm. It matches the Nyquist frequency perfectly. A perfect optical setup can still reach this resolution.

Yes, green pixels are packed a bit closer. The closest distance between 2 green pixels is measured diagonally at 247nm. However, the wave length of green light is ~500nm. The diagonal green pixel pitch matches ½λ perfectly.

Blue sensor frequency is well below the Nyquist frequency. I don't think I need to elaborate here.

After Bayer interpolation, a red sub pixel reading is shared by 4 final pixels each composed of 3-channel. What you mean by 30% lost on resolution is the combined lost of resolution of 3 channels. However the resolution of red channel is just ½ of stated pixel count on both horizontal and vertical directions. It is 50% lost.

to be cont...

I prefer Samsung LSI