Random Bayer pattern idea

Reading all these discussions about X3 vs Bayer interpolation comparisons I just thought if a random Bayer pattern can eliminate interpolation artifacts?

As far as I can see, most artifacts are actually aliasing between Bayer pattern and some regular structure in the picture subject. If so, making RGB sequence random, rather than regular should eliminate this aliasing.

Is there any drawback of this? Interpolation filter becomes more complex, obviously. In it's simple form it might be less optimized than the regular structure filter. On the other hand, there is no chance the real-world picture pattern would match this random Bayer filter, so artifacts should become much less visible.

Can this be the Bayer camp answer on Foveon's challange?
--Vladimir.

Vladimir Koifman said:

Reading all these discussions about X3 vs Bayer interpolation
comparisons I just thought if a random Bayer pattern can eliminate
interpolation artifacts?

Click to expand...

Currently the color filters are a fixed part of the sensor. So either you are proposing a scheme that would mechanically move them or one that would electronically move them.

If you could electronically move them, then all sensor sites would be sensitive to all colors and you'd just have an X3 sensor again.

BTW, it might be interesting to combine the two techniques. Say the Foveon sensor has a lot of noise in one color channel. You could average the color data from nearby sensors for just that channel. You would obviously reduce resolution in that color, but the results may be better than the noisy version. Perhaps the "variable pixel resolution" already does this. (Of course, this would only be sensible if one color was much noiser than others.)--ErikFree Windows JPEG comment editor http://home.cfl.rr.com/maderik/edjpgcom

Vladimir Koifman said:

Reading all these discussions about X3 vs Bayer interpolation
comparisons I just thought if a random Bayer pattern can eliminate
interpolation artifacts?

Click to expand...

You can't recover information that is discarded.

Vladimir Koifman said:

As far as I can see, most artifacts are actually aliasing between
Bayer pattern and some regular structure in the picture subject.
If so, making RGB sequence random, rather than regular should
eliminate this aliasing.

Click to expand...

The regular patterns that you're mentioning are the best way to get artifacts over a large area. However, they can happen in tiny spots all over the picture the regular structure.

Vladimir Koifman said:

Is there any drawback of this? Interpolation filter becomes more
complex, obviously. In it's simple form it might be less optimized
than the regular structure filter. On the other hand, there is no
chance the real-world picture pattern would match this random Bayer
filter, so artifacts should become much less visible.

Can this be the Bayer camp answer on Foveon's challange?

Click to expand...

It might help reduce color moire in some fixed patterns. It would do nothing to help the detail information that is discarded and by making the pattern random, it would create areas with very poor performance.

--Ron ParrFAQ: http://www.cs.duke.edu/~parr/photography/faq.htmlGallery: http://www.pbase.com/parr/

Erik,

Let me clarify my suggestion.
I meant fixed in time random pattern, rather than randomization in time.

It's a spacial randomization, once designed and manufactured, it is not changing in time.

I.e. instead of periodic RGBGRGB pattern put a fixed seemingly random pattern like RBGBRGGBGR... Like in the original Bayer pattern green should be twice more dense than red and blue, but there is no periodicity. The difference between the classic pattern and "random" Bayer is that the mapping of RGB filter cells is random.

The idea here is to avoid matching of the repeating RGB structure to a smilar regular texture in the real world. If such a matching occurs this generated a colorful moire picture which is very visible. If Bayer filter cells are randomly mapped, it's very improbably that this palcement matches anything in the world.

On top of it, this randomness might create film-looking pictures, where grains are not located in a very periodic structure. Well, I'm not sure the film pictures are better.

--Vladimir.

Ron Parr said:

You can't recover information that is discarded.

Click to expand...

Agree. The lost information is lost forever. I'm not trying to recover it. I'm just trying to reduce artifacts visibility. This does not add any new information, rather it just organizes it differently, so loss of information does not manifest itself in colorful moire anymore.

Member said:

The regular patterns that you're mentioning are the best way to get
artifacts over a large area. However, they can happen in tiny
spots all over the picture the regular structure.

Click to expand...

If there are just tiny spots, does this mean the artifacts are less visible?

Member said:

It might help reduce color moire in some fixed patterns. It would
do nothing to help the detail information that is discarded and by
making the pattern random, it would create areas with very poor
performance.

Click to expand...

Agree. Details are lost forever. But Bayer sensors have more pixels for the time being. This can compensate a loss of absolute resolution, to an extent.

Actually, I started this thread just as a mental exercise of what can be done on the Bayer's side to counter the X3 superiority claim. Is there something in your opinion?

--Vladimir.

It is well know that that Stocastic Dithering/Screening can break up moire type artifacts in the area of printing. This is the process of randomizing the screening pattern. This has been using in printing for a long time. It is in effect what photographic "grain does as well." It breaks up patterns that que the human eye to a shape and as well as moire.

The following is a reference to its use in printing. http://www.lorrainepress.com/stochastic.htm

As I remember it, it is usually used when you have the ability to print very fine dots but have little or no control over the intensity of each dot.

I'm not sure the pixels in a digital camera would qualify as "very fine" yet. Plus there is the issue that they are analog in their ability to sense light.

Using a stocatic/random paterning of the filters I would think would tend to break up moire, but it would also tend to reduce the effective resolution.

Thus probably not a winner but certainly an interesting idea. (pardon the pun on "probably" as stocatic processes are base on random probability)

Karl

Vladimir Koifman said:

Reading all these discussions about X3 vs Bayer interpolation
comparisons I just thought if a random Bayer pattern can eliminate
interpolation artifacts?

As far as I can see, most artifacts are actually aliasing between
Bayer pattern and some regular structure in the picture subject.
If so, making RGB sequence random, rather than regular should
eliminate this aliasing.

Is there any drawback of this? Interpolation filter becomes more
complex, obviously. In it's simple form it might be less optimized
than the regular structure filter. On the other hand, there is no
chance the real-world picture pattern would match this random Bayer
filter, so artifacts should become much less visible.

Can this be the Bayer camp answer on Foveon's challange?

--
Vladimir.

Click to expand...

--Karl

Karl,

Thank you for the link. Judging by the samples given, stochasticaly dithered images might not have a higher resolution, but visually they are much more pleasing.

Me too not sure, if this is the net effect of Bayer filter randomization. But what's interesting in this article is that same resolution and detail, if presented differently, can be less or more pleasing.

--Vladimir.

Vladimir Koifman said:

Reading all these discussions about X3 vs Bayer interpolation
comparisons I just thought if a random Bayer pattern can eliminate
interpolation artifacts?

Click to expand...

A random pattern should get rid of moire (at least, reduce it to such
low probabilities, you will never see it in a million years), but it would
still not get rid of aliasing errors for interpolators that violate Nyquist
criterion. I.e., unless the antialiasing filters are "soft" enough, high
resolution areas (such as high lp/mm parts of test charts) will still
exhibit color noise.

Since most photographs are not test charts, what you suggest is
a very viable solution. The only question I have is how such an
image would scale.

It is always good to use random patterns. The HVS is very bad at
resolving random patterns. Stochastic screens (or visually-stochastic
methods such as Floyd-Steinberg dithering) is the preferred dithering
method for printers which has good control of pixel sizes (such as
ink-jet printers).

• kc

Vladimir Koifman said:

If there are just tiny spots, does this mean the artifacts are less
visible?

Click to expand...

Sure. The Foveon guys will tell you that they're capturing fine details better and the Bayer guys will say that their mistakes only affect tiny areas. (Note that they are saying the same thing. The first is focusing on the issue, while the second is downplaying it.)

Vladimir Koifman said:

Actually, I started this thread just as a mental exercise of what
can be done on the Bayer's side to counter the X3 superiority
claim. Is there something in your opinion?

Click to expand...

They could use multiple sensors, like some video cameras, but this would add size and expense.

They could try to maintain a large advantage over foveon in pixel count, but I suspect that this will be difficult as transistors get smaller.

They could find other uses for multilayer technology that could give them an advantage. For example, they might find a way layer more of the non light sensitive parts of the chip under the light sensitive areas, which would let them increase the amount of the surface of the chip that they are using for light collection. Of course, Foveon could exploit these advantages too, but since their design is already multilayer, it might be more difficult for them.

Of course, Foveon could crash and burn for any number of reasons, some technical, some not, and the Bayer guys could win by default.

--Ron ParrFAQ: http://www.cs.duke.edu/~parr/photography/faq.htmlGallery: http://www.pbase.com/parr/

Even if there's a chance for this to be a very good idea, it's too late now. Bayer sensors days are numbered.
--Michael

Vladimir Koifman said:

Actually, I started this thread just as a mental exercise of what
can be done on the Bayer's side to counter the X3 superiority
claim. Is there something in your opinion?

Click to expand...

Nothing can be done to save the bayer sensor.... IF the X3 is as good as it seems to be, and IF it can be made more inexpensively than a bayer sensor of similar pixel size, there is no contest and the X3 will be the future winner...

--Sincerely,Bob the Printer

Michael w. said:

Bayer sensors days are numbered.

Click to expand...

In the past there were many industry battles where seemingly inferior technology managed to improve itself and win.
Probably, the most well known is Intel

x86 procesors vs. PowerPC alliance. Even though Microsoft supported PowerPC in their Windows NT, it did not help.

Other recent examples include Transmeta vs. Intel and AMD; IOmega ZipDrive vs. SyQuest, Imation and others; Sega Dreamcast vs. Sony Playstation and more.

In all these cases the new technology essentially failed to deliver its promise in one way or another. At the same time, the old tech managed to fill its deficiencies and release new better or cheaper version.

Based on these history lessons I think it's to soon to proclaim Foveon's victory.

--Vladimir.

Yes, I believe this would get rid of moire: it's very much equivalent to non-uniform sampling in the time domain, which is sometimes done to get rid of aliasing there. You don't get owt for nowt, though, and in this case the high frequency information which causes moiree on a regular bayer pattern would end up as increased pixel to pixel noise (if of a relatively benign random nature), compared with proper anti-alias filtering or an X3 type.

An alternative, and I think the route we're likely to travel, is to make the sensor pixels so small that they're finer than the resolution of the lens(giving very conservative analogue anti-alias filtering). This will tend to make them sufficiently noisy that you then average them down (a crude for of sampling + digital low pass filtering) to a lower viewable resolution. This is in fact what occurs in practice when a high-megapixel consumer digicam is printed out at a 'normal size': if you're eye can't resolve the individual pixels, it effectively does such an averaging.

Such an approach is still left with the theoretical disadvantage over the X3 of low sensitivity (as the colour filters throw 2/3rds of the light away everywhere), but from what we've seen of the first-generation specs, I don't think the X3 is in practice any better (yet).

Vladimir Koifman said:

Reading all these discussions about X3 vs Bayer interpolation
comparisons I just thought if a random Bayer pattern can eliminate
interpolation artifacts?

As far as I can see, most artifacts are actually aliasing between
Bayer pattern and some regular structure in the picture subject.
If so, making RGB sequence random, rather than regular should
eliminate this aliasing.

Is there any drawback of this? Interpolation filter becomes more
complex, obviously. In it's simple form it might be less optimized
than the regular structure filter. On the other hand, there is no
chance the real-world picture pattern would match this random Bayer
filter, so artifacts should become much less visible.

Can this be the Bayer camp answer on Foveon's challange?

--
Vladimir.

Click to expand...

--Kingsley ReavellMSM, Cambridge