My understanding of Foveon & Bayer

Riaan Jutte

Leading Member
Messages
735
Reaction score
0
Location
Colchester, UK
Just trying to work out a layman's view of the differences between Foveon and Bayer of the same pixel count.

Bayer: If you take 14mp and arrange it on a flat layer, you maximise your spatial data by ignoring around 66% of the colour data. (You can capture only 1 colour value per pixel location)

Foveon: Here the 14mp is arranged in 3 layers, so you are maximizing your colour data by ignoring around 66% of the spatial data. (You can capture only 1 luminance value per pixel location)

Or as Bob Atkins put it so succinctly : "Foveon sensors have to do spatial interpolation. Bayer sensors have to do color interpolation. They both have their good and bad points. There's no free lunch".
 
Just trying to work out a layman's view of the differences between
Foveon and Bayer of the same pixel count.

Bayer: If you take 14mp and arrange it on a flat layer, you
maximise your spatial data by ignoring around 66% of the colour
data. (You can capture only 1 colour value per pixel location)
Correct.
Foveon: Here the 14mp is arranged in 3 layers, so you are
maximizing your colour data by ignoring around 66% of the spatial
data. (You can capture only 1 luminance value per pixel location)
Wrong. Stop thinking of 14 mp, and instead think of 4.7 MP x 3. Then you're capturing all the color information and spatial information for this array of pixels.
Or as Bob Atkins put it so succinctly : "Foveon sensors have to do
spatial interpolation. Bayer sensors have to do color
interpolation. They both have their good and bad points. There's no
free lunch".
This statement is misleading and nearly opposite. In a Foveon chip, there is interpolation done to determine the color of a physical pixel (one of those 4.7MP x 3). There is no spatial interpolation at all (unless you upsize the resulting file to compare to a bayer).

In the Bayer system, there is color interpolation that is done with the "penality" of having to use spatially different sites to gather that color information. In addition, there is the spatial penality of a blur filter on top of the sensor (for most CFA cameras).

The bottom line is Foveon pixels are "clean"---perfect high quality pixels with full color data. Bayer pixels are messy, because they share data from their neighbors because of the color interpolation and the AA blur filter. How those pixels and files behave is what these endless nausiating debates are about. If you took the pixel definition as photosites, and compared the two systems---the bayer would outresolve the Foveon system. The Foveon system would have much cleaner edges and better microcontrast (it always does).

The good news is that there's no real theoretical reason you can't get a 10MP 1.5 crop chip with X3 goodness. Having perfect pixels. That's a lot of data coming off a chip, and the sensor might really show you how bad your lenses are---but there's no theoretical limitation in the Foveon technology with regards to scaling.

--
Jim
 
Jim, you are a very kind and patient man.

;)
Just trying to work out a layman's view of the differences between
Foveon and Bayer of the same pixel count.

Bayer: If you take 14mp and arrange it on a flat layer, you
maximise your spatial data by ignoring around 66% of the colour
data. (You can capture only 1 colour value per pixel location)
Correct.
Foveon: Here the 14mp is arranged in 3 layers, so you are
maximizing your colour data by ignoring around 66% of the spatial
data. (You can capture only 1 luminance value per pixel location)
Wrong. Stop thinking of 14 mp, and instead think of 4.7 MP x 3.
Then you're capturing all the color information and spatial
information for this array of pixels.
Or as Bob Atkins put it so succinctly : "Foveon sensors have to do
spatial interpolation. Bayer sensors have to do color
interpolation. They both have their good and bad points. There's no
free lunch".
This statement is misleading and nearly opposite. In a Foveon chip,
there is interpolation done to determine the color of a physical
pixel (one of those 4.7MP x 3). There is no spatial interpolation
at all (unless you upsize the resulting file to compare to a bayer).

In the Bayer system, there is color interpolation that is done with
the "penality" of having to use spatially different sites to gather
that color information. In addition, there is the spatial penality
of a blur filter on top of the sensor (for most CFA cameras).

The bottom line is Foveon pixels are "clean"---perfect high quality
pixels with full color data. Bayer pixels are messy, because they
share data from their neighbors because of the color interpolation
and the AA blur filter. How those pixels and files behave is what
these endless nausiating debates are about. If you took the pixel
definition as photosites, and compared the two systems---the bayer
would outresolve the Foveon system. The Foveon system would have
much cleaner edges and better microcontrast (it always does).

The good news is that there's no real theoretical reason you can't
get a 10MP 1.5 crop chip with X3 goodness. Having perfect pixels.
That's a lot of data coming off a chip, and the sensor might really
show you how bad your lenses are---but there's no theoretical
limitation in the Foveon technology with regards to scaling.

--
Jim
--
Chunsum.

'See with your eyes and shoot with your heart.'
~ Georges Noblet, UK 2005.

http://www.chunsum.com
http://www.pbase.com/chunsum
http://www.pbase.com/sigmadslr/chunsum_choi
 
The real Jim is a mean, egocentric ba$tard...

:-)

If we can brighten on person's day with a piece of illuminative wisdom---imagine what a wonderful world this would be..

--
Jim
 
Foveon: Here the 14mp is arranged in 3 layers, so you are
maximizing your colour data by ignoring around 66% of the spatial
data. (You can capture only 1 luminance value per pixel location)
Wrong. Stop thinking of 14 mp, and instead think of 4.7 MP x 3.
Then you're capturing all the color information and spatial
information for this array of pixels.
I don't get this, I understand you are capturing all the colour information at each pixel location, but how can you also capture 3 different bits of spatial information at each pixel location? In that case you should have a file of the same X by Y as a Bayer file. Perhaps you don't understand or disagree what 'spatial information' means? Spatial information is luminance or black & white data, there is only one value at each specific pixel location.
 
This statement is misleading and nearly opposite. In a Foveon chip,
there is interpolation done to determine the color of a physical
pixel (one of those 4.7MP x 3). There is no spatial interpolation
at all (unless you upsize the resulting file to compare to a bayer).
Of course, that is the whole point, about printing. If you want to print a Foveon 14mp image to the same DPI as a 14mp Bayer you have to do spatial interpolation on the Foveon data.
 
The sensor in the SD14 has:
2652 x 1768 = 4,688,736 "red" photosites
2652 x 1768 = 4,688,736 "green" photosites
2652 x 1768 = 4,688,736 "blue" photosites

Each photosite's sample runs through a 12-bit A/D converter, giving a total of 14,066,208 samples.

Because of the quantum physics behind how the X3 sensor can have 3 sensors on top of each other, the raw data doesn't 100% correspond to a normal RGB colour-space - on average blue light gets captured by the "blue" photosite, but it's not 100%, and so on. Fixing this is a pretty simple calculation and you can see the colour conversion matrix on page 4 here:
http://www.alt-vision.com/documentation/5074-35.pdf
(the SD14's sensor might have slightly different)

To create a 2652 x 1768 colour computer image from the sensor is actually quite simple - AFAIK, you just have to do the colour conversion and that's it.

The best thing about such an image is that it would be very close to perfect. No light is "filtered" by the sensor which means no values have to be "guessed". The sensor does not need an anti-alias filter (blur filter). The only real problem is when few photons hit a particular vertical sensor group (ie dark areas) or when one or two of the photosites is subject to signal clipping (bright colour areas) - the colour conversion matrix doesn't work quite as well. This results in more colour noise for dark parts of the image, or slightly odd colours in over-exposed colour areas. However, the luminescence noise is not affected by the colour conversion matrix.

If you want to turn the 2652 x 1768 colour computer image into a 4608 x 3072 (14,155,776) image, or what the SD14 will give you if you save a max resolution JPEG image, then any decent upscaling algorithm will do. But that's purely a software thing.

Personal opinion time: if the world only had Foveon style vertical sensor groups and no Bayer sensors, and then someone tried to introduce a Bayer sensor, I think the Bayer sensor people would get a LOT more flak than Foveon currently get. Basically, the Foveon style is much more intuitive and simple to explain to normal people (the physics behind the actual sensor is rather more complex though) while Bayer is not particularly intuitive to explain while the physics behind the actual sensor are a lot simpler.

A typical 4608 x 3072 Bayer sensor would have:
3,538,944 red photosites
7,077,888 green photosites
3,538,944 blue photosites

Note that the Foveon sensor would have 32.5% more red and blue sensors, and 0.66x the number of green sensors. While the Foveon has balanced colour accuracy, the Bayer is unbalanced.

To create a 4608 x 3072 colour computer image from the sensor requries using a demozaicing algorithm - which can get very complicated and there's a lot of research going into such things. Putting it another way, that 4608 x 3072 image needs 10,616,832 red values, 7,077,888 green values and 10,616,832 blue values to be guessed.

Many of those guesses will be wrong and occur mostly in areas of sharp transition. In other words, the accuracy is variable. It would be particularly bad in areas with little green light as well. In addition, to try to counter the more noticible image defects, an anti-alias filter is used which further hurts the low level accuracy.

This is why it is inherantly impossible to reduce the resolution of a Foveon sensor vs a Bayer sensor to a single number such as "megapixels" - the relative difference in image quality is entirely scene dependant.

PS To save repitition I've decided to save this write-up here:
http://www.aceshardware.com/chris/vcgVbayer.html

--
New-ish SLR user and Sigma owner in London.
See my profile for my equipment list
 
Foveon: Here the 14mp is arranged in 3 layers, so you are
maximizing your colour data by ignoring around 66% of the spatial
data. (You can capture only 1 luminance value per pixel location)
Wrong. Stop thinking of 14 mp, and instead think of 4.7 MP x 3.
Then you're capturing all the color information and spatial
information for this array of pixels.
I don't get this, I understand you are capturing all the colour
information at each pixel location, but how can you also capture 3
different bits of spatial information at each pixel location? In
that case you should have a file of the same X by Y as a Bayer
file. Perhaps you don't understand or disagree what 'spatial
information' means? Spatial information is luminance or black &
white data, there is only one value at each specific pixel location.
This is the part that you don't get. You're not "capturing" 14 Megalocations with the Foveon chip. You're capturing 4.7 Megalocations in full color, without any blur.

In a 14MP Bayer, you're capturing 14 Megalocations---sort of. The data comes out blurred because of the demosiacing algorithm AND the anti-aliasing. So those 14 Megalocations are somewhat smeary.

--
Jim
 
how can you also capture 3 different bits of spatial information at
each pixel location?
You don't. It's as simple as this: With Bayer, you capture one of three possible color channels at each spacial location, either red, green or blue. With Foveon, you capture all three color channels, red, green, and blue, at each and every spatial location. By way of comparison, the 3.4 megapixel output from the SD10 is observed to be about the same quality as the output from 6 to 8 megapixel Bayer-sensor cameras. If this holds for the SD14 (no reason why it shouldn't, but we just haven't seen any samples yet) then the SD14 output should be about the same quality as the output from a 10 to 12 megapixel Bayer-sensor camera.
 
This statement is misleading and nearly opposite. In a Foveon chip,
there is interpolation done to determine the color of a physical
pixel (one of those 4.7MP x 3). There is no spatial interpolation
at all (unless you upsize the resulting file to compare to a bayer).
Of course, that is the whole point, about printing. If you want to
print a Foveon 14mp image to the same DPI as a 14mp Bayer you have
to do spatial interpolation on the Foveon data.
That is exactly the point. What works "better" for prints? A 4.7 MP Foveon image or a 14 MP Bayer image? I can tell you that the bayer image will be able to outresolve fine lines at this ratio (assuming good lenses, no motion blur), but the foveon image will always have "crisper" edges (hair won't be mush, microcontrast will be better). The above hypothesis is currently impossible to test (the SD14 isn't on the market, and there is not 14 MP imager that I know of in a CFA chip), but you should be able to compare against the Canon 400D, the Nikon D80 and D200.

From a printing standpoint, the Foveon files are easier to work with and have higher enlargability due to their "better" pixels. You can always add blur effects to jagged "aliasing" (which practically I don't do, because my printer handles this fine)---but you can never unmush CFA mush.

--
Jim
 
let it put it the way I understand it. may be it will help because I have no engineering back ground. (sorry guys :))

Ok here we go.

the Foveon concept is a 3 layer imager whic captures each of the primary color from each layer. So take SD10 for example, 3.40X3 would tranflate to 3.4 of R, 3.4 of G, 3.4 of B. so you have a TOTAL of 10MP of information. the thing with the Foveon is that the final "optimal output image" is put together by using the RGB value of each "spot"(let's not us pixel here) on the image. yes, that would give you a 3.4mp image but an image made of 10mp of information. more importantly 10mp of color information which I think is call chroma.

you were asking if the SD14's sensor captures 14mp of spatial/BW data. YES it does, it captures 14mp of Spatial/BW data in 3D while an Bayer Array does it in 2D.

there, I've reached my limits. hope this help or maybe someone like JL can jump in to correct me. ;)

--
Chunsum.

'See with your eyes and shoot with your heart.'
~ Georges Noblet, UK 2005.

http://www.chunsum.com
http://www.pbase.com/chunsum
http://www.pbase.com/sigmadslr/chunsum_choi
 
jakepatterson wrote:
, the 3.4 megapixel output from the SD10 is observed
to be about the same quality as the output from 6 to 8 megapixel
Bayer-sensor cameras.
there's a LOT of argument about that 6 to 8 megapixel business. The output from my SD10 absolutely outclasses the output from all the 6 megapizel Bayer-sensor cameras I've used... I just tell people who ask what my large camera is..." it's a different type sensor, equivalent to about a 10 MP camera." [I iniitally wrote "knocks the socks off" instead of outclasses, but then I thought of translations of colloquial expressions. See the marvelous Japanese article translation of how the SD14 performs]

If this holds for the SD14 (no reason why it
shouldn't, but we just haven't seen any samples yet) then the SD14
output should be about the same quality as the output from a 10 to
12 megapixel Bayer-sensor camera.
we'll see soon and then non-Sigma SD14 users will argue about it for years to come. My prediction.
Best regards, Sandy
[email protected]
 
Just trying to work out a layman's view of the differences between
Foveon and Bayer of the same pixel count.
If you really want to start understanding, you need to go beyond pixel counting. You need to start thinking about MTF and resolution. I suggest reading the stuff at http://www.normankoren.com
Bayer: If you take 14mp and arrange it on a flat layer, you
maximise your spatial data by ignoring around 66% of the colour
data. (You can capture only 1 colour value per pixel location)
Yes. Just remember that virtually every Bayer camera has an AA filter in front of it to make sure that detail much smaller than two pixels widths doesn't get through. This is necessary in order to keep the sensor from doing crazy things with color when detail is very fine. So the 1 color per pixel location is correct, but it doesn't mean you get one pixels's worth of resolution per pixel location. You get a bit less.
Foveon: Here the 14mp is arranged in 3 layers, so you are
maximizing your colour data by ignoring around 66% of the spatial
data. (You can capture only 1 luminance value per pixel location)
You can derive a luminance value from the three captured color values, yes.

The 66% number is not a good comparison because the Bayer sensor doesn't get 100% of the spatial resolution implied by its photosite count.

a) Bayer sensors have no true "luminance" sensors. Luminace data, just like color data, is interpolated/constructed with the demosaicing algorithm.

b)The blur filter in front of the sensor limits spatial data before the sensor has a chance to record it.

The net result is that DSRL sized Bayer sensors typically record about 80% of the luminance spatial resolution that you'd expect from just counting pixels(70% is typical for smaller sensors). For instance, the D2X gets 84% of the implied resolution on the horizontal and 70% on the vertical. If we calculate that for the area, we end up with 56% of 12.2Mp or 7.2MP.

In other words, be careful about suggesting that the Foveon "ignores" spatial resolution without considering how a Bayer sensor "ignores" spatial resolution. Once you've adjusted for that, then you can begin to start comparing some of the more subtle issues - like base MTF response vs. sharpening vs. chrominance resolution vs. artifact charicteristics and so on.
Or as Bob Atkins put it so succinctly : "Foveon sensors have to do
spatial interpolation. Bayer sensors have to do color
interpolation. They both have their good and bad points. There's no
free lunch".
That's backwards. It is also too succinct for me. Here's the way I'd put it.

Counting pixels can be misleading. Stop doing it. Look at resolution charts, MTF responses and actual images instead. And if you had to pick just one thing, look at images.

--
Jay Turberville
http://www.jayandwanda.com
 
Yes, that's completely true. If you want 14 MP from 3x4.7 MP Foveon, you'll have to do spatial interpolation. If you want 14 MP of color information from a 14 MP Bayer sensor, you'll have to do color interpolation.

The problem here is - human eyes have a much greater spatial than color resolution. Hence, you're more susceptible to detect artifacting from a spatial interpolation then artifacting from color interpolation.

BUT... that's not the whole story, because color interpolation isn't the only thing that happens during a bayer demosaicing. There are certain types of spatial interpolation happening as well (because sites sensitive to different wavelengths aren't colocated)!

So, in fact, getting a 14 MP photo from a 3x4.7 MP Foveon requires interpolating spatial information, while getting a 14 MP photo from a Bayer requires interpolations (of different kinds) of BOTH spatial AND color information.

--
--------------------------------------------
Ante Vukorepa

My (perpetually) temporary gallery can be found here -
http://www.flickr.com/photos/orcinus/
 
Rather than trying to compare Bayer and Foveon images directly I have focused more on trying to describe why I like what I see when I am looking at a Foveon image. Yet how can we define this quality without making it sound mysterious? I think I have found a word that may help. It is verisimilitude—the quality of appearing real.

I like this term because it leaves open why an image might have this quality, it leaves open the issue of whether or not other sensors/cameras can have it, etc. (I believe the reason Foveon cameras have this quality is not mysterious, but is rooted in the specific nature of the way the Foveon sensor gathers light and the firmware/software that processes it. Actually, there have been some interesting and useful debates over various quantifiable elements that contribute to the quality of the Foveon based images. But perhaps other approaches can produce it as well. I do suspect the Bayer approach is at a disadvantage not just because of the well discussed need for AA filters and interpolation, but, as Sandy F pointed out, the Bayer images are not uniform in the effect they produce. Even when they have higher resolution in some areas in others they fail us and fall back into incoherent mush. ( This will get better, but the variability may always be sensed and that hurts the sense of verisimilitude I prize). I like the fact that even "beyond nyquist" the Foveon sensor still gives me some useful information (that, in the notorious 9 to 5 lines example, I still am given the fact that there are "lines there.") I suspect that these and perhaps other qualities help to create this sense of verisimilitude of these images.

The fact that people do experience this quality while looking at pictures produced by the SD10/9 does not mean that everyone will see this quality in the Foveon based images (or at any rate believe they have it any more than other sensor/cameras.) The eye/brain combination allows for all sorts of judgments, some operating at the level of immediate perception, and each of us may use different clues and cues for what appears real to us.

Let me give two examples.

I have a nephew who, when he sees jaggies or noise is instantly turned off. In his case he does not see the Foveon images as having verisimilitude, because he occasionally sees a jaggy. He prefers the smoothed images his 20D produces.

I am turned off by the over-smoothed images created by some sorts of processing of images (and not just Bayer processing, I do not like what certain noise-reduction programs such as Neat-Image do to images.) So in my case the 20D images lack a certain liveliness or verisimilitude, despite the fact that I can agree they are fine images of their type.

There are other elements I suspect contribute to the verisimilitude of Foveon images. There is the subtle shifts in the shades and intensities of color, and the fine gradations of light, I see in the Sigma camera images. There is a clear demarcation of edges (and the stair-stepping, in the rare cases I sense it, does not condemn the whole picture for me unless it intrudes too much). I suspect there are others I do not notice as well.

I like the end results. I had been trying to capture and convey the feel of sunlight filtered down through trees in a forest for years with my Nikon 35mm film camera, and my first Sigma SD9 finally let me do it in the way I had been trying to capture. When I posted my first forest and falls picture someone commented;: “I feel as though I am there.” I have an A1 print (thanks to Laurence and Dominic) of one of my SD10 Nay Aug falls pictures in my office and the other day someone looked at it and said, “I can hear the water running.”

Which is why, in all of these debates, I start and end with the advice to look at the pictures. It does not end the debates for all, but it may give you a clue as to why some of us are so happy with these cameras. And I expect the SD14 and DP1 will produce the same feel (plus higher resolution, lower noise at higher iso, and a better body).

Pete
 
Basically....

Riaan is right.

Obviously, If you're printing a 4.7mp Sd14 print at a resolution of 4.7mp, there is no interpolation compared to a 4.7 MP Bayer.

But nobody will be comparing the Sd14 to a 4.7mp bayer, but rather to 10-14mp bayer imagers. One could just as easily downconvert 5D or 400d images to 4.7 and compare them appropriately to the Sd14.

The simple fact is that the extra resolution is only of practical use for enlargements in which case the Foveon image will be spatially interpolated. Now of course it will interpolate VERY well, and in my own usage, a sharp Sd9 picture usually looks better than an 8mp bayer when upscaled - but both are upscaled; one with a chrominance filling bayer algortihm, and the other with a luminance filling bicubic algorithm.

So if you're comparing apples to apples by keeping the number of luminance points constant between both imaging formats, the foveon is the clear winner. If however you're comparing a a 4.7mp foveon to a 14mp bayer- well, not so clear.

There are however, advantages to the foveon format that go beyond resolution which are my primary reasons for investing in the technology.

http://forums.dpreview.com/forums/read.asp?forum=1027&message=20264524 )

The detail of the Sd14 should be better than 12MP, but there's alot more to a good picture than luminance points.
 
Hi Ante,

your argumentation about spacial resolution would be true if the pixels on Bayer sensor wouldn't be covered by color filters. There are some assumptions like "the eye is much more sensitive to luminance information than to colors" which are used to design the sensors like they are BUT this has not to be necessarily true.

I learnt at biology some 30 years ago that in dim light we don't see colors well while in bright light it is opposite. Probably another truth is that you can hardly distinguish very different colors side by side but the eye is very sensitive to similar colors nearby. My impression is that as soon there are colors the luminance information is of less importance. Furthermore the eye is not able to recognize absolute colors - a recognized color tone changes with surrounding colors. This somehow exclusive behaviour could be an explaination that we accept color pictures transformed into B&W.

You see there are many parameters which should be pressed into one formulae. As an end user I only care about output. All the technical stuff could be true or not - rely on your eyes - they can see the difference but technological religiosity will only blind you.

"In the end it doesn't matter how you cooked it but how it tastes. Without good material the best cook cannot glance. There are too many parameters and in the end the result is what matters."

The industry developped an accoustic model for audio compression: mp3. Even at 192kbps I don't like it at all. It is inferior to the CD by quite a margin. It is even worse listened to with a good hifi set. And even worse - this accoustic model is developped for a young people's good ear. The elder you get the worse it sounds...

You should understand that there are assumptions which allow for optimisations. Once optimised it limits the quality even if technology gets better.

Everybody's eyes are different and the reception of a picture is a eye-brain effort. How good you see can be probably trained but can also be limited as some people are blind for colors. It doesn't hurt if there is too much information which cannot be resolved but it is by far less enjoyable if there is not enough.

In the end: rely on your eyes. Experiment yourself and validate your assumptions. You can get raw files from people and use Lightroom for reveiling the differences. There is no absolute truth but with experimenting you can get at least a better understanding - if this is really important for you.

Regards

Wolfgang
 

Keyboard shortcuts

Back
Top