CCD and CMOS colour

[hjulenissen]

The other extreme case is three filters, each with 1 nm-wide passbands. That produces inaccurate color, too.

If there is a "preferred" filter spectral response (the most accurate model of average human beings?), then any deviation from this seems like (in itself) a drawback producing inaccurate color. I believe that:

Luckily, it doesn’t have to do that, it just has to be a 3x3 matrix multiply away from that.

1. Practical color filter design does not allow arbitrary responses, so one source of undesirable filter response would be the "noise" in settling for physically realilsable filters.

2. When designing filters, spectral sensitivity is only one requirement. Passing through many photons so as to keep luminance noise in check is another. Longevity, thickness, cost, may be other factors.

3. Further, digital color correction consists of both making colors "correct", but also in trade-offs of visible noise and banding.

Really grasping what different CFA responses + different strategies for "color correction" does for the end-to-end response i hard for many photography practitioners and technically minded people as well. What is the range of different filters that are "sort of Luther-Ives compatible",

It is infinite, as is the range of filters that is precisely LI compatible.

and why would a digital color correction sway away from "perfect" color correction?

-h

Market forces. With four dyes in the CFA you can do a lot better than with three, but that approach has not gotten any traction to speak of.

--
https://blog.kasson.com

I would think that when designing the 3x3 correction matrix, there are limits to the «spectral sharpening» that one would practically do. If the overlap is reall large i regions, the matrix would have to compensate with large off diagonal terms and risk cbroma noise or banding.

-h

 

[JimKasson]

The other extreme case is three filters, each with 1 nm-wide passbands. That produces inaccurate color, too.

If there is a "preferred" filter spectral response (the most accurate model of average human beings?), then any deviation from this seems like (in itself) a drawback producing inaccurate color. I believe that:

Luckily, it doesn’t have to do that, it just has to be a 3x3 matrix multiply away from that.

1. Practical color filter design does not allow arbitrary responses, so one source of undesirable filter response would be the "noise" in settling for physically realilsable filters.

2. When designing filters, spectral sensitivity is only one requirement. Passing through many photons so as to keep luminance noise in check is another. Longevity, thickness, cost, may be other factors.

3. Further, digital color correction consists of both making colors "correct", but also in trade-offs of visible noise and banding.

Really grasping what different CFA responses + different strategies for "color correction" does for the end-to-end response i hard for many photography practitioners and technically minded people as well. What is the range of different filters that are "sort of Luther-Ives compatible",

It is infinite, as is the range of filters that is precisely LI compatible.

and why would a digital color correction sway away from "perfect" color correction?

-h

Market forces. With four dyes in the CFA you can do a lot better than with three, but that approach has not gotten any traction to speak of.

I would think that when designing the 3x3 correction matrix, there are limits to the «spectral sharpening» that one would practically do. If the overlap is reall large i regions, the matrix would have to compensate with large off diagonal terms and risk cbroma noise or banding.

The rho/gamma cone spectral overlap is considerable, so it's not clear to me that "spectral sharpening" is always required. Unless you're talking about the spectral sharpening that you'd need to do to get to, say sRGB. You'd need to do that even if the camera filters were identical to the cone responses.

 

[bobn2]

I'm just trying to say that "weaker" or "stronger" (more dense) are the terms that don't predict colour separation well.

I remember a discussion on this same topic some years ago, and it was pointed out. by Emil Martinec as I recall, that what the 'strength' of the CFA does have an effect on is the balance between so-called 'chroma' and 'luma' noise after processing. Whilst this is not strictly a matter of colour separation, it might affect what is perceived as such in some conditions.

 

[Iliah Borg]

I'm just trying to say that "weaker" or "stronger" (more dense) are the terms that don't predict colour separation well.

I remember a discussion on this same topic some years ago, and it was pointed out. by Emil Martinec as I recall, that what the 'strength' of the CFA does have an effect on is the balance between so-called 'chroma' and 'luma' noise after processing. Whilst this is not strictly a matter of colour separation, it might affect what is perceived as such in some conditions.

It's possible to calculate a set of metameric errors against XYZ, an (sRGB/Adobe RGB) display, a printer, a projector. I think it's also possible to calculate noise introduced by colour transform. Maybe such calculations will allow us to determine what is 'strength'? But maybe if we have those calculations we can come up with a better term?

 

[J A C S]

I remember a discussion on this same topic some years ago, and it was pointed out. by Emil Martinec as I recall, that what the 'strength' of the CFA does have an effect on is the balance between so-called 'chroma' and 'luma' noise after processing. Whilst this is not strictly a matter of colour separation, it might affect what is perceived as such in some conditions.

I think that the notion of strong vs. weak CFA in these forums comes from the naïve expectation that the curves must actually separate some spectral bands, which is wrong. People eyeball the spectral curves and are very unhappy when they do not see the separation they expect. They are not so unhappy to do that with eyes having spectral responses with a significant overlap...

If we compare CFAs with channels spanning the same 3D space (say, all satisfy the LI condition), the "transfer" of noise under the linear color transformation to, say sRGB, is well understood - the variation is the original one multiplied by the sum of the eigenvalues of A*A, where A is the color matrix. Matrices with "extreme entries" but all normalized so that they would preserve the luminosity, for example, would generate more noise.

Now, this is not saying the whole truth. First, the recorded noise depends on the CFA in the first place. A "weaker" CFA would have less noise, but the image converted to sRGB would be penalized more with the color matrix. Which factor wins is not clear a priori but a "weaker" CFAs would win with DXO. The second factor is the color matrix, as I said above. The third one is that the computed noise would be in the sRGB space. Our eyes perform another transformation and it is not clear to me that a higher noise in sRGB (which noise is hue dependent) means a higher perceived noise.

 

[bobn2]

I remember a discussion on this same topic some years ago, and it was pointed out. by Emil Martinec as I recall, that what the 'strength' of the CFA does have an effect on is the balance between so-called 'chroma' and 'luma' noise after processing. Whilst this is not strictly a matter of colour separation, it might affect what is perceived as such in some conditions.

I think that the notion of strong vs. weak CFA in these forums comes from the naïve expectation that the curves must actually separate some spectral bands, which is wrong. People eyeball the spectral curves and are very unhappy when they do not see the separation they expect. They are not so unhappy to do that with eyes having spectral responses with a significant overlap...

Yes, I agree with that. It's the same fallacy that says that three sensor cameras with dichroic splitters should give 'better colour' than Bater sensors, and they don't.

If we compare CFAs with channels spanning the same 3D space (say, all satisfy the LI condition), the "transfer" of noise under the linear color transformation to, say sRGB, is well understood - the variation is the original one multiplied by the sum of the eigenvalues of A*A, where A is the color matrix. Matrices with "extreme entries" but all normalized so that they would preserve the luminosity, for example, would generate more noise.

Now, this is not saying the whole truth. First, the recorded noise depends on the CFA in the first place. A "weaker" CFA would have less noise, but the image converted to sRGB would be penalized more with the color matrix. Which factor wins is not clear a priori but a "weaker" CFAs would win with DXO. The second factor is the color matrix, as I said above. The third one is that the computed noise would be in the sRGB space. Our eyes perform another transformation and it is not clear to me that a higher noise in sRGB (which noise is hue dependent) means a higher perceived noise.

The point that Emil was making was not about the magnitude of the noise, but the balance between what is called 'luma' noise (that is, noise in which where just the lightness is changed) and what is called 'chroma' noise (where the apparent colour is changed by the noise), which is I think what you're saying that the weaker CFA being 'penalised' more.

I have to say, that my facility with this topic is not good enough to argue the case in detail, I just recall it as being an argument that was made at the time.

--
Is it always wrong
for one to have the hots for
Comrade Kim Yo Jong?

 

[bobn2]

I'm just trying to say that "weaker" or "stronger" (more dense) are the terms that don't predict colour separation well.

I remember a discussion on this same topic some years ago, and it was pointed out. by Emil Martinec as I recall, that what the 'strength' of the CFA does have an effect on is the balance between so-called 'chroma' and 'luma' noise after processing. Whilst this is not strictly a matter of colour separation, it might affect what is perceived as such in some conditions.

It's possible to calculate a set of metameric errors against XYZ, an (sRGB/Adobe RGB) display, a printer, a projector. I think it's also possible to calculate noise introduced by colour transform. Maybe such calculations will allow us to determine what is 'strength'? But maybe if we have those calculations we can come up with a better term?

That sounds to me to be an interesting approach. I think at that time the term 'strength' was being accepted without much thought as to what it might actually mean.

I would add though, I think the question was not noise introduced by the colour transform, but how the colour transform behaves with noise as a source.

--
Is it always wrong
for one to have the hots for
Comrade Kim Yo Jong?

 

[Iliah Borg]

I'm just trying to say that "weaker" or "stronger" (more dense) are the terms that don't predict colour separation well.

I remember a discussion on this same topic some years ago, and it was pointed out. by Emil Martinec as I recall, that what the 'strength' of the CFA does have an effect on is the balance between so-called 'chroma' and 'luma' noise after processing. Whilst this is not strictly a matter of colour separation, it might affect what is perceived as such in some conditions.

It's possible to calculate a set of metameric errors against XYZ, an (sRGB/Adobe RGB) display, a printer, a projector. I think it's also possible to calculate noise introduced by colour transform. Maybe such calculations will allow us to determine what is 'strength'? But maybe if we have those calculations we can come up with a better term?

That sounds to me to be an interesting approach. I think at that time the term 'strength' was being accepted without much thought as to what it might actually mean.

I would add though, I think the question was not noise introduced by the colour transform, but

how the colour transform behaves with noise as a source.

Right. That's also a question.

 

[J A C S]

If we compare CFAs with channels spanning the same 3D space (say, all satisfy the LI condition), the "transfer" of noise under the linear color transformation to, say sRGB, is well understood - the variation is the original one multiplied by the sum of the eigenvalues of A*A, where A is the color matrix. Matrices with "extreme entries" but all normalized so that they would preserve the luminosity, for example, would generate more noise.

Now, this is not saying the whole truth. First, the recorded noise depends on the CFA in the first place. A "weaker" CFA would have less noise, but the image converted to sRGB would be penalized more with the color matrix. Which factor wins is not clear a priori but a "weaker" CFAs would win with DXO. The second factor is the color matrix, as I said above. The third one is that the computed noise would be in the sRGB space. Our eyes perform another transformation and it is not clear to me that a higher noise in sRGB (which noise is hue dependent) means a higher perceived noise.

The point that Emil was making was not about the magnitude of the noise, but the balance between what is called 'luma' noise (that is, noise in which where just the lightness is changed) and what is called 'chroma' noise (where the apparent colour is changed by the noise), which is I think what you're saying that the weaker CFA being 'penalised' more.

I hinted on this with my "hue dependent" comment. If the noise is additive (and unless it is horrible, we can linearize and think that it is additive near each tonal level), it is a random vector in a 3D space, transformed by the linear transformation into another 3D vector (in sRGB). The expectation of its norm squared is the variance of the noise (its "strength") but this hides the directional dependence of the noise. That directional dependence tells us how much of this is luma, chroma, etc. The fundamental object is A*A, not its egienvalues.

The luma/chroma channels are a kind of polar coordinates, and one needs to project the noise to the luma axis and to the chroma plane. I would assume that "weaker" CFAs would generate a higher chroma noise and to have a weaker effect on the luma noise. This can be computed but one needs to know the spectral content of the scene, the WB, and what a weak CFA means.

 

[hjulenissen]

Yes, I agree with that. It's the same fallacy that says that three sensor cameras with dichroic splitters should give 'better colour' than Bater sensors, and they don't.

If standards and litterature define the «optimal» spectral selectivity of CFA in order to produce some normalized output, then I would think it to be pretty straight forward that «weak» CFA filtering means that the spectral overlap is larger than desired, and some subtraction is needed in order to produce the desired response. While «too strong» CFA filtering would be taken to mean that spectral overlap is too small, and some averaging is needed in order to get that red dress looking the way we percieve it?

As someone said, the luminance/chroma noise at output is dependent on the color correction matrix and the noise contribution from each channel. But the shape of the CFA also affects the noise in each channel and the required shape of the color correction matrix.

I am sure that someone have plugged those numbers and come up with engineering numbers like «what spectral shape minimize luminance + blue channel noise in sRGB given that we shall correct color response fully».

-h

 

[bobn2]

Yes, I agree with that. It's the same fallacy that says that three sensor cameras with dichroic splitters should give 'better colour' than Bater sensors, and they don't.

If standards and litterature define the «optimal» spectral selectivity of CFA in order to produce some normalized output, then I would think it to be pretty straight forward that «weak» CFA filtering means that the spectral overlap is larger than desired, and some subtraction is needed in order to produce the desired response. While «too strong» CFA filtering would be taken to mean that spectral overlap is too small, and some averaging is needed in order to get that red dress looking the way we percieve it?

I think the problem with this is that you've just replaced one concept that doesn't fit the situation (weak/strong) with another one - 'spectral overlap'. This is summed up in the Luther-Ives-Maxwell condition - that the spectral responsivity of the three channels should be a linear combination of the colour matching functions.

 

[D Cox]

I remember a discussion on this same topic some years ago, and it was pointed out. by Emil Martinec as I recall, that what the 'strength' of the CFA does have an effect on is the balance between so-called 'chroma' and 'luma' noise after processing. Whilst this is not strictly a matter of colour separation, it might affect what is perceived as such in some conditions.

I think that the notion of strong vs. weak CFA in these forums comes from the naïve expectation that the curves must actually separate some spectral bands, which is wrong. People eyeball the spectral curves and are very unhappy when they do not see the separation they expect. They are not so unhappy to do that with eyes having spectral responses with a significant overlap...

All that is required for recording hues and saturation is that each wavelength should give a unique combination of three numbers (one or two can be zero).

If we compare CFAs with channels spanning the same 3D space (say, all satisfy the LI condition), the "transfer" of noise under the linear color transformation to, say sRGB, is well understood - the variation is the original one multiplied by the sum of the eigenvalues of A*A, where A is the color matrix. Matrices with "extreme entries" but all normalized so that they would preserve the luminosity, for example, would generate more noise.

Now, this is not saying the whole truth. First, the recorded noise depends on the CFA in the first place. A "weaker" CFA would have less noise, but the image converted to sRGB would be penalized more with the color matrix. Which factor wins is not clear a priori but a "weaker" CFAs would win with DXO. The second factor is the color matrix, as I said above. The third one is that the computed noise would be in the sRGB space. Our eyes perform another transformation and it is not clear to me that a higher noise in sRGB (which noise is hue dependent) means a higher perceived noise.

Don Cox

 

[hjulenissen]

Yes, I agree with that. It's the same fallacy that says that three sensor cameras with dichroic splitters should give 'better colour' than Bater sensors, and they don't.

If standards and litterature define the «optimal» spectral selectivity of CFA in order to produce some normalized output, then I would think it to be pretty straight forward that «weak» CFA filtering means that the spectral overlap is larger than desired, and some subtraction is needed in order to produce the desired response. While «too strong» CFA filtering would be taken to mean that spectral overlap is too small, and some averaging is needed in order to get that red dress looking the way we percieve it?

I think the problem with this is that you've just replaced one concept that doesn't fit the situation (weak/strong) with another one - 'spectral overlap'. This is summed up in the Luther-Ives-Maxwell condition - that the spectral responsivity of the three channels should be a linear combination of the colour matching functions.

 

[JimKasson]

Yes, I agree with that. It's the same fallacy that says that three sensor cameras with dichroic splitters should give 'better colour' than Bater sensors, and they don't.

If standards and litterature define the «optimal» spectral selectivity of CFA in order to produce some normalized output, then I would think it to be pretty straight forward that «weak» CFA filtering means that the spectral overlap is larger than desired, and some subtraction is needed in order to produce the desired response. While «too strong» CFA filtering would be taken to mean that spectral overlap is too small, and some averaging is needed in order to get that red dress looking the way we percieve it?

I think the problem with this is that you've just replaced one concept that doesn't fit the situation (weak/strong) with another one - 'spectral overlap'. This is summed up in the Luther-Ives-Maxwell condition - that the spectral responsivity of the three channels should be a linear combination of the colour matching functions.

If there exists one or a somewhat limited set of «ideal» CFA responses (at least with respect to one performance metric),

Do we have such a set? What are the performance metrics for it?

it would seem that one could discuss how a given filter deviates from that ideal. While «excessive spectral overlap» might not be your preferred term, it would seem like a solvable problem.

dxo writes:

https://www.dxomark.com/glossary/color-depth/

«The channel decomposition is a low-level description of the sensor spectral response, while the color matrix is what engineers use to make the sensor react as though it had sRGB primaries.

A color matrix with large singular values yields a dramatic amplification of noise»

If you're going to measure noise, it would be better to measure it in a perceptually uniform color space, instead of sRGB.

 

[Tristimulus]

Just wondering.

The substrates for CCDs and CMOS image sensors have different spectral sensitivity from what I have seen (ESA). Seems like both kind of sensors cover the visual spectrum equally well but differ at the extremes well byond visual. So color differences based on that should be down to the filter array. So - did I get this one right (wonderful to be corrected)?

Noise patterns obviously differ between CCD and CMOS.

Any color differences because this?

 

[JimKasson]

Just wondering.

The substrates for CCDs and CMOS image sensors have different spectral sensitivity from what I have seen (ESA). Seems like both kind of sensors cover the visual spectrum equally well but differ at the extremes well byond visual. So color differences based on that should be down to the filter array. So - did I get this one right (wonderful to be corrected)?

Noise patterns obviously differ between CCD and CMOS.

Not for shot noise if spectral responses and compromise matrices the same.

Any color differences because this?

 

[J A C S]

I remember a discussion on this same topic some years ago, and it was pointed out. by Emil Martinec as I recall, that what the 'strength' of the CFA does have an effect on is the balance between so-called 'chroma' and 'luma' noise after processing. Whilst this is not strictly a matter of colour separation, it might affect what is perceived as such in some conditions.

I think that the notion of strong vs. weak CFA in these forums comes from the naïve expectation that the curves must actually separate some spectral bands, which is wrong. People eyeball the spectral curves and are very unhappy when they do not see the separation they expect. They are not so unhappy to do that with eyes having spectral responses with a significant overlap...

All that is required for recording hues and saturation is that each wavelength should give a unique combination of three numbers (one or two can be zero).

Not really. Where is the human color vision involved in this?

 

[The Ghost of Caravaggio]

...

Noise patterns obviously differ between CCD and CMOS.

Any color differences because this?

"Noise patterns" implies correlated (time-independent) noise. In many fields these are called artifacts. Correlated noise could be different for CCD and CMOS. I'm not sure how the well CCD blooming is controlled in the newest sensor technologies.

Anyway, I don't think differences in correlated noise will affect color rendering aesthetics.

 

[bobn2]

Yes, I agree with that. It's the same fallacy that says that three sensor cameras with dichroic splitters should give 'better colour' than Bater sensors, and they don't.

If standards and litterature define the «optimal» spectral selectivity of CFA in order to produce some normalized output, then I would think it to be pretty straight forward that «weak» CFA filtering means that the spectral overlap is larger than desired, and some subtraction is needed in order to produce the desired response. While «too strong» CFA filtering would be taken to mean that spectral overlap is too small, and some averaging is needed in order to get that red dress looking the way we percieve it?

I think the problem with this is that you've just replaced one concept that doesn't fit the situation (weak/strong) with another one - 'spectral overlap'. This is summed up in the Luther-Ives-Maxwell condition - that the spectral responsivity of the three channels should be a linear combination of the colour matching functions.

If there exists one or a somewhat limited set of «ideal» CFA responses (at least with respect to one performance metric), it would seem that one could discuss how a given filter deviates from that ideal. While «excessive spectral overlap» might not be your preferred term, it would seem like a solvable problem.

The problem there is with that first clause 'If there exists one or a somewhat limited set of «ideal» CFA responses'. As has been pointed out elsewhere in this thread, the set of responses which satisfy the LIM condition is infinite.

 

[bobn2]

I remember a discussion on this same topic some years ago, and it was pointed out. by Emil Martinec as I recall, that what the 'strength' of the CFA does have an effect on is the balance between so-called 'chroma' and 'luma' noise after processing. Whilst this is not strictly a matter of colour separation, it might affect what is perceived as such in some conditions.

I think that the notion of strong vs. weak CFA in these forums comes from the naïve expectation that the curves must actually separate some spectral bands, which is wrong. People eyeball the spectral curves and are very unhappy when they do not see the separation they expect. They are not so unhappy to do that with eyes having spectral responses with a significant overlap...

All that is required for recording hues and saturation is that each wavelength should give a unique combination of three numbers (one or two can be zero).

Colour is not wavelength. For instance, there is no magenta wavelength.