CFA Strength and Color Discrimination

[Iliah Borg]

Erik Kaffehr wrote: What I would ask a bit about is the importance of the left peak on the red curve. How would rendition be affected by zeroing it out?

Hi Erik, here are results with and without hump:


With Hump SMI = 99.9


Without Hump SMI = 94.4


Correlated Color Temperature of 4910 and 4908K respectively

My layman's conclusion from the work presented:

• There seems to be a strong relation between SMI and colour rendition accuracy using an optimized matrix. No big surprise really, as SMI really measures accuracy.
• It is not obvious that small overlaps are advantageous to colour accuracy, at least not in the SMI sense.
• The shape and distribution of the curves matter. That is also expected.
• It also seems that it is possible to get very high SMI for a CFA design that would approximate the SML curves for human vision.

What is your take on the "Colour Science" presented by Phase One?


The standard sensor, which I would assume describes the IQ3100 MP seems to have very high response in green and blue channels in near infrared, probably because it lacks an IR filter. But it does have the left peak on the R-channel which I have not seen on any of RIT curves I have plotted.

Would be interesting to see SMI calculated for the Trichromatic sensor. It could of course be that the curves are just "artist's vision" reminiscent of NASA's images of exoplanets and black holes colliding.

They look like artist renditions to me which, once normalized, would seem to be fairly similar to a gaussian variant in the OP.

Jack

You can try running both against a wider palette, like Finnish, or against a database of painter's pigments, which is relevant for one of the applications of MFs, reproduction.

--

About LibRaw | LibRaw

www.libraw.org

 

[JimKasson]

Erik Kaffehr wrote: What I would ask a bit about is the importance of the left peak on the red curve. How would rendition be affected by zeroing it out?

Hi Erik, here are results with and without hump:


With Hump SMI = 99.9


Without Hump SMI = 94.4


Correlated Color Temperature of 4910 and 4908K respectively

Nice work. You beat me to it, I'm working on the genetic algorithm.

My layman's conclusion from the work presented:

• There seems to be a strong relation between SMI and colour rendition accuracy using an optimized matrix. No big surprise really, as SMI really measures accuracy.
• It is not obvious that small overlaps are advantageous to colour accuracy, at least not in the SMI sense.
• The shape and distribution of the curves matter. That is also expected.
• It also seems that it is possible to get very high SMI for a CFA design that would approximate the SML curves for human vision.

What is your take on the "Colour Science" presented by Phase One?


The standard sensor, which I would assume describes the IQ3100 MP seems to have very high response in green and blue channels in near infrared, probably because it lacks an IR filter. But it does have the left peak on the R-channel which I have not seen on any of RIT curves I have plotted.

Would be interesting to see SMI calculated for the Trichromatic sensor. It could of course be that the curves are just "artist's vision" reminiscent of NASA's images of exoplanets and black holes colliding.

They look like artist renditions to me which, once normalized, would seem to be fairly similar to a gaussian variant in the OP.

As you saw, the big difference that I noticed was the large near-IR response of both the green and blue filters in the "overlap" graphs. If the left graph has a hot mirror and the right was doesn't, that is stacking the deck, and something that P1 should be ashamed of.

Jim



--

the last word the last word - Photography meets digital computer technology. Photography wins -- most of the time.

Photography meets digital computer technology. Photography wins -- most of the time.

blog.kasson.com

 

[JimKasson]

What I would ask a bit about is the importance of the left peak on the red curve. How would rendition be affected by zeroing it out?

Looking at the real world, you'd lose spectral violet, and a lot of things that look violet in nature would look blue.

I recently found this out in a very personal way:

A personal look at metameric failure - the last word

I recently had cataract surgery; two weeks ago on my left eye, and yesterday on my right. So for two weeks, I got to walk around comparing the colors with the treated versus those with the untreated eye. Cataracts are known to block short wavelengths to a greater extent than long ones, so I expected

blog.kasson.com

In my experience, CFAs tend not to have that short-wavelength read peak, and I think that's a bad thing.

Jim

 

[Jack Hogan]

Erik Kaffehr wrote: What I would ask a bit about is the importance of the left peak on the red curve. How would rendition be affected by zeroing it out?

Hi Erik, here are results with and without hump:


With Hump SMI = 99.9


Without Hump SMI = 94.4


Correlated Color Temperature of 4910 and 4908K respectively

Nice work. You beat me to it, I'm working on the genetic algorithm.

Cool. Another interesting insight comes from the matrices. Why isn't the top one the identity matrix?

I think it's because CMF curves are designed to subtend equal areas in linear energy units. This ensures that when exposed to the uniform energy illuminant Se all three result in the same value for XYZ (1/3,1/3,1/3) .

When dealing with raw data (via photons) we effectively perform area equalization through white balance (which I've done as part of the procedure in this thread). If we then applied the identity matrix we would end up in XYZ with a white point of Se. Therefore we need the matrix to move the white point to the correct position, in this case D50. Those Sums of Row Coefficients above are, within a small margin of error, D50 in XYZ.

 

[JimKasson]

Erik Kaffehr wrote: What I would ask a bit about is the importance of the left peak on the red curve. How would rendition be affected by zeroing it out?

Hi Erik, here are results with and without hump:


With Hump SMI = 99.9


Without Hump SMI = 94.4


Correlated Color Temperature of 4910 and 4908K respectively

Nice work. You beat me to it, I'm working on the genetic algorithm.

Cool. Another interesting insight comes from the matrices. Why isn't the top one the identity matrix?

I think it's because CMF curves are designed to subtend equal areas in linear energy units. This ensures that when exposed to the uniform energy illuminant Se all three result in the same value for XYZ (1/3,1/3,1/3) .

When dealing with raw data (via photons) we effectively perform area equalization through white balance (which I've done as part of the procedure in this thread). If we then applied the identity matrix we would end up in XYZ with a white point of Se. Therefore we need the matrix to move the white point to the correct position, in this case D50. Those Sums of Row Coefficients above are, within a small margin of error, D50 in XYZ.

Hmm... I'll have to think about that.

What was your thinking in minimizing the sum of the dE's instead of something that penalizes the outliers like the sum of the squares?

Jim

--

the last word the last word - Photography meets digital computer technology. Photography wins -- most of the time.

Photography meets digital computer technology. Photography wins -- most of the time.

blog.kasson.com

 

[J A C S]

No because the quantal CMFs only come into play with Jim's graphs. In the OP only the linear energy CMFs are used to determine reference XYZ values, performing all those operations in the energy domain.

But you did get 99.9 once ("bingo!") and 90 before for the same CFA. OK, not in the OP, so maybe I am missing something.

 

[Jack Hogan]

What was your thinking in minimizing the sum of the dE's instead of something that penalizes the outliers like the sum of the squares?

For this thread the objective was to maximize SMI and I thought the straight sum would do it. Otherwise your point is well taken.

 

[J A C S]

No because the quantal CMFs only come into play with Jim's graphs. In the OP only the linear energy CMFs are used to determine reference XYZ values, performing all those operations in the energy domain.

But you did get 99.9 once ("bingo!") and 90 before for the same CFA. OK, not in the OP, so maybe I am missing something.

Replying to myself - I went over the posts again (I am on a small traveling laptop now), and I can see that this was Jim, not you. Forget about what I asked.

 

[Joofa]

Jack suggested weighting the XYZ camera curves.

Kindly write the explicit formula you are using to weight the curves. If any normalization is done after weighting then write the formula for that also. Matlab line(s) would be better.

 

[JimKasson]

Jack suggested weighting the XYZ camera curves.

Kindly write the explicit formula you are using to weight the curves. If any normalization is done after weighting then write the formula for that also. Matlab line(s) would be better.

for i = [1:441]
CFA(i,2) = CFA(i,2)/ CFA(i,1);
CFA(i,3) = CFA(i,3)/ CFA(i,1);
CFA(i,4) = CFA(i,4)/ CFA(i,1);
end

Column 1 is the wavelength column. No normalization.

Jim

 

[Joofa]

Jack suggested weighting the XYZ camera curves.

Kindly write the explicit formula you are using to weight the curves. If any normalization is done after weighting then write the formula for that also. Matlab line(s) would be better.

for i = [1:441]
CFA(i,2) = CFA(i,2)/ CFA(i,1);
CFA(i,3) = CFA(i,3)/ CFA(i,1);
CFA(i,4) = CFA(i,4)/ CFA(i,1);
end

Column 1 is the wavelength column. No normalization.

Thanks.

But, why do you do this? Isn't Jack using energy formulation so wavelength mult or div is not needed?

--
Dj Joofa
http://www.djjoofa.com

 

[JimKasson]

Jack suggested weighting the XYZ camera curves.

Kindly write the explicit formula you are using to weight the curves. If any normalization is done after weighting then write the formula for that also. Matlab line(s) would be better.

for i = [1:441]
CFA(i,2) = CFA(i,2)/ CFA(i,1);
CFA(i,3) = CFA(i,3)/ CFA(i,1);
CFA(i,4) = CFA(i,4)/ CFA(i,1);
end

Column 1 is the wavelength column. No normalization.

Thanks.

But, why do you do this? Isn't Jack using energy formulation so wavelength mult or div is not needed?

Look at this post:

Loading…

www.dpreview.com

Specifically, the paragraph below the one numbered 2).

Jim

 

[Jack Hogan]

Jack suggested weighting the XYZ camera curves.

Kindly write the explicit formula you are using to weight the curves. If any normalization is done after weighting then write the formula for that also. Matlab line(s) would be better.

For everyone's reference here is the script used to generate the plots above (version with CMF-CFA shown). The 2 deg linear energy CMF file can be obtained from here. Functions colorchecker and illuminant come from OptProp. Function bsxfun multiplies or divides every column or row of an array by a single vector, element-by-element. Function cc2xyzMatrix finds the compromise 'forward' matrix from white balanced raw to XYZ.



Jim is correct, the compromise matrix finding problem has in some cases local minima/maxima and the Nelder-Mead simplex direct search used above sometimes does not find the global maximum. In any case it looks like I was stunting the algorithm by leaving a slight buffer when normalizing, which I now have removed. With Jim's identity seed I now get SMI of 100 and a clean matrix when using the quantal CMF curves as the CFA:



XYZ white point is [0.9658 1 0.8064] of CCT 4910K instead of D50's official [0.9642 1 0.8252] because here the latest recommended 2 deg linear energy CMF are used instead of 1931 data (see exact .csv file in the script). If OptProp's D50 illuminant vector is multiplied by the CMFs and normalized the result in XYZ is [0.9657 1 0.8050] - I'll call that close enough for our purposes here given interpolation and rounding errors (working with 1nm spacing between non-standard 390 and 830nm).

Jack

 

[Jack Hogan]

Having realized that the best possible CFA from a digital sensor color 'accuracy' point of view are the relevant Quantal Color Matching Functions below


Ideal Quantal CFA, before normalization

we now turn to checking what impact increasing their width has on SMI (a proxy for practical color discrimination) and effective QE (the allegedly intended benefit of deviating from the ideal).

First, though, let's normalize the Quantal CMF-derived CFA spectral sensitivity curves so that they all peak at 1 (below right). They are meant to be interpreted as linear filters such that for instance the probability of photons of wavelength 450nm making it through them is about [36 8 97]% for r, g and b respectively.


Right: Quantal CFA Spectral Sensitivity curves derived from linear energy 2 degree CMF in file lin2012xyz2e_1_7sf.csv from www.cvrl.org (left)

if we assume that we have an illuminant with a unity equi-photon output per small wavelength interval we can easily calculate effective Quantum Efficiency (eQE) of the filters by taking the integral of each curve and dividing it by 351 (the range of interest is 380 to 730nm included*). eQEs in this ideal case are 29.7, 32.2 and 16.4% for r, g and b respectively - this will be our benchmark, associated with best color 'accuracy'.

To make modeling simpler I am going to simulate the quantal CFA by gaussians as in the OP



Not bad, this combo yields an approximate** SMI of 97.9 (note the change in deltaE scale)


Peaks and standard deviation refer to r,g,b and red bump respectively



* The curves are shown from 380-730nm because that's the range of my ColorMunki Photo spectrophotometer, in case I may want to play with some of its output in the future.

** Since we discovered that my matrix finding routine may get stuck in a local maximum there is no guarantee that this is the highest SMI possible with this CFA. Absent a more advanced routine

 

[J A C S]

I do not see the widths changed. About the heights (or the normalization) - they should play no role, noise aside.

 

[Jack Hogan]

I do not see the widths changed. About the heights (or the normalization) - they should play no role, noise aside.

Yeah, that was a premature posting due to a stuck key, I had to do it otherwise I would have lost the work done. Continued below in a sec. And yes, the normalization should play no role, I am just showing it that way because I think it is easier to see the relative 'strength' of each curve.

 

[Joofa]

Jack suggested weighting the XYZ camera curves.

Kindly write the explicit formula you are using to weight the curves. If any normalization is done after weighting then write the formula for that also. Matlab line(s) would be better.

For everyone's reference here is the script used to generate the plots above (version with CMF-CFA shown). The 2 deg linear energy CMF file can be obtained from here. Functions colorchecker and illuminant come from OptProp. Function bsxfun multiplies or divides every column or row of an array by a single vector, element-by-element. Function cc2xyzMatrix finds the compromise 'forward' matrix from white balanced raw to XYZ.



Jim is correct, the compromise matrix finding problem has in some cases local minima/maxima and the Nelder-Mead simplex direct search used above sometimes does not find the global maximum. In any case it looks like I was stunting the algorithm by leaving a slight buffer when normalizing, which I now have removed. With Jim's identity seed I now get SMI of 100 and a clean matrix when using the quantal CMF curves as the CFA:



XYZ white point is [0.9658 1 0.8064] of CCT 4910K instead of D50's official [0.9642 1 0.8252] because here the latest recommended 2 deg linear energy CMF are used instead of 1931 data (see exact .csv file in the script). If OptProp's D50 illuminant vector is multiplied by the CMFs and normalized the result in XYZ is [0.9657 1 0.8050] - I'll call that close enough for our purposes here given interpolation and rounding errors (working with 1nm spacing between non-standard 390 and 830nm).

Jack

Thanks.

Though, as pointed out a number of times before, you are doing div and mult by wavelength. Which is not needed as it cancels out. This is an energy based calculation, despite you might think that it is photon based. IMHO, no need to refer to it as 'Quantal CMF' - which is not a correct idea in the first place as I see it.

See, you might have div and mult by factorial and called it Factorial CMF. Or tangential CMF, or Trignometric CMF, etc., as long as div and mult by the same entity to cancel it out!

--
Dj Joofa

欢迎回来！！请牢记本站域名！

 

[Jack Hogan]

Joofa wrote: Though, as pointed out a number of times before, you are doing div and mult by wavelength. Which is not needed as it cancels out. This is an energy based calculation, despite you might think that it is photon based. IMHO, no need to refer to it as 'Quantal CMF' - which is not a correct idea in the first place as I see it.

See, you might have div and mult by factorial and called it Factorial CMF. Or tangential CMF, or Trignometric CMF, etc., as long as div and mult by the same entity to cancel it out!

You may very well be right Joofa and I still don't have the difference straight in my mind. I am approaching this exercise from the point of view of a photographer who just has some raw data and is given some reference Lab values - nothing else (with my script I can calculate the reference Lab values and simulate the raw data, but that's just the starting point).

Raw data is proportional to photons, not energy. So if things are to work linearly they need to be referred to photons, including the CFA, and that's what I am doing with the script and the graphs. As long as folks know what they are looking at we should all be good.



Jack

 

[Jack Hogan]

So before my crash we had the Gaussian approximation to the Quantal CMF-derived CFA yielding SMI of 97.9 and eQE of 29.7, 32.1 and 16.4% for r,g,b respectively, pretty close to the ideal (peaks are at 596, 557, 446nm throughout this post).



I played a bit with the widths before deciding to just concentrate on the green channel only. Here is what the sequence looks like if the standard deviation of the green channel is varied between 35 and 65nm with the others unchanged:


Only the green filter stDev was varied, the others were left at the original ideal value

Naturally as the standard deviation is increased more photons will make it through and the eQE of the green filter improves. The improvement in eQE from peak SMI (eQE = 32.1%) when SMI drops below 88 * (eQE = 37.1%) is about 0.2 Stops. That's from the green filter only. So if we assume that after some tweaking we would be able to get a similar improvement from the other two filters, we may indeed get an increase in the overall signal of 0.2 stops in the raw data. Here is one attempt at increasing the standard deviation of all three filters:



The improvement here is about 0.3 stops compared to the benchmark, mainly due to the blue channel. One would need to play a lot more with the variables to see what happens but if these are the improvements that we are talking about my feeling is that in these days of clean sensors I would gladly give up 0.3 stops sensitivity for optimum color. In fact, as a landscaper, I really do not care for additional sensitivity

Jack

* I used an SMI of 88 as the threshold because I went back and calculated the SMI of my own Nikon D610 with a matrix fine tuned to yield the best deltaE, not deltaE2000. This is the method relevant to this thread. You guessed it, SMI was 88.

 

[Joofa]

Joofa wrote: Though, as pointed out a number of times before, you are doing div and mult by wavelength. Which is not needed as it cancels out. This is an energy based calculation, despite you might think that it is photon based. IMHO, no need to refer to it as 'Quantal CMF' - which is not a correct idea in the first place as I see it.

See, you might have div and mult by factorial and called it Factorial CMF. Or tangential CMF, or Trignometric CMF, etc., as long as div and mult by the same entity to cancel it out!

You may very well be right Joofa and I still don't have the difference straight in my mind. I am approaching this exercise from the point of view of a photographer who just has some raw data and is given some reference Lab values - nothing else (with my script I can calculate the reference Lab values and simulate the raw data, but that's just the starting point).

Lets get a few thing straight in the start. Firstly, these topics are not "view of a photographer" - not sure which photographer has been stopped from taking good photographs thinking that it is photon or energy based raw number. Secondly, lines 16, 18 in your script just mult and div, and cancel out. You might as well take these mult / div out to make the script a bit faster. But, then you won't be able to claim the so-called 'quantal CMF'

Raw data is proportional to photons, not energy.

The number you are calculating is not a raw number. What you are calculating is the XYZ in energy units. One can calculate XYZ in photon units. But, before we even go there one must ponder that the actual raw number in a digital camera (not the one you are calculating) is the same whether you think of the impinging EM intensity in energy units or photon units. So what is going on?

I shall give you a starting point along the lines that my thinking is aligned. Presented below are normalized energy-based and so called 'quantal-based' CMF's, which I derived from XYZ CMFs. They seem to be pretty close to each other.


Normalized energy-based and 'quantal-based' CMFs.

So if things are to work linearly they need to be referred to photons, including the CFA, and that's what I am doing with the script and the graphs.

Your script is doing energy-based calculations. I can't emphasize it more, having done that a number of times before.

As long as folks know what they are looking at we should all be good.

As long as folks don't introduce jargon because it seems catchy.

--
Dj Joofa
http://www.djjoofa.com