Many questions warrant many (long) answers
Hi Frans,
I apologize in advance for the ridiculous number of questions below. As you can tell I am quite interested in this subject these days. Any help would be greatly appreciated
Of course, the expected increased resolution at blue wavelengths (lower diffraction) can be countered by focus (i.e., green is more in focus than blue), which appears to be the case with these D610 images. I do have actual D7000 photos where the blue channel produces slightly higher resolution than the green.
I never thought of it in such depth. Of course though: Spherical and Chromatic Aberrations + Diffraction would account for the differences and some would even out across channels. Does
your model account for the Aberrations? I'd be interested to see the mathematics behind it in a future blog post
No, unfortunately I am only modelling diffraction and photosite aperture, with the optional inclusion of a 4-dot OLPF. I have not yet read up anything on spherical aberrations, but I suspect that they will be hard to include in my current rendering algorithm. I suspect that a ray tracing approach would be required; I have actually considered this, but where would I obtain sufficiently accurate parameters for the lenses --- you would have to know the exact optical formula for a given lens.
Chromatic aberration is straightforward to simulate with mtf_generate_rectangle: simply change the magnification for the three channels, and/or add an offset. For example, you could render three channels like this:
./mtf_generate_rectangle --b16 -n 0 -d 100 -x 1 -y 1 -p airy-box --lambda 0.63 -o ca_red.png
./mtf_generate_rectangle --b16 -n 0 -d 100 -x 0 -y 0 -p airy-box --lambda 0.55 -o ca_green.png
./mtf_generate_rectangle --b16 -n 0 -d 100 -x -1 -y -1 -p airy-box --lambda 0.45 -o ca_blue.png
These images are then combined into a Bayered mosaic (I have a little program for that --- maybe I should make a package that combines the Bayer mosaic and DNG creation steps?), and then passed through my hacked makeDNG tool. This gives us the following image:
Bayer Mosaic of the three ca_*.png images, scaled down to 8 bits for display here. This image is exactly what dcraw -D gives you
Passing this through dcraw (without any further options) gives us this:
Maybe the WB is not ideal, but you can clearly see the red/blue fringes.
So we can now try three experiments:
a) pass the Bayered mosaic image through MTF Mapper as-is. Since the original images generated by mtf_generate_rectangle all had the same intensity range, they are already perfectly white balanced (the zippering you see in the gray Bayer image above is the simulated CA colour fringes).
b) pass the demosaiced image (dcraw -A 80 80 72 72 -6 ca.dng) through MTF mapper.
c) pass the Bayered mosaic image through MTF Mapper using the "--bayer" options.
So here goes:
Option a): passing the mosaiced image (comparable to dcraw -D) through MTF Mapper as-is
Option b): demosaiced image using dcraw
Option c.1): mtf_mapper --bayer blue
Option c.2): mtf_mapper --bayer green
Option c.3): mtf_mapper --bayer red
By the way, the images simulated with mtf_generate_rectangle (above) have the following expected MTF50 values: red=0.309194, green=0.337129, blue=0.377227)
This sequence of images demonstrate the following:
1) In the presence of significant CA (about 1 pixel shift in R/B in both x and y), the only accurate method for measuring MTF50 is to use the "mtf_mapper --bayer" modes.
2) The white-balanced mosaiced image (i.e., dcraw -D) produces more consistent results across the four edges, but all four values are far too low.
3) The demosaiced image (dcraw defaults to AHD, it seems) suffers from both inaccurate values, and large variations.
The relatively short edges (100 pixels) would lead to a small number of samples for --bayer red and --bayer blue (only 25 pixels, effectively), but this does not lead to large inaccuracies in this example because sensor noise was suppressed ("-n 0" option when generating the synthetic images).
In short, unless you know that CA is non-existent, it seems safer to use the "dcraw -D" image followed by "--bayer red/green/blue".
Am I correct in assuming that MTFMapper uses a maximum of 400 pixels even if it is fed more?
So here is the trade-off: we know how noise affects MTF measurements, i.e., the mean value over many individual measurements is unbiased, but the deviation from the mean can be quite large for any single measurement (at high noise levels). A difference between the blue and green (for example) focal plane position relative to the sensor would be systematic, i.e., it would not decrease with repeated measurements. This means that an un-demosaiced image (dcraw -D) followed by white balancing will produce an edge that is a blend of three individual edges (red, green and blue).
I see, that's an interesting way of looking at it: the same physical edge sampled four times at different sampling rates and observed wavelenths - then averaged together.
[ATTACH alt="A "perfectly white-balanced" mosaiced bayer image passed through MTF mapper. Same process as above, except that no CA was introduced"]446137[/ATTACH]
A "perfectly white-balanced" mosaiced bayer image passed through MTF mapper. Same process as above, except that no CA was introduced
It seems that in this case (no noise, etc.) the green channel dominates the result. (Incidentally, if we use the --bayer red/green/blue options on the above image, we get exactly the same result as above, as we would expect). In other words, if there is no significant CA, then the "dcraw -D followed by WB" method is very similar to the "--bayer green" method, which is a good thing
However, by thinking of it as 'blending' the four edge images together aren't we giving up some of the spatial resolution information intrinsic in the fact that we know where each sampled 'edge' is positioned wrt to the others? Ignoring wavelength specific effects for a moment and assuming that the neutral subject is 'uniformly' illuminated (D50 is good enough for this discussion, I think), isn't a white balanced dcraw -d file as precise a reconstruction of the light intensity at the scene as possible with a CFA - at full sensor resolution, approximating the results expected from a monochrome sensor (albeit with a lower base ISO)?
Ok, at this point I will have to reveal my secrets. When you use MTF Mapper's --bayer options I cheat a bit. The first phase of the slanted edge method, i.e., finding the edge location and orientation, simply pretends that we are dealing with a grayscale image. I do this because good demosaicing algorithms are slow, and the fast ones produce even less accurate edge orientation and position estimates (I tested this component individually to come to this conclusion). If you have significant CA, this means that your edge location will be estimated mostly based on the green channel, which would mean that your extracted PSF will not be centred perfectly. Fortunately, the FFT applied to the PSF to obtain the MTF is not sensitive to this shift at all. Unfortunately, I do perform some apodization using a Hamming window (IIRC), which may introduce a tiny bit of sensitivity to exact edge location. Either way, the edge orientation has a far greater impact on MTF accuracy, and as far as I can tell (or remember) the orientation is extracted just fine when treating the Bayer mosaic image as a grayscale image.
Summary: I doubt that edge position is a major factor in accuracy. The "blending" of the three edges (R,G,B) has more of a "broadening" effect on the edge transition area, which lowers MTF, just as illustrated above in the simulated CA experiment. In the absence of CA, green appears to dominate.
Would this dcraw -d undemosaiced, white-balanced approach (see underlining below) result in a more precise Edge Spread Function than looking at the individual channels?
No. Based on various experiments while developing MTF Mapper, the single most important step is accurate edge orientation estimation (which is why MTF Mapper will combine parallel edges, if possible, when square targets shapes are used). Normally, the accuracy of edge orientation estimation would be tied to the overall edge length, with ~25 pixels as a rough lower limit for reasonable results. If we assume the sensor noise is Gaussian (which is true enough for our purposes), then we can assume that the noise will show up with roughly equal magnitude at all frequencies in our MTF curve. The trick is that we usually have very little signal at higher frequencies (i.e., above Nyquist), so the signal-to-noise ratio at the lower frequencies is actually quite good. In addition, if we compute MTF50, then we simply do not care about the noise that ends up above Nyquist anyway.
Summary: since our edge orientation estimation is performed on all photosites (regardless of CFA channel), I would predict that the "--bayer" option in MTF Mapper is able to extract the benefit of a longer edge (say, 100 pixels regardless of CFA channel) even when only using the red or blue channel (effectively only 25 pixels along edge) to compute the per-channel MTF. The smaller number of samples will still produce a poorer signal-to-noise ratio, but the overall error might be manageable. I think I should test this --- maybe a future blog post.
The trade-off is that the full resolution -d white balanced image offers up to four times as many samples, but it is affected by larger differences in diffraction and aberrations due to the wider frequency bands observed. Single channel analysis has only 1/4 the samples but it is less affected by differences in diffraction and aberrations.
Agreed. As explained above, my but feeling is that the single-channel analysis would be safer overall (especially in the presence of CA), and that the larger number of samples in the white-balanced image would only start making a meaningful difference when the noise levels are very high. Again, more experiments for me
White balancing, in itself, does not affect edge sharpness, so a perfectly white-balanced mosaiced edge image will probably produce a weighted MTF curve (25% red curve, 50% green curve, 25% blue curve).
Why 'mosaiced'? To produce such an image I would suggest the dcraw -d switch (as opposed to -D) because -D does not subtract the black point (immaterial for most Nikons other than the D5300 but quite critical for many other brands) and it does not allow for white balancing by dcraw - which introduces several more steps to generating the TIFF to feed MTFMapper. Since DPR says that they measure the illuminant color temperature and set it right in-camera, dcraw -d -w will produce the undemosaiced white balanced raw data required (-4 and -T will ensure that no gamma or scaling is applied and the output will be a 16-bit tiff file).
Agreed. By "mosaiced" I simply meant "dcraw -d" or "dcraw -D", as opposed to a demosaiced image. Once we introduce demosaicing, all bets are off (as shown above in the extreme CA case).
If absolute accuracy is important, and multiple images are available, then repeated measurements followed by single-channel analysis (e.g., mtf mapper's "--bayer green" option) would be the best strategy.
Does --bayer work with 'gray' TIFFs like those generated by dcraw -D/-d? Would --bayer green use values both in quartet location G1 and G2? Would "--bayer red" work if fed a TIFF that has just values for red in the correct location in each quartet but with the other three colors set to zero (e.g. output of RawDigger 'export to TIFF')?
Yes. It
requires a -D/-d image to work as intended, although it will still "work" on a demosaiced image (but your results will probably be worse). Keep in mind, though, that MTF Mapper will convert a demosaiced RGB input image to a grayscale image using one of the "typical" blends: 0.299R + 0.587G + 0.114B. This gives you roughly a luminance-MTF output, which is probably what most people wanted.
The "--bayer green" treats G1 and G2 as one green channel.
In theory you could feed MTF Mapper an image that is zero except at the red CFA photosite locations, but it will not work well. As I discussed above, I use all available pixels for edge orientation estimation. I also think that my thresholding and rectangle-detection code will fail quite badly on such an input. I would still recommend "dcraw -d / -D" as the preferred input when using the --bayer option.
