megapixels and post-processing algorithm performance

"All things being equal" e.g. assuming both images are captured by a state-of-the-art Bayer CFA sensor based camera system, and assuming both images are eventually downsized for a standardized output device...

Does an abundance of megapixels in an image enable certain post-processing algorithms (e.g. noise reduction, sharpening, aberration correction, distortion correction, tone mapping, shadow/highlight recovery, perspective correction, saturation adjustment, contrast adjustment) to produce higher-quality results than they would from a lower-megapixel image?

Does the answer depend only on the ratio between the two images being considered, or does it also depend on the ratio of the two images to the output resolution?

Suppose you want to post images on a website; for the sake of argument, lets say at 640x427. Also suppose that you use a Bayer array camera with the same resolution — that would be a pretty pathetic one, for sure, but for the sake of argument, we can allow this camera to have the same optics as the comparison multi-megapixel camera.

This makes me wonder about the base ISO of this kind of camera — assuming that it is an APS-C or FF sensor. If so, the noise would be minuscule and the dynamic range could be spectacular.

However, this camera would need an antialias filter, otherwise the aliasing due to demosaicing would be really bad, with visible color fringing — and there are very few pixels to spare for this nonsense.

For example, I used Bruce Lindbloom’s artificial 6 megapixel image as a test:

Here is the image directly resampled to 640 pixels across:



And here is the image, where I processed it as if it were taken by a 640x427 pixel camera with Bayer filter:



It is really soft, and sharpening looks pretty bad at this resolution. I also used a simple bilinear demosaicing filter — more complex filters would likely give an even softer result.

Now this is pretty much the worst-case example. But clearly the extra megapixels helped a lot.

--
http://therefractedlight.blogspot.com

Alex Notpro said:

Does an abundance of megapixels in an image enable certain post-processing algorithms (e.g. noise reduction, sharpening, aberration correction, distortion correction, tone mapping, shadow/highlight recovery, perspective correction, saturation adjustment, contrast adjustment) to produce higher-quality results than they would from a lower-megapixel image?

Click to expand...

Many folks think so.

I do have my doubts, especially when it comes to the editablity of high-noise images. Things like tone mapping and shadow recovery are quite sensitive to noise, and often these processes are limited due to the heavy noise that they might produce.

I like to distinguish between an image that is good for display versus an image that is good for editing. A display-only image can have lots of noise and still be adequate for its purpose. A highly noisy image might require lots of noise reduction for editing, which partially goes against the idea of having lots of megapixels to begin with. Good algorithms help, though — for example, I often use noise reduction which only targets the darkest of shadows, leaving the clean highlights intact.

Alex Notpro said:

"All things being equal" e.g. assuming both images are captured by a state-of-the-art Bayer CFA sensor based camera system, and assuming both images are eventually downsized for a standardized output device...

Click to expand...

Sounds pretty much like the DXO "print" option?

Alex Notpro said:

Does an abundance of megapixels in an image enable certain post-processing algorithms (e.g. noise reduction, sharpening, aberration correction, distortion correction, tone mapping, shadow/highlight recovery, perspective correction, saturation adjustment, contrast adjustment) to produce higher-quality results than they would from a lower-megapixel image?

Click to expand...

1. Higher (sensor) spatial resolution in itself does have some advantages. If the (individual color channel) sensor passband far exceeds the combined passband of lens, OLPF, camera/scene movement etc, then you will have (in the limit) no aliasing. This means less errors even in lower frequencies (such as would be presented in e.g. a webpage). Further, when the sensor samples everything of interest that the lens "delivers", then we have a nearly perfect digital representation of the lens output, and can work on fixing lens flaws without having to fix sensor flaws at the same time (e.g. deconvolution).

2. Higher (sensor) spatial resolution may not be possible without some sacrifices. To some degree, the sensor total active area may be reduced, the needed micro lenses might reduce the benefit of large-aperture lenses (or wide-angles), the sensor might be hotter,.... All of theses are product/technology-specific factors, and I think that few of us can really _know_ to what degree they affect the camera design. What we can do is compare the cameras actually available and make our choices based on that.

-h

Mark S Abeln said:

A highly noisy image might require lots of noise reduction for editing, which partially goes against the idea of having lots of megapixels to begin with.

Click to expand...

For the case of ideal "MP increase", you are only increasing the information about the scene, not reducing it. If a larger amount of pixels are representing the same scene, each pixel will be lower quality, but the high-rez image can always be converted to the same form as the low-rez image. Thus, a general algorithm will have more to work with in the high resolution case. Think of it as election polls, if you have two polls of 100 people each, they will (making some assumptions) together be every bit as good as a single poll of 200 people, and may (in some cases) be somewhat better.

Now, MP increase may not be "ideal" in the sense that I am suggesting, and all processing algorithms may not be general in the sense that they perfectly exploit all of the information in the file. Both are practical snags, not fundamental limits.

-h

Alex Notpro said:

"All things being equal" e.g. assuming both images are captured by a state-of-the-art Bayer CFA sensor based camera system, and assuming both images are eventually downsized for a standardized output device...

Click to expand...

This seems to me to be a somewhat irrelevant assumption. You say "standardised output device", but there is very little standardisation in terms of number of pixels.

Every time I upgrade to a new computer, the screen has a higher pixel count. Surely, part of the point of having more megapixels than you currently need is future-proofing. It seems quite reasonable to expect that in a few years time output devices will be able to use far more pixels than is the case at present.

hjulenissen said:

Mark Scott Abeln said:

A highly noisy image might require lots of noise reduction for editing, which partially goes against the idea of having lots of megapixels to begin with.

Click to expand...

For the case of ideal "MP increase", you are only increasing the information about the scene, not reducing it. If a larger amount of pixels are representing the same scene, each pixel will be lower quality, but the high-rez image can always be converted to the same form as the low-rez image. Thus, a general algorithm will have more to work with in the high resolution case. Think of it as election polls, if you have two polls of 100 people each, they will (making some assumptions) together be every bit as good as a single poll of 200 people, and may (in some cases) be somewhat better.

Click to expand...

I understand what you are saying.

hjulenissen said:

Now, MP increase may not be "ideal" in the sense that I am suggesting, and all processing algorithms may not be general in the sense that they perfectly exploit all of the information in the file. Both are practical snags, not fundamental limits.

Click to expand...

Current tools often aren’t up to the job. Many noise reduction algorithms — as implemented in popular software — don’t target noise in a good manner, killing highlight detail as well as shadow noise, which effectively brings us back to fewer and larger pixels. Many of the advanced tools I like to use handle shadow noise quite poorly.

However, I like RawTherapee, since it places the noise reduction early in the processing chain — and it targets noise in the shadows rather better than other software that I use.

Tom Axford said:

Every time I upgrade to a new computer, the screen has a higher pixel count. Surely, part of the point of having more megapixels than you currently need is future-proofing. It seems quite reasonable to expect that in a few years time output devices will be able to use far more pixels than is the case at present.

Click to expand...

4K is becoming popular, and I see a lot of interest in this technology on these forums.

Here's some thoughts on how many sensels are sufficient based on an approach developed for other kinds on imaging systems. It may have something to teach us about the topic of this thread. I have been reading a book by Robert Fiete, entitled Modeling the Imaging Chain of Digital Cameras:

http://spie.org/Publications/Book/868276

There’s a chapter on balancing the resolution of the lens and the sensor, which introduces the concept of system Q, defined as:

Q = 2 * fcs / fco, where fcs is the cutoff frequency of the sampling system (sensor), and fco is the cutoff frequency of the optical system (lens). An imaging system is in some sense “balanced” when the frequencies are the same, and thus Q=2.

The assumptions of the chapter in the book where Q is discussed are probably appropriate for the kinds of surveillance systems the author works with, but they are not usually met in the photographic systems that most of us work with.

1) Monochromatic sensors (no CFA)

2) Diffraction-limited optics

3) No anti-aliasing filter

Under these assumptions, the cutoff frequency of the sensor is half the inverse of the sensel pitch; we get that from Nyquist. To get the cutoff frequency of the lens, we need to define the point where diffraction prevents the detection of whether we’re looking at one point or two. Lord Rayleigh came up with this formula in the 19 century:

R = 1.22 * lambda * N, where lambda is the wavelength of the light, and N is the f-stop.

Fiete uses a criterion that makes it harder on the sensor, the rounded Sparrow criterion:

S = lambda * N

Or, in the frequency domain, fco = 1 / (lambda * N)

Thus Q is:

Q = lambda * N / pitch

I figure that some of the finest lenses that we use are close to diffraction-limited at f/8. If that’s true, for 0.5 micrometer light (in the middle of the visible spectrum), a Q of 2 implies:

Pitch = N /4

At f/8 we want a 2-micrometer pixel pitch, finer than currently available for any available sensors sized at micro 4/3 and larger. A full frame sensor with that pitch would have 216 megapixels.

You can try to come up with correction to take into account the Bayer array. Depending on how the assumptions, the correction should be between 1 and some number greater than 2, but in any case, the pixel pitch should be at least as fine as for a monochromatic sensor. With a correction actor of 2, we're talking about a 800 Mp full frame sensor!

As an aside, note that you don’t need an AA filter for a system with a Q of 2, since the lens diffraction does the job for you. That’s not true with a Bayer CFA.

I have several questions:

1) Is any of this relevant to our photography?

2) Have I made a math or logical error?

3) At what aperture do our best lenses become close to diffraction-limited?

For details about the Sparrow criterion: http://blog.kasson.com/?p=5720

For more details on calculating Q: http://blog.kasson.com/?p=5742

For ruminations on corrections for a Bayer CFA: http://blog.kasson.com/?p=5752

Jim

Thank you for this detailed technical post. I need a few days to study it.

JimKasson said:

Here's some thoughts on how many sensels are sufficient based on an approach developed for other kinds on imaging systems. It may have something to teach us about the topic of this thread. I have been reading a book by Robert Fiete, entitled Modeling the Imaging Chain of Digital Cameras:

http://spie.org/Publications/Book/868276

There’s a chapter on balancing the resolution of the lens and the sensor, which introduces the concept of system Q, defined as:

Q = 2 * fcs / fco, where fcs is the cutoff frequency of the sampling system (sensor), and fco is the cutoff frequency of the optical system (lens). An imaging system is in some sense “balanced” when the frequencies are the same, and thus Q=2.

The assumptions of the chapter in the book where Q is discussed are probably appropriate for the kinds of surveillance systems the author works with, but they are not usually met in the photographic systems that most of us work with.

1) Monochromatic sensors (no CFA)

2) Diffraction-limited optics

3) No anti-aliasing filter

Under these assumptions, the cutoff frequency of the sensor is half the inverse of the sensel pitch; we get that from Nyquist. To get the cutoff frequency of the lens, we need to define the point where diffraction prevents the detection of whether we’re looking at one point or two. Lord Rayleigh came up with this formula in the 19 century:

R = 1.22 * lambda * N, where lambda is the wavelength of the light, and N is the f-stop.

Fiete uses a criterion that makes it harder on the sensor, the rounded Sparrow criterion:

S = lambda * N

Or, in the frequency domain, fco = 1 / (lambda * N)

Thus Q is:

Q = lambda * N / pitch

I figure that some of the finest lenses that we use are close to diffraction-limited at f/8. If that’s true, for 0.5 micrometer light (in the middle of the visible spectrum), a Q of 2 implies:

Pitch = N /4

At f/8 we want a 2-micrometer pixel pitch, finer than currently available for any available sensors sized at micro 4/3 and larger. A full frame sensor with that pitch would have 216 megapixels.

You can try to come up with correction to take into account the Bayer array. Depending on how the assumptions, the correction should be between 1 and some number greater than 2, but in any case, the pixel pitch should be at least as fine as for a monochromatic sensor. With a correction actor of 2, we're talking about a 800 Mp full frame sensor!

As an aside, note that you don’t need an AA filter for a system with a Q of 2, since the lens diffraction does the job for you. That’s not true with a Bayer CFA.

I have several questions:

1) Is any of this relevant to our photography?

2) Have I made a math or logical error?

3) At what aperture do our best lenses become close to diffraction-limited?

For details about the Sparrow criterion: http://blog.kasson.com/?p=5720

For more details on calculating Q: http://blog.kasson.com/?p=5742

For ruminations on corrections for a Bayer CFA: http://blog.kasson.com/?p=5752

Click to expand...

Hi Jim, good post. I don't have the book: what does he mean by cutoff frequency?

If he means when the relative MTF curve hits a first zero then:

1) fcs would be equal to the inverse of pitch, as opposed to half of it; and
2) fcd (d for diffraction) would indeed be 1 / (lambda*N) - without the 1.22 factor that would instead be required to indicate the first Airy PSF zero

So in this scenario the two frequencies would be the same when pitch = lambda * N. For a lambda of 0.5 microns and f/8, pitch = 4 microns. Is this a 'balanced' system?

For photographic purposes I would venture to say that the criterion of MTF=0 is perhaps less relevant then it could be in other applications, the main reason being that it is unbalanced (meaning that in this ideal example the system is diffraction limited) everywhere except for at the very end of the scale. Said graphically, for a perfect monochrome system with lambda=0.5 microns, pitch=4 microns and f/8:


Balanced? Only at the extremes with these parameters, otherwise always diffraction limited.

Imho a balanced system for us 'togs should instead require that the optics and the sensor be matched at relevant contrast/frequencies. It's what I clumsily attempted to say in this recent post , suggesting MTF50 as the balancing criterion.


Perfect monochrome system with square pixels, lambda=0.5 microns, pitch=6 microns, f/8

With these parameters (lambda=0.5microns, pitch=6microns, f/8) diffraction is less dominant in real frequencies below nyquist (0.5 cycles/pixel). Coincidentally this is approximately the pixel pitch of a D600/A7, both of which however have AAs.

Another criterion for matching/balancing the optics to the sensor could be to have the MTF curves cross over earlier, say at nyquist (perhaps what you were thinking in your post?). That would happen with a pitch of about 6.9 microns at f/8 with the assumptions above, D4/s territory.

Once we start putting reality into the equation everything needs to be recalculated, because Bayer/OLPF effects need to be added to the sensor model and aberrations/defocus/blur to the optics'. For instance stretching things a bit we could say that the last graph applies to sensors with AAs in the 4 micron pitch range.*

Jack

*BTW if anyone is wondering about how these pitches would relate to 'equivalent ' situations in formats other than FF, simply divide the f/number and the pitch by the crop factor.

Since I had the spreadsheet out, I thought I would try to see what an MTF50 'balanced' situation closer to reality might look like when modelling Sensor MTF as pitch+AA and Optics MTF as diffraction+lens aberrations:


Balanced at f/8.6

The curves were obtained with Pitch = 5.9 microns, Lambda = 0.55 microns, AA strength = 0.39 pixels (x2). Lens aberrations were modeled by a simple gaussian with standard deviation 0.3 pixels*. The Sensor and Optics MTF curves meet at MTF50 when aperture is f/8.6.

So I would say that this camera/lens system appears to be fairly well 'balanced' according to the criteria discussed in the last couple of posts.

Jack

* These are real world values from a previous thread . The parameters come from fitting the measured MTF curve of the green channel of a D610+85mm:1.8G in the direction in which it has stronger AA action to the theoretical model:


Measured solid Green curve matches fairly well the solid Black curve modeled with the parameters shown.

The de-Bayer component is still notably absent from this discussion.

Jack, thanks so much for the very informative response. I may be off by a factor of two, as you indicate. I'm trying to figure exactly what the author meant by "cutoff frequency" in the sensor case. It could be the frequency where the aliased signal becomes dc, in which case I'm off. Or it could be the frequency at which aliasing starts to occur, in which case a) my numbers are OK, and 2) Q=2 absolutely guarantees no aliasing, since the lens won't deliver spatial frequencies above the Nyquist frequency.

I think I've got it right, as the author states at one point that Q=2 means that there are 4.88 samples across the first ring of the Airy disk, and, for the example of an f/8 lens, and 0.5 micrometer light, that means the first ring is 9.76 micrometers across, and a 2 micrometer pixel pitch would yield 4.88 samples.

Whether this Q stuff is useful for photography remains an open question, since, as you point out, designing for a Q of two does not yield the best MTF50 results measured in cycles per pixel -- far from it, in fact.

I'll do some more work and report back.

BTW, I'm interested in your methodology for creating the simulated MTF curves. I'll PM you.

Jim

Jack Hogan said:

For photographic purposes I would venture to say that the criterion of MTF=0 is perhaps less relevant then it could be in other applications, the main reason being that it is unbalanced (meaning that in this ideal example the system is diffraction limited) everywhere except for at the very end of the scale. Said graphically, for a perfect monochrome system with lambda=0.5 microns, pitch=4 microns and f/8:

Balanced? Only at the extremes with these parameters, otherwise always diffraction limited.

Imho a balanced system for us 'togs should instead require that the optics and the sensor be matched at relevant contrast/frequencies.

Click to expand...

Jack, in the curves you've plotted, isn't everything over half a cycle per pixel aliased?

Jim

--

Jack, here's a reference that indicates that I got the Q equation right; it wasn't stated this way in the book.


You don't have to pay to download the article; the formula is in the abstract.

The jury's still out on the relevance of Q to our photography.

In the book, Fiete points out that there are a lot of practical considerations that would keep you from designing a system with a Q as high as 2. There seems to be no reason to design a monochromatic, diffraction-limited system with a Q of greater than 2, which would seem to bear on the original post.

The reasons Fiete mentions for staying below Q = 2 are SNR, sensitivity (we'd call it ISO), size and weight, vibration, and, bearing on your MTF50 point, contrast. You can see that many of these may be traded off against each other.

Thanks,

Jim

JimKasson said:

I think I've got it right, as the author states at one point that Q=2 means that there are 4.88 samples across the first ring of the Airy disk, and, for the example of an f/8 lens, and 0.5 micrometer light, that means the first ring is 9.76 micrometers across, and a 2 micrometer pixel pitch would yield 4.88 samples.

Click to expand...

Right.

JimKasson said:

Whether this Q stuff is useful for photography remains an open question, since, as you point out, designing for a Q of two does not yield the best MTF50 results measured in cycles per pixel -- far from it, in fact.

Click to expand...

This is what the relative MTF curves would look like for f/8, 0.5micon light and 2 micron pitch when plotted in cycles/pixels:


Is this balanced, or is it a sledgehammer cracking a nut?

Imho, unless one is forced to, using diffraction as an antialiasing filter is extremely inefficient as far as transferring the desirable spatial frequencies' contrast/sharpness below Nyquist is concerned. Much more inefficient than a typical antialiasing filter, for instance, whose MTF curve is designed to be much more forgiving of desirable frequencies (a cosine shape in the case of a 4-dot beam splitter as can be seen in the simulation of my previous post).

Keeping in mind that one sharpness metric is the area under the Total MTF curve between the spatial frequency at MTF50 and Nyquist, we can see just how much sharpness we are leaving at the scene in both cases: MTF50 of about 0.181 above vs 0.245 cy/px in the 'MTF50 Balanced' simulation with AA.

So unless I made a mistake somewhere (always a possibility as I am learning as I go along the above criteria for balancing optics to sensor would appear to me to be too stringent as far as photography is concerned, resulting in too small a pitch.

JimKasson said:

I'll do some more work and report back.

BTW, I'm interested in your methodology for creating the simulated MTF curves. I'll PM you.

Click to expand...

Excellent, a chance to check my math.

Jack

JimKasson said:

JimKasson wrote: Jack, in the curves you've plotted, isn't everything over half a cycle per pixel aliased

Click to expand...

Yes indeed. Hence my suggestion to choose a criterion other than fco=fcd for system 'balancing', and the potential need for an AA filter to abate (some) frequencies above Nyquist - as shown in the 'MTF50 Balanced' graph.

Jack Hogan said:

For photographic purposes I would venture to say that the criterion of MTF=0 is perhaps less relevant then it could be in other applications, the main reason being that it is unbalanced (meaning that in this ideal example the system is diffraction limited) everywhere except for at the very end of the scale. Said graphically, for a perfect monochrome system with lambda=0.5 microns, pitch=4 microns and f/8:

Balanced? Only at the extremes with these parameters, otherwise always diffraction limited.

Click to expand...

Jack, I'm not sure how useful this is, since it doesn't address your point that there are better ways to avoid aliasing than diffraction, but I did a plot of contrast, defined as (peak-valley)/(peak+valley), vs normalized separation, with unity being the rounded Sparrow distance, lambda*N. Here it is:


Horizontal axis is in multiples of lambda times N

You can see that 50% contrast is achieved if the PSFs are separated by about 1.65 * lambda * N. If that's the criterion, then Q = 2 / 1.65 = 1.21 is "balanced". Of course, there will be aliasing if there's no AA filter.

Here's the Matlab code:







BTW, you must think that the current trend of leaving off the AA filter is a step in the wrong direction. Is that right?

Jim

http://blog.kasson.com

Jack Hogan said:

Since I had the spreadsheet out, I thought I would try to see what an MTF50 'balanced' situation closer to reality might look like when modelling Sensor MTF as pitch+AA and Optics MTF as diffraction+lens aberrations:

Click to expand...

Hi Jack,

Some personal thoughts regarding some things (of which I suspect that you are well aware).

I presume that your modelling graphed below (designating "Optics: Diffraction + Aberrations") uses the identity for diffraction through a circular aperture opening without any influence of optical lens-aberrations ?

The description of a lens-system being considered "diffraction limited" (above some particular F-Ratio value) being imbued for lens-system F-Ratios of values greater than where an (empirically somehow measured) composite lens-camera system does not (in itself) mean that all of the effects upon the (lens-system only) spatial frequency response (MTF) of various lens-system optical aberrations are entirely absent above such particular F-Ratios.

Actual measurements of the (lens-system only) spatial frequency (MFT) response are required in order to accurately determine that. (Somewhat ironically), interpretation of such measurements would require calculation (and subsequent numerical removal) of the spatial frequency responses of: any (cosine function) periodic spatial frequency response optical ("AA") filtering assemblies that may exist; actual active-area (per) photosite-apertures, as well as photosite-pitch (in order to determine the spatial sampling frequency).

The same procedures as above would also apply to any relevant camera and/or subject movement, as well as any known lens-system de-focusing (relative to the location of a flat test target).

(As you noted, below), "The de-Bayer component is still notably absent from this discussion".

Given the complexity of the many and various possible demosaicing processes applied to the RAW-level color-filtered photosite data - as well as any potential non-linearities involved in (what may be either fixed, or adaptive) specific algorithm characteristics applied to that data - it seems that the demosaicing algorithm apllied (could, possibly) be tested (with specific data sets) in order to determine the (demosaicing algorithm only) spatial frequency (MTF) response.

However, any [fixed] algorithm non-linearities imply the qualification of test results to a specific set of parameters - and any algorithm characteristics that may be [adaptive] in nature based upon the nature of the test-target itself appear to (potentially) throw a massive "monkey-wrench" into discussions regarding such test results ... i.e. "what particular test-target(s) may (or may not) correspond to photographic subject-matter as it typically apperas and is recorded within particular photographic applications of interest by this or that particular photgrapher ?".

In my view, the inclusion of individual demosaing algorithm spatial frequency (MTF) response(s) - with the above-stated limitations (and related potential interpretational complexities) - would (in all various cases) equally affect the data resulting from the here being discussed numerical multiplcation in the spatial frequency domain (ideally actual) lens-system, image-sensor based optical filtering assemblies, photosite-apeture, and photosite-pitch (to determine spatial sampling frequency).

For Red and Blue color-filtered and sampled photosites, spatial sampling frequency reduces by 2.

Jack Hogan said:

Balanced at f/8.6

The curves were obtained with Pitch = 5.9 microns, Lambda = 0.55 microns, AA strength = 0.39 pixels (x2). Lens aberrations were modeled by a simple gaussian with standard deviation 0.3 pixels*. The Sensor and Optics MTF curves meet at MTF50 when aperture is f/8.6.

Click to expand...

The net, composite spatial frequency (MTF) response that you present above looks very close to my posted graphic displayed below - resulting when using an ("AA") filter) with a physical beam offest equal to 0.4 (from my previous simulations using ideal circular diffraction, "AA" filter, and 100% fill-factor (the ratio of the photosite aperture divided by the photosite-pitch).

Offset of 0.400000 yields first zero response at 0.625000 times the Spatial Sampling Frequency.


Diffraction (BLU)_Photosite+AA=0.625 (RED)_Half-Photosite+AA=0.625 (GRN)

From: http://www.dpreview.com/forums/thread/3475094

Note that while the Nyquist spatial frequency relating to the RED-colored composite MTF response plot displayed exists in the middle of the X-axis (at the small green dot in the center), ...

... the Nyquist spatial frequency related to the GREEN-colored MTF composite plot (representing a different sized photosite that is combined a with correspondingly sized "AA" filter assembly that is 100% fill-factor and represents a photosite assembly having physical dimensions equal to 1/2 that of the photosite size represented by the RED-colored plot.

The lens-system (circular aperture opening, no optical aberrations included) diffraction (the BLUE-colored plot) remains the same spatial frequency values in both (RED/GREEN) simulations - demonstrating what appears to be a notable increase in the area under the curve (between MTF=50% and the interpretationally designated MTF "extinction" (zero-value) spatial frequency.

If considering that "extinction" spatial frequency to be equal to 1 cycles/pixel (at the far-right of the displayed X-axis), the ratio of the area under the curves is (even) numerically larger in value than an interpretation involving the "extinction" spatial frequency to be equal to 1/2 cycles/pixel (existing at the center of the displayed X-axis; in the middle; at the small green dot ).

Those (simulated) results seem to provide something of an answer to the OP's original question:

Does an abundance of megapixels in an image enable certain post-processing algorithms (e.g. noise reduction, sharpening, aberration correction, distortion correction, tone mapping, shadow/highlight recovery, perspective correction, saturation adjustment, contrast adjustment) to produce higher-quality results than they would from a lower-megapixel image?

in terms of composite MTF responses - without including Signal/Noise Ratios at spatial frequencies.

.

A notable aside:

When an (AA") filter physical offset that is equal to 0.5 photosite dimensions is simulated below:


Diffraction (BLU)_Photosite+AA=0.500 (RED)_Half-Photosite+AA=0.500 (GRN)

... similar benefits are seen in the case of the ratio of the areas under the curves (over spatial frequency domains) of the 1/2 size photosite and "AA" filter assemblies. However, this case (of using a unit-sized photosite combined with an "AA" filter with a physical offset equal to 1/2 of the photosite aperture, with the same lens-system diffraction) is not unlike simply using a similar (100% fill-factor) photosite that is 2 times the physical dimensions of the (unit) photosite itself.

Nevertheless, (due to the periodic cosine nature of the "AA" filter beam-splitting assembly spatial frequency response), note that the (RED-colored plot, unit photosite) output (again) rises as high as MTF=5% above the relevant Nyquist spatial frequency prior to "extinction".

An "AA" filter having physical offset of 0.667 (2/3) photosite-aperture looks like this:


Diffraction (BLU)_Photosite+AA=0.667 (RED)_Half-Photosite+AA=0.667 (GRN)

Such composite spatial frequency response "rebounding" appears to nearly abate at offset=0.75:


Diffraction (BLU)_Photosite+AA=0.750 (RED)_Half-Photosite+AA=0.750 (GRN)

While the diffraction through a circular aperture (sans lens-system optical aberrations) scales the composite spatial frequency (MTF), the suppression of the periodic MTF "rebounding" behavoir appears to primarily be a result of the (spatial frequency) multiplicative combination of the photosite-aperture together with the "AA" filter assembly physical offset (only).

As a result, (regardless of lower diffraction situations, when diffraction "extinction" occurs at a higher spatial frequency), a (flatter) composite response can achieved - "price paid" being a larger area under the curves of image-data existing where higher than Nyquist frequency is allowed.

Jack Hogan said:

So I would say that this camera/lens system appears to be fairly well 'balanced' according to the criteria discussed in the last couple of posts.

Jack

* These are real world values from a previous thread . The parameters come from fitting the measured MTF curve of the green channel of a D610+85mm:1.8G in the direction in which it has stronger AA action to the theoretical model:

Measured solid Green curve matches fairly well the solid Black curve modeled with the parameters shown.

The de-Bayer component is still notably absent from this discussion.

Click to expand...

Question: Was any (evidently Gaussian approximated) lens-system defocus included in modelling ?

.



Questions: Is there some demonstrable relationship regarding the numerical characteristics of the individual spatial frequency (MTF) funtions of: diffraction (through a circular aperature); "AA" filter periodic (cosine) response; and photosite-aperture which implies a particular advantage to a matching of the MTF=50% values as depicted in your graph (displayed directly above) ?

If so, does such a relationship relate to integrals taken between MTF=50% and MTF "extinction" ?

.

Regarding "sharpness metrics". The "SQF" related forms of "metrics" appear to relate to the viewed images where the limits of integration do not appear to be based upon MTF=50% points:

Grainger then developed a model, which basically states that the subjective sharpness of a print corresponds to the area under the MTF curve between the spatial frequencies of (0.5 x magnification) and (2 x magnification) when spatial frequency is plotted on a logarithmic scale.

From: http://www.bobatkins.com/photography/technical/mtf/mtf4.html

... which is discussed in greater detail on Pages 23-25 of Section 2.2 - "Subjective Quality Factor":

http://www.cis.rit.edu/people/faculty/johnson/pub/gmj_phd.pd

DM

Detail Man said:

Detail Man said:

Balanced at f/8.6

The curves were obtained with Pitch = 5.9 microns, Lambda = 0.55 microns, AA strength = 0.39 pixels (x2). Lens aberrations were modeled by a simple gaussian with standard deviation 0.3 pixels*. The Sensor and Optics MTF curves meet at MTF50 when aperture is f/8.6.

Click to expand...

The net, composite spatial frequency (MTF) response that you present above looks very close to my posted graphic displayed below - resulting when using an ("AA") filter) with a physical beam offest equal to 0.4 (from my previous simulations using ideal circular diffraction, "AA" filter, and 100% fill-factor (the ratio of the photosite aperture divided by the photosite-pitch).

Offset of 0.400000 yields first zero response at 0.625000 times the Spatial Sampling Frequency.


Diffraction (BLU)_Photosite+AA=0.625 (RED)_Half-Photosite+AA=0.625 (GRN)

From: http://www.dpreview.com/forums/thread/3475094

Note that while the Nyquist spatial frequency relating to the RED-colored composite MTF response plot displayed exists in the middle of the X-axis (at the small green dot in the center), ...

... the Nyquist spatial frequency related to the GREEN-colored MTF composite plot (representing a different sized photosite that is combined a with correspondingly sized "AA" filter assembly that is 100% fill-factor and represents a photosite assembly having physical dimensions equal to 1/2 that of the photosite size represented by the RED-colored plot.

The lens-system (circular aperture opening, no optical aberrations included) diffraction (the BLUE-colored plot) remains the same spatial frequency values in both (RED/GREEN) simulations - demonstrating what appears to be a notable increase in the area under the curve (between MTF=50% and the interpretationally designated MTF "extinction" (zero-value) spatial frequency.

If considering that "extinction" spatial frequency to be equal to 1 cycles/pixel (at the far-right of the displayed X-axis), the ratio of the area under the curves is (even) numerically larger in value than an interpretation involving the "extinction" spatial frequency to be equal to 1/2 cycles/pixel (existing at the center of the displayed X-axis; in the middle; at the small green dot ).

Those (simulated) results seem to provide something of an answer to the OP's original question:

Does an abundance of megapixels in an image enable certain post-processing algorithms (e.g. noise reduction, sharpening, aberration correction, distortion correction, tone mapping, shadow/highlight recovery, perspective correction, saturation adjustment, contrast adjustment) to produce higher-quality results than they would from a lower-megapixel image?

in terms of composite MTF responses - without including Signal/Noise Ratios at spatial frequencies.

.

A notable aside:

When an (AA") filter physical offset that is equal to 0.5 photosite dimensions is simulated below:

Click to expand...

SHOULD READ:

When an "AA" filter physical offset that is equal to 0.5 photosite dimensions (yielding a first zero response at 0.5, or 1/2 times the spatial sampling frequency) is simulated below:

Detail Man said:

Diffraction (BLU)_Photosite+AA=0.500 (RED)_Half-Photosite+AA=0.500 (GRN)

... similar benefits are seen in the case of the ratio of the areas under the curves (over spatial frequency domains) of the 1/2 size photosite and "AA" filter assemblies. However, this case (of using a unit-sized photosite combined with an "AA" filter with a physical offset equal to 1/2 of the photosite aperture, with the same lens-system diffraction) is not unlike simply using a similar (100% fill-factor) photosite that is 2 times the physical dimensions of the (unit) photosite itself.

Nevertheless, (due to the periodic cosine nature of the "AA" filter beam-splitting assembly spatial frequency response), note that the (RED-colored plot, unit photosite) output (again) rises as high as MTF=5% above the relevant Nyquist spatial frequency prior to "extinction".

Click to expand...

.

Detail Man said:

An "AA" filter having physical offset of 0.667 (2/3) photosite-aperture looks like this:

Click to expand...

SHOULD READ:

An "AA" filter having a physical offset of 0.375 photosite dimensions (yielding a first zero response at 0.667, or 2/3 times the spatial sampling frequency) looks like this:

Detail Man said:

Diffraction (BLU)_Photosite+AA=0.667 (RED)_Half-Photosite+AA=0.667 (GRN)

Click to expand...

.

Detail Man said:

Such composite spatial frequency response "rebounding" appears to nearly abate at offset=0.75:

Click to expand...

SHOULD READ:

Such composite spatial frequency response "rebounding" appears to nearly abate at a physical offset of 0.333 photosite dimensions (yielding a first zero response at 0.75, or 3/4 times the spatial sampling frequency):

Detail Man said:

Diffraction (BLU)_Photosite+AA=0.750 (RED)_Half-Photosite+AA=0.750 (GRN)

While the diffraction through a circular aperture (sans lens-system optical aberrations) scales the composite spatial frequency (MTF), the suppression of the periodic MTF "rebounding" behavoir appears to primarily be a result of the (spatial frequency) multiplicative combination of the photosite-aperture together with the "AA" filter assembly physical offset (only).

As a result, (regardless of lower diffraction situations, when diffraction "extinction" occurs at a higher spatial frequency), a (flatter) composite response can achieved - "price paid" being a larger area under the curves of image-data existing where higher than Nyquist frequency is allowed.

Click to expand...

Notes:

References (appearing above) to "photosite dimensions" refer to the case of the (100% fill-factor based) simulation model utilized (where the photosite-aperture equals the photosite-pitch).

See this previous post for the mathematical models utilized in the simulations performed:

http://www.dpreview.com/forums/post/5132390

Please post any comments relating to the above subjects on this thread, and on this DPR forum.

DM