This is a continuation of previous efforts here: http://www.dpreview.com/forums/thread/3447868

How lens diffraction effects, sensor photosite size, and optical low-pass filter (OLPF, AA Filter) effects influence the net composite MTF of a lens/camera system when the product of Wavelength and F-Number is increased (for the purpose of reducing the effects of certain types of lens aberrations, and/or realizing deeper Depth of Field) is a matter of interest to me.

The following analysis is not intended to be a rigorous accounting for all factors involved in determining the net composite Modulation Transfer Function (MTF) of a lens/camera system.

For the sake of simplicity, it assumes photosite aperture equal to photosite pitch (100% fill factor).

It does not account for the effects of micro-lenses, and it does not account for effects of the other optical filtering located within an image-sensor's "filter stack". It appears (from some things that I have read) that effects of lens aberrations - as well as diffusion in epitaxial layers (at least when photosite aperture does equal photosite pitch) - are not directly multiplicative in the spatial frequency domain, thus requiring ray-tracing analysis in order to determine their effects upon the composite system MTF.

While it is indeed true that a number of factors (lens aberrations, focus integrity, and camera movement, in addition to post-capture processing, print/display limitations, etc.) will all act to decrease the composite MTF, the relevant contributions of diffraction, photosite aperture, and optical low-pass filtering can still be compared in order to determine their composite effects.

Because de-mosaicing algorithms are complicated and differing in how many individual photosites are combined in the process of rendering their image-data into pixels in some RGB color-space, they are not considered here. The analytical model used relates to a monochromatic image-sensor.

.

All BLUE colored plots in the diagrams below represent the MTF resulting from lens-system diffraction through a circular aperture (only). The product of the Wavelength multiplied by the F-Number (in spatial units of distance) is equal to the reciprocal of the unit spatial sampling frequency (X-axis).

.

All RED colored plots in the diagrams below represent the composite MTF resulting from lens-system diffraction through a circular aperture combined (convolved together) with reference photosites of dimensions equal to the reciprocal of the unit spatial sampling frequency (appearing along the X-axis).

In all but the first diagram appearing below, AA Filters of varying "strengths" (varying locations of the first zero response spatial frequency as a fraction of the appropriate spatial sampling frequency) are also combined (convolved) together with the lens-system diffraction and photosites in order to calculate the composite MTF response.

Note that for the RED colored plots, the Shannon-Nyquist spatial frequency limit exists in the center of the X-axis (denoted by a small green colored square). In those cases, the area of primary interest exists in the left half of the diagrams, and the comparative analysis describing the diagrams below will relate to the MTF response existing at that Shannon-Nyquist spatial frequency limit.

.

All GREEN colored plots in the diagrams below represent the composite MTF resulting from lens-system diffraction through a circular aperture combined (convolved together) with photosites of dimensions that are equal to one-half the dimensions of the reference photosites represented in the RED plots. The purpose is to compare results when photosite size is halved.

In all but the first diagram appearing below, AA Filters of varying "strengths" (varying locations of the first zero response spatial frequency as a fraction of the appropriate spatial sampling frequency) are also combined (convolved) together with the lens-system diffraction and photosites in order to calculate the composite MTF response.

Note that for the GREEN colored plots, the Shannon-Nyquist spatial frequency limit exists at the right-most end of the X-axis. In those cases, the area of interest extends across the entire diagram. This additional spatial frequency resolution afforded by the higher spatial sampling frequency represents a significant portion of the benefits realized from using the smaller photosites.

.

In this diagram, the BLUE plot represents the lens-system diffraction, the RED plot represents the lens-system diffraction combined with the reference photosites, and the GREEN plot represents the lens-system diffraction combined with the smaller (one-half size) photosites. No AA Filter exists in the model in the case of this first diagram. The use of the smaller photosite results in a moderate (25% > 35%) improvement in the MTF response at the Shannon-Nyquist frequency:

Diffraction(BLU)_Photosite(RED)_Half-Photosite(GRN)

.

In this diagram, the BLUE plot represents the lens-system diffraction, the RED plot represents the lens-system diffraction combined with the reference photosites, and the GREEN plot represents the lens-system diffraction combined with the smaller (one-half size) photosites. AA Filters with a first spatial frequency of zero response equal to 3/4 (75.00%) of the spatial sampling frequency are included in the model. The use of the smaller photosite results in a more significant (13% > 30%) improvement in the MTF response at the Shannon-Nyquist frequency:

Diffraction (BLU)_Photosite+AA=0.750 (RED)_Half-Photosite+AA=0.750 (GRN)

.

In this diagram, the BLUE plot represents the lens-system diffraction, the RED plot represents the lens-system diffraction combined with the reference photosites, and the GREEN plot represents the lens-system diffraction combined with the smaller (one-half size) photosites. AA Filters with a first spatial frequency of zero response equal to 2/3 (66.67%) of the spatial sampling frequency are included in the model. The use of the smaller photosite results in a slightly larger (10% > 29%) improvement in the MTF response at the Shannon-Nyquist frequency:

Diffraction (BLU)_Photosite+AA=0.667 (RED)_Half-Photosite+AA=0.667 (GRN)

.

In this diagram, the BLUE plot represents the lens-system diffraction, the RED plot represents the lens-system diffraction combined with the reference photosites, and the GREEN plot represents the lens-system diffraction combined with the smaller (one-half size) photosites. AA Filters with a first spatial frequency of zero response equal to 5/8 (62.50%) of the spatial sampling frequency are included in the model. The use of the smaller photosite results in a slightly larger (8% > 28%) improvement in the MTF response at the Shannon-Nyquist frequency:

Diffraction (BLU)_Photosite+AA=0.625 (RED)_Half-Photosite+AA=0.625 (GRN)

.

In this diagram, the BLUE plot represents the lens-system diffraction, the RED plot represents the lens-system diffraction combined with the reference photosites, and the GREEN plot represents the lens-system diffraction combined with the smaller (one-half size) photosites. AA Filters with a first spatial frequency of zero response equal to 1/2 (50.00%) of the spatial sampling frequency are included in the model. The use of the smaller photosite results in a more significant (0% > 25%) improvement in the MTF response at the Shannon-Nyquist frequency:

Diffraction (BLU)_Photosite+AA=0.500 (RED)_Half-Photosite+AA=0.500 (GRN)

.

Comments:

Degradation of composite system Modulation Transfer Function (MTF) due to lens-system diffraction through a circular aperture, though gradual in nature, is indeed tangible. When spatial frequency equals the product of the Wavelength multiplied by the F-Number, the composite system MTF is reduced to 0% (complete "extinction" of the MTF composite response).

Nothing will make the effects of lens-system diffraction upon the composite system MTF better without exacting some cost in terms of performance. Sharpening techniques may be able to improve matters somewhat by selectively boosting higher frequencies using subtractive high-pass or deconvolution filtering techniques - but at some cost of reduced signal/noise ratio. Attempting to increase the signal/noise ratio using noise-reduction techniques reduces the composite system MTF.

Reducing the physical dimensions of photosites (and any associated AA Filters) will (up to a point) increase the composite system MTF somewhat, and more importantly will extend the spatial frequency response to higher spatial frequencies. The lower limit of photosite size exists where the photosite size equals one-half of the product of the Wavelength multiplied by the F-Number. Further, meaningful resolution also depends upon the existence of a viable system signal/noise ratio.

DM ...