Alvy Ray Smith wrote a wonderful white paper when he was working for Microsoft. The title was “A pixel is not a little square.” I have referred people to it many times over the years, but for some it just doesn’t click. I think that’s because Smith’s audience was graphics engineers, not photographers. So here is my crack at the same subject matter, but aimed at photographers. I'm doing this because it has been apparent to me over the last couple of weeks that some people on this forum are having a hard time with the concept.
When many people think of pixels, they imagine a grid of tiny colored squares that make up a digital photograph. That mental picture is popular but wrong. A pixel is not a little colored tile. It is a number -- or a set of numbers for red, green, and blue -- that represents a sample taken from an underlying continuous image. The subject is a smooth scene in the world. The sensor samples that scene at discrete points, just as a microphone samples a sound wave. The grid of pixels is like graph paper on which those samples are recorded.
Pixels are measurements, not rectangles. In audio, when you record sound sampling at 44.1 kilohertz, you are not capturing forty-four thousand one hundred boxes of sound per second. You are taking samples of a continuous waveform. When you play the sound back, those samples are used to reconstruct a smooth wave. In the same way, each pixel in a photograph is a sample of brightness and color at a position on the sensor. The picture is continuous, but we have only sparse data points from which we must reconstruct it.
The image you see on your monitor when you zoom in, with little colored squares, is a processing convenience, not the underlying reality. It is as if a music player drew a staircase instead of a smooth sine wave. The software must draw something, and the computationally cheapest thing to draw is a rectangle of uniform color for each sample. That gives people the wrong idea about what an image really is. A better way to think of pixels is as infinitesimal points on a smooth landscape. The display software has to paint the spaces between them, and squares are simply the easiest way to do it. More sophisticated resampling methods use smoother transitions, much as a high-quality audio converter uses filters to turn discrete samples into continuous sound.
This distinction matters to photographers. The number of pixels in a sensor is not the same as the amount of detail in an image. Detail depends on how well the sampling grid captures the variations in the scene. A sensor that samples too coarsely will miss fine detail and produce aliasing -- the visual equivalent of distortion in sound. That is what causes jagged edges, moiré patterns, and false color. Sharpening and scaling operations are forms of reconstruction, and if the data is undersampled, no amount of processing can restore what was never captured. Conversely, when a lens and sensor combination samples the scene finely enough, the reconstruction can be faithful and smooth.
Every time software enlarges, sharpens, or demosaics an image, it is performing mathematical reconstruction, filling in missing values between samples. The quality of that process depends on how well the original sampling captured the information in the first place, and on the algorithm used for reconstruction. The image becomes visible only when those discrete samples are reconstructed into a continuous picture, just as a sequence of audio samples becomes music when it is turned back into a smooth waveform.
There are pernicious effects of using nearest-neighbor resampling -- the method that produces those little squares of uniform color. It is far from the best way to reconstruct an image. In fact, it is hard to think of a worse one. Yet presenting pixels as little squares in image editing software invites photographers to judge image quality by the sharpness of those squares, and to take the pseudo-logical next step of trying to make them look crisp. In a well-sampled image, that is the last thing you want. Sharp-looking nearest-neighbor squares almost always mean that the image was sampled too coarsely, has been brutally sharpened, or that something else has gone wrong.
To get a realistic print or display of an image, it’s important to get the right reconstruction. Some printer software is good at that. QImage is an exemplar here. Other software is moderately good at resampling for print or display, like Photoshop and Lightroom. There is specialized software that is even better, but it’s somewhat difficult to use. If you want to get the right amount of sharpening for an image, it’s important to look at the image after it has been resampled for the output device.
