Resolution, aliasing and light loss - why we love Bryce Bayer's baby anyway
It's unlikely Kodak's Bryce Bayer had any idea that, 40 years after patenting a 'Color Imaging Array,' his design would underpin nearly all contemporary photography and live in the pockets of countless millions of people around the world.
|It seems so obvious, once someone else has thought of it, but capturing red, green and blue information as an interspersed, mosaic-style array was breakthrough.
Image: based on original by Colin M.L Burnett
The Bayer Color Filter Array is a genuinely brilliant piece of design: it's a highly effective way of capturing color information from silicon sensors that can't inherently distinguish color. Most importantly, it does a good job of achieving this color capture while still capturing a good level of spatial resolution.
However, it isn't entirely without its drawbacks: It doesn't capture nearly as much color resolution as a camera's pixel count seems to imply, it's especially prone to sampling artifacts and it throws away a lot of light. So how bad are these problems and why don't they stop us using it?
There's a limit to how much resolution you can capture with any pixel-based sensor. Sampling theory dictates that a system can only perfectly reproduce signals at half the sampling frequency (a limit known as the Nyquist Frequency). If you think about trying to represent a single pixel-width black line, you need at least two pixels to be sure of representing it properly: one to capture the line and another to capture the not-line.
Just to make things more tricky, this assumes your pixels are aligned perfectly with the line. If they're slightly misaligned, you may get two grey pixels instead. This is taking into consideration by the Kell factor, which says that you'll actually only reliably capture resolution around 0.7x your Nyquist frequency.
But, of course, a Bayer sensor doesn't sample all the way to its maximum frequency because you're only sampling single colors at each pixel, then deriving the other color values from neighboring pixels. This lowers resolution (effectively slightly blurring the image).
So, with these two factors (the limitations of sampling and Bayer's lower sampling rate) in mind, how much resolution should you expect from a Bayer sensor? Since human vision is most sensitive to green information, it's the green part of a Bayer sensor that's used to provide most of the spatial resolution. Let's have a look at how it compares to sampling luminance information at every pixel.
|Counter-intuitive though it may sound, the green channel captures just as much horizontal and vertical detail as the sensor capturing data at every pixel. Where it loses out is on the diagonals, which sample at 1/2 the frequency.|
Looking at just the green component, you should see that a Bayer sensor can still capture the same horizontal and vertical green (and luminance) information as a sensor sampling every pixel. You lose something on the diagonals, but you still get a good level of detail capture. This is a key aspect of what makes Bayer so effective.*
It's a less good story when we look at the red and blue channels. Their sampling resolution is much lower than the luminance detail captured by the green channel. It's worth bearing in mind that human vision is much more sensitive to luminance resolution than it is to color information, so viewers are likely to be more tolerant of this shortcoming.
So what happens to everything above the Nyquist frequency? Well, unless you do something to stop it, your camera will try to capture this information, then present it in a way it can represent. A process called aliasing.
Think about photographing a diagonal black stripe with a low resolution camera. Even with a black and white camera, you risk the diagonal being represented as a series of stair steps: a low-frequency pattern that acts as an 'alias' for the real pattern.
The same thing happens with fine repeating patterns that are a higher frequency than your sensor can cope with: they appear as spurious aliases of the real pattern. These spurious patterns are known as moiré. This isn't unique to Bayer, though, it's a side-effect of trying to capture higher frequencies than your sampling can cope with. It will occur on all sensors that use a repeating pattern of pixels to capture a scene.
Sensors that use the Bayer pattern are especially prone to aliasing though, because the red and blue channels are being sampled at much lower frequencies than the full pixel count. This means there are two Nyquist frequencies (a green/luminance limit and a red/blue limit) and two types of aliasing you'll tend to encounter: errors in detail too fine for the sensor to correctly capture the pattern of and errors in (much less fine) detail that the camera can't correctly assess the color of.
To reduce this first kind of error most cameras have, historically, included Optical Low Pass Filters, also known as Anti-Aliasing filters. These are filters mounted in front of the sensor that intentionally blur light across nearby pixels, so that the sensor doesn't ever 'see' the very high frequencies that it can't correctly render, and doesn't then misrepresent them as aliasing.**
|The point at the center of the Siemens star is too fine for this monochrome camera to represent, so it's produced a spurious diamond-shaped 'alias' at the center instead.||This image second was shot with a very high resolution camera, blurred to remove high frequencies, then downsized to the same resolution as the first shot. It still can't accurately represent the star, but doesn't alias when failing.|
These aren't so strong as to completely prevent all types of aliasing (very few people would be happy with a filter that blurred the resolution down to 1/4 of the pixel height: the Nyquist frequency of red and blue capture), instead they blur the light just enough to avoid harsh stair-stepping and reduce the severity of the false color on high-contrast edges.
|With a Bayer filter, you get a fun color component to this aliasing. Not only has the camera tried to capture finer detail than its sensor can manage, you get to see the side-effect of the different resolutions the camera captures each color with.||Again, if you compare this with a significantly over-sampled image, blurred then downsized, you don't see this problem. However, look closely you can still see traces of the false color that occurred at the much higher frequency this camera was shooting at.|
This means that, a camera with an anti-aliasing filter, you shouldn't see as much false color in the high-contrast mono targets within our test scene, but it'll do nothing to prevent spurious (aliased) patterns in the color resolution targets.
At the silicon level, modern sensors are pretty amazing. Most of them operate at an efficiency (the proportion of light energy converted into electrons) around 50-80%. This means there's less than 1EV of performance improvement to be had in that respect, because you can't double the performance of something that's already over 50% effective. However, before the light can get to the sensor, the Bayer design throws away around 1EV of light, because each pixel has a filter in front of it, blocking out the colors it's not meant to be measuring.
around 1EV of light'
This is why Leica's 'Monochrom' models, which don't include a color filter array, are around one stop more sensitive than their color-aware sister models. (And, since they can't produce false color at high-contrast edges, they don't include anti aliasing filters, either).
It's this light loss component that may eventually spell the end of the Bayer pattern as we know it. For all its advantages, Bayer's long term dominance is probably most at risk if it gets in the way of improved low-light performance. This is why several manufacturers are looking for alternatives to the Bayer pattern that allow more light through to the sensor. It's telling, though, that most of these attempts are essentially variations on the Bayer theme, rather than total reinventions.
These variations aren't the only alternatives to the Bayer design, of course.
Sigma's Foveon technology uses a fundamentally different approach to measure multiple colors at the same location, so promises higher color resolution, no light loss to a color filter array and less aliasing. But, while these sensors are capable of producing very high pixel-level sharpness, this currently comes at an even greater noise cost (which limits both dynamic range and low light performance), as well as struggling to compete with the color reproduction accuracy that can be achieved using well-tuned colored filters. More recent versions reduce the color resolution of two of their channels, sacrificing some of their color resolution advantage for improved noise performance.
Meanwhile, Fujifilm has struck out on its own, with the X-Trans color filter pattern. This still uses red, green and blue filters but features a larger repeat unit: a pattern that repeats less frequently, to reduce the risk of it clashing with the frequency it's trying to capture. However, while the demosaicing of X-Trans by third-party software is improving, and the processing power needed to produce good-looking video looks like it's being resolved, there are still drawbacks to the design.
Ironically, devoting so much of the sensor to green/luminance capture appears to have the side-effect of reducing its ability to capture and represent foliage (perhaps because it lacks the red and blue information required to render the subtle tint of different greens).
Which leaves Bayer in a situation akin to Winston Churchill's take on Democracy as: 'the worst form of Government except all those other forms that have been tried from time to time.'
40 not out
As we've seen before, the sheer amount of effort being put into development and improvement of Bayer sensors and their demosaicing is helping them overcome the inherent disadvantages. Higher pixel counts keep pushing the level of color detail that can be resolved, despite the 1/2 green, 1/4 red, 1/4 blue capture ratio.
And, because the frequencies that risk aliasing relate to the sampling frequency, higher pixel count sensors are showing increasingly little aliasing. The likelihood of you encountering frequencies high enough to cause aliasing falls as your pixel count helps you resolve more and more detail.
Add to this the fact that lenses can't perfectly transmit all the detail that hits them, and you start to reach the point that the lens will effectively filter-out the very high frequencies that would otherwise induce aliasing. In recent years we've seen filter-less sensors starting at 36MP on full frame, 24MP on APS-C and Four Thirds sensors from 16MP (all of which are sampling their lenses at over 200 pixels per mm), and seen that these only produce significant moiré when paired with very sharp lenses shot wide-enough open that diffraction doesn't end up playing the anti-aliasing role.
So, despite the cost of light and of color resolution, and the risk of error, Bryce Bayer's design remains firmly at the heart of digital photography, more than 40 years after it was first patented.
Nov 20, 2017
Nov 1, 2017
Oct 24, 2017
Oct 16, 2017
|Madrid subway by MAGMATCICO62|
from Your City - Public Transport
|Incandescent Bulb by Kukla|
from Illuminate- Macro only
|Curiousity by PERCY2|
from Macro - Your Best Macro Ever
|Hoar Frosted Trees by sabishiT3T|
There's never been a better time to shop for a new camera, but the number of options available can be overwhelming. In this series of buying guides we've provided customized recommendations for several use cases, from shooting landscapes to buying a first camera for a student photographer.
Shopping for a camera with a set budget? No problem! We've rounded up our favorite cameras, broken them into price brackets and picked the best of the bunch.
Looking for a lightweight compact camera that's easy to bring with you anywhere? Or maybe you're smartphone-shopping and want the one that takes the best picture. And what if you want to shoot from above? In these buyers guides we have recommendations for the best compact cameras, smartphones and drones.
Despite reports to the contrary, analysis of DPReview images by our friend Jim Kasson confirms a disappointing fact: Sony a7R III is still a Star Eater. But there may be some improvements.
As the saying goes: A photo is worth a thousand words. And if you're sending that photo through Facebook Messenger, your thousand words now look twice as nice after today's update to 4K resolution.
Get to know the new Leica CL in short order by giving our 90 second 'First look' video a watch.
Leica has just released the CL, the forth in its series of APS-C L-mount cameras. Despite sharing a name with a camera released in the mid-70s, the new CL is a thoroughly modern ILC, with a 24MP sensor and built-in electronic viewfinder.
The Leica CL is a 24MP rangefinder-style mirrorless camera, which sits alongside the TL2 in the company's APS-C lineup. We've been using one for a few days – check out our gallery of images.
While it shares a name with one of Leica's most popular and affordable cameras of the 1970s, the new CL is separated from its namesake by more than just years. We've been using one for a few days - click through for a detailed first-impressions report.
We're counting down our top 10 most popular sample galleries of 2017, and the #4 ranking goes to the Leica M10.
Sigma is discounting 13 different high-performance 'Art' series lenses from today until November 30th. The company is calling it an 'unprecedented' sale.
See DJI's 'AeroScope' drone-tracking technology in action. This is the system that DJI says can help law enforcement and airport (among others) track and identify rogue drones.
iPhone X owners can already accessorize their new phone with high-quality smartphone photography lenses courtesy of Moment's new lineup.
Considering buying Sigma's exciting new 16mm F1.4 DC DN Contemporary lens for crop-sensor E-Mount and M43? Check out these official full-res samples first!
Vimeo has just added support for 8K HDR 10-bit content, making it possible to show up to 75% of the colors the human eye can perceive vs the usual 35%. Take THAT YouTube.
The holidays are coming, but your gear isn't cutting it? It's time to treat yourself!
We're counting down our top 10 most popular sample galleries of 2017, and sitting pretty at #5 is the Fujifilm X-T20.
See some of the most iconic black-and-white photographs throughout history brought to life by a community of colorization enthusiasts and professional retouchers in the new book Retrographic.
Shopping for a photographer? Whether you are one yourself or not, chances are you could use some ideas. From stocking stuffers on up, we've got some photography gift suggestions for every budget.
We're counting down our top 10 most popular sample galleries of 2017. Drum roll please... the #6 spot belongs to none other than the Sigma 85mm F1.4 DH HSM Art.
Read the story behind this gorgeous wedding photo captured at Trolltunga in Norway by husband and wife duo Priscila Valentina Photography. The 14 hour hike in the rain that preceded this shot was TOTALLY worth it.
Go behind the scenes with filmmaker Nick Arcivos, who recently created a beautiful cinematic short film in Paris using only the iPhone X, a couple of gimbals, and a few lights. The results are very impressive.
A Bay Area startup offering a pay-by-the-photo camera service cleverly addresses the pain points photographers experience when they pick up their first DSLR. But can it survive the smartphone?
It's been a big year for software innovations, dual cameras and huge displays. Take a look at our picks for the top smartphone cameras and why we think they stand out.
We're counting down our top 10 most popular sample galleries of 2017. At the #7 spot is the ready-for-any-weather Olympus Tough TG-5.
By combining his skills as a time-lapse filmmaker and an engineer, Julian Tryba created this out-of-this-world creative 'layer-lapse' of New York City that alternates between night and day in time with the music.
Canon Japan's new lineup of novelty camera-themed gifts was just revealed online, including a lens mug and lens thermos, two retro camera-themed USB drives, and a picnic mat.
The Profoto A1 most certainly isn’t for everyone [...] But for those who are used to using the Profoto systems, and want something that pairs seamlessly with the strobes you already have, there is no better companion.
Fujifilm has asked a US district court to clear it of any wrongdoing, after allegedly being threatened with trademark litigation by Polaroid.
While a couple of our reviewers are out testing the Sony a7R III in Arizona, back in Seattle we slapped the camera in front of our studio scene to get a close look at its image quality. See how it stacks up against the competition.