The Raw and the cooked: pulling apart Sony's Raw compression
|Raw compression won't be apparent in every image, but there are circumstances in which it risks undermining what should be a great image. Photo by Rishi Sanyal|
Update: September 15, 2015: Sony has announced that the a7S II will have the option to shoot uncompressed Raw, with firmware being rolled out for some existing models, including the a7R II used here, later.
A Raw file is a Raw file, right? Well, not exactly. Lately, there's been a lot of talk (and a lot of anger) about the compression Sony uses in its Raw files. Compressed Raw files aren't uncommon, but they're usually compressed in a way that retains all the original 'raw' data from the sensor. Or, at least, all the information that's visually meaningful.
Instead Sony has, for several years now, chosen to apply non-optional lossy compression to its Raw output. This isn't likely to be an issue for many users, since the impact is generally quite small, but as the company shows its ambitions in the high end market, with the launch of the a7 series and a dedicated support system for professionals, the impact on image quality deserves a little scrutiny.
What exactly is going on with Sony's Raw files, and what might the potential impact be? The compression system has been investigated and detailed by Iliah Borg and Alex Tutubalin, and we've tried to distill their findings. It's important to keep this in perspective, though: in many circumstances you won't see this impact or encounter the limitations it can impose.
Sony's compression process has two parts, and each of the two aspects have a different impact on the Raw files. The first step applies a compression curve to the data. This is a bit like a tone curve and is used to map the 14 bits of captured data down to an 11-bit space.
Although this part of the process is lossy, a well-designed compression curve has little to no impact on image quality. This is because although shot noise makes up a decreasing proportion of the captured data in bright regions, the actual magnitude of the noise increases. This means it doesn't make sense to retain all the information about bright regions of the images, since a lot of that information will just be recording the subtleties of the noise.
Ideally, then, you can get away with a lot of compression in bright tones so long as you preserve increasing amounts of information at the dark end of the file. Unlike the (optional) compression curve Nikon uses, Sony's doesn't fully exploit this phenomenon, meaning that some useful data is lost, as well as some of the noise. This will, theoretically, reduce the dynamic range available in the files.
Sony's Raw compression then has a second stage, where the image is divided up into a series of 16 pixel stripes for each color channel. Rather than recording a separate value for each of these pixels, the Sony system records the brightest and darkest value in each stripe, and a series of simple notes about how all the other pixels vary from those extremes. These notes are recorded using fewer bits than it would take to record the actual pixel values and it's this step that appears to cause most of the problems.
When there's not much difference between the brightest and darkest pixel, the system is able to describe the scene pretty well. However, as soon as you have a big gap between bright and dark, the 7-bit values used to note the differences aren't sufficient to precisely describe the original image information.
This imprecise recording of the original image data leads to artifacts in stripes around high-contrast edges in the photos. These errors can become even more pronounced if you increase the brightness or contrast in those regions when processing the files.
14-bit and 12-bit readout
To compound matters, several Sony cameras we've tested appear to switch their sensor read-out from 14 bits to 12 bits in certain modes: further reducing the amount of dynamic range that the camera ever captures, even before the effects of the compression process are brought into play. Continuous shooting, Bracketing and Bulb exposure modes will all push most Sonys down to 12-bit capture mode, which is then subjected to the two stage compression.
This only really matters at base ISO however, since that's the only time you're likely to capture more dynamic range than a 12-bit file can accurately store.
These approaches probably made sense in consumer-grade cameras, back in the days where processing power and storage space came at a significant premium. However, on a camera as expensive as the a7R II, which is likely to be used for quality critical shooting, it's hard to justify clumsy compression that can, depending on the image, throw away data you were expecting to have access to.
Overall, the effects of this compression aren't often visually significant. Their impact should mostly be understood as a reduction in processing latitude, since it tends only to be when you push and pull the Raw files that the missing data becomes visible. The compression curve throws away more shadow data than would be ideal: reducing dynamic range. There's a further reduction if you shoot in a mode that drops the camera into 12-bit readout mode. Meanwhile, the localized compression of tonal differences only has an impact near high-contrast edges.
|And it's not just heavily-pushed images that start to reveal the lost data: this is a straight out-of-camera JPEG file (shot with DRO Auto), still showing stripes extending from a high-contrast edge. It looks still worse in a gently processed Raw conversion.
It's worth noting, though, that the impact of this process is lessened in the company's higher pixel-count bodies, since a 32-pixel stripe will be a smaller proportion of the a7R II's 40MP images than it would in a 12MP image from the a7S.
It's quite possible to shoot for years and never notice the impact of these design choices Sony has made, but they do add up to mean that you can't access the full capability of the camera. Most people shoot Raw precisely because they want to preserve the maximum possible processing latitude and keep their creative options open. Raw compression isn't the end of the world by any means, but it throws away a little bit of a camera's capability, which might be a little hard to swallow if you've paid multiple thousands of dollars for a cutting-edge camera.
Nov 16, 2017
Nov 20, 2017
Nov 21, 2017
Nov 17, 2017
|Madrid subway by MAGMATCICO62|
from Your City - Public Transport
|Incandescent Bulb by Kukla|
from Illuminate- Macro only
|Curiousity by PERCY2|
from Macro - Your Best Macro Ever
|Hoar Frosted Trees by sabishiT3T|
There's never been a better time to shop for a new camera, but the number of options available can be overwhelming. In this series of buying guides we've provided customized recommendations for several use cases, from shooting landscapes to buying a first camera for a student photographer.
Shopping for a camera with a set budget? No problem! We've rounded up our favorite cameras, broken them into price brackets and picked the best of the bunch.
Looking for a lightweight compact camera that's easy to bring with you anywhere? Or maybe you're smartphone-shopping and want the one that takes the best picture. And what if you want to shoot from above? In these buyers guides we have recommendations for the best compact cameras, smartphones and drones.
Despite reports to the contrary, analysis of DPReview images by our friend Jim Kasson confirms a disappointing fact: Sony a7R III is still a Star Eater. But there may be some improvements.
As the saying goes: A photo is worth a thousand words. And if you're sending that photo through Facebook Messenger, your thousand words now look twice as nice after today's update to 4K resolution.
Get to know the new Leica CL in short order by giving our 90 second 'First look' video a watch.
Leica has just released the CL, the forth in its series of APS-C L-mount cameras. Despite sharing a name with a camera released in the mid-70s, the new CL is a thoroughly modern ILC, with a 24MP sensor and built-in electronic viewfinder.
The Leica CL is a 24MP rangefinder-style mirrorless camera, which sits alongside the TL2 in the company's APS-C lineup. We've been using one for a few days – check out our gallery of images.
While it shares a name with one of Leica's most popular and affordable cameras of the 1970s, the new CL is separated from its namesake by more than just years. We've been using one for a few days - click through for a detailed first-impressions report.
We're counting down our top 10 most popular sample galleries of 2017, and the #4 ranking goes to the Leica M10.
Sigma is discounting 13 different high-performance 'Art' series lenses from today until November 30th. The company is calling it an 'unprecedented' sale.
See DJI's 'AeroScope' drone-tracking technology in action. This is the system that DJI says can help law enforcement and airport (among others) track and identify rogue drones.
iPhone X owners can already accessorize their new phone with high-quality smartphone photography lenses courtesy of Moment's new lineup.
Considering buying Sigma's exciting new 16mm F1.4 DC DN Contemporary lens for crop-sensor E-Mount and M43? Check out these official full-res samples first!
Vimeo has just added support for 8K HDR 10-bit content, making it possible to show up to 75% of the colors the human eye can perceive vs the usual 35%. Take THAT YouTube.
The holidays are coming, but your gear isn't cutting it? It's time to treat yourself!
We're counting down our top 10 most popular sample galleries of 2017, and sitting pretty at #5 is the Fujifilm X-T20.
See some of the most iconic black-and-white photographs throughout history brought to life by a community of colorization enthusiasts and professional retouchers in the new book Retrographic.
Shopping for a photographer? Whether you are one yourself or not, chances are you could use some ideas. From stocking stuffers on up, we've got some photography gift suggestions for every budget.
We're counting down our top 10 most popular sample galleries of 2017. Drum roll please... the #6 spot belongs to none other than the Sigma 85mm F1.4 DH HSM Art.
Read the story behind this gorgeous wedding photo captured at Trolltunga in Norway by husband and wife duo Priscila Valentina Photography. The 14 hour hike in the rain that preceded this shot was TOTALLY worth it.
Go behind the scenes with filmmaker Nick Arcivos, who recently created a beautiful cinematic short film in Paris using only the iPhone X, a couple of gimbals, and a few lights. The results are very impressive.
A Bay Area startup offering a pay-by-the-photo camera service cleverly addresses the pain points photographers experience when they pick up their first DSLR. But can it survive the smartphone?
It's been a big year for software innovations, dual cameras and huge displays. Take a look at our picks for the top smartphone cameras and why we think they stand out.
We're counting down our top 10 most popular sample galleries of 2017. At the #7 spot is the ready-for-any-weather Olympus Tough TG-5.
By combining his skills as a time-lapse filmmaker and an engineer, Julian Tryba created this out-of-this-world creative 'layer-lapse' of New York City that alternates between night and day in time with the music.
Canon Japan's new lineup of novelty camera-themed gifts was just revealed online, including a lens mug and lens thermos, two retro camera-themed USB drives, and a picnic mat.
The Profoto A1 most certainly isn’t for everyone [...] But for those who are used to using the Profoto systems, and want something that pairs seamlessly with the strobes you already have, there is no better companion.
Fujifilm has asked a US district court to clear it of any wrongdoing, after allegedly being threatened with trademark litigation by Polaroid.
While a couple of our reviewers are out testing the Sony a7R III in Arizona, back in Seattle we slapped the camera in front of our studio scene to get a close look at its image quality. See how it stacks up against the competition.