The Raw and the cooked: pulling apart Sony's Raw compression
|Raw compression won't be apparent in every image, but there are circumstances in which it risks undermining what should be a great image. Photo by Rishi Sanyal|
Update: September 15, 2015: Sony has announced that the a7S II will have the option to shoot uncompressed Raw, with firmware being rolled out for some existing models, including the a7R II used here, later.
A Raw file is a Raw file, right? Well, not exactly. Lately, there's been a lot of talk (and a lot of anger) about the compression Sony uses in its Raw files. Compressed Raw files aren't uncommon, but they're usually compressed in a way that retains all the original 'raw' data from the sensor.
Instead Sony has, for several years now, chosen to apply non-optional lossy compression to its Raw output. This isn't likely to be an issue for many users, since the impact is generally quite small, but as the company shows its ambitions in the high end market, with the launch of the a7 series and a dedicated support system for professionals, the impact on image quality deserves a little scrutiny.
What exactly is going on with Sony's Raw files, and what might the potential impact be? The compression system has been investigated and detailed by Iliah Borg and Alex Tutubalin, and we've tried to distill their findings. It's important to keep this in perspective, though: in many circumstances you won't see this impact or encounter the limitations it can impose.
Sony's compression process has two parts, and each of the two aspects have a different impact on the Raw files. The first step applies a compression curve to the data. This is a bit like a tone curve and is used to map the 14 bits of captured data down to an 11-bit space.
Although this part of the process is lossy, a well-designed compression curve has very little impact on image quality. This is because although shot noise makes up a decreasing proportion of the captured data in bright regions, the actual magnitude of the noise increases. This means it doesn't make sense to retain all the information about bright regions of the images, since a lot of that information will just be recording the subtleties of the noise.
Ideally, then, you can get away with a lot of compression in bright tones so long as you preserve increasing amounts of information at the dark end of the file. Unlike the (optional) compression curve Nikon uses, Sony's doesn't fully exploit this phenomenon, meaning that some useful data is lost, as well as some of the noise. This will, theoretically, reduce the dynamic range available in the files.
Sony's Raw compression then has a second stage, where the image is divided up into a series of 16 pixel stripes for each color channel. Rather than recording a separate value for each of these pixels, the Sony system records the brightest and darkest value in each stripe, and a series of simple notes about how all the other pixels vary from those extremes. These notes are recorded using fewer bits than it would take to record the actual pixel values and it's this step that appears to cause most of the problems.
When there's not much difference between the brightest and darkest pixel, the system is able to describe the scene pretty well. However, as soon as you have a big gap between bright and dark, then the 7-bit values used to note the differences of the individual pixels aren't sufficient to precisely describe the original image information.
This imprecise recording of the original image data leads to artifacts in stripes around high-contrast edges in the photos. These errors can become even more pronounced if you increase the brightness or contrast in those regions when processing the files.
14-bit and 12-bit readout
To compound matters, several Sony cameras we've tested appear to switch their sensor read-out from 14 bits to 12 bits in certain modes: further reducing the amount of dynamic range that the camera ever captures, even before the effects of the compression process are brought into play. Continuous shooting, Bracketing and Bulb exposure modes will all push most Sonys down to 12-bit capture mode, which is then subjected to the two stage compression.
These approaches probably made sense in consumer-grade cameras, back in the days where processing power and storage space came at a significant premium. However, on a camera as expensive as the a7R II, which is likely to be used for quality critical shooting, it's hard to justify clumsy compression that can, depending on the image, throw away data you were expecting to have access to.
Overall, the effects of this compression aren't often visually significant. Their impact should mostly be understood as a reduction in processing latitude, since it tends only to be when you push and pull the Raw files that the missing data becomes visible. The compression curve throws away more shadow data than would be ideal: reducing dynamic range. There's a further reduction if you shoot in a mode that drops the camera into 12-bit readout mode. Meanwhile, the localized compression of tonal differences only has an impact near high-contrast edges.
|And it's not just heavily-pushed images that start to reveal the lost data: this is a straight out-of-camera JPEG file (shot with DRO Auto), still showing stripes extending from a high-contrast edge. It looks still worse in a gently processed Raw conversion.
It's worth noting, though, that the impact of this process is lessened in the company's higher pixel-count bodies, since a 32-pixel stripe will be a smaller proportion of the a7R II's 40MP images than it would in a 12MP image from the a7S.
It's quite possible to shoot for years and never notice the impact of these design choices Sony has made, but they do add up to mean that you can't access the full capability of the camera. Most people shoot Raw precisely because they want to preserve the maximum possible processing latitude and keep their creative options open. Raw compression isn't the end of the world by any means, but it throws away a little bit of a camera's capability, which might be a little hard to swallow if you've paid multiple thousands of dollars for a cutting-edge camera.
May 24, 2017
May 18, 2017
May 22, 2017
May 18, 2017
|Fangorn Forest by cand1d|
|Yosemite Falls with Moonbow by Jonathan Shapiro|
from Best Landscape of the Week 4
The new stuff should have better red hues, improved sensitivity and finer grain - but don't worry - will still shift blues to green, greens to purple and yellows to pink.
Ricoh has introduced a new rugged compact camera with a 16MP CMOS sensor, 28-140mm lens, 2.7" LCD and built-in LED macro lights. Read more
This compact drone can shoot HD video using a 2-axis stabilized 12MP camera. Read more
The new Prynt Pocket can print a photo directly from their iPhone simply by inserting the phone into the printer, then snapping a photo. Each print will cost about 50 cents. Read more
Updates for Adobe Camera Raw and Lightroom CC bring support for the Sony A9 and Panasonic ZS70/TZ90, along with bug fixes.
The Triggertrap remote camera control system is no longer sold due to the company folding, but now users will be able to build their own. Read more
The Magic Format Converter comes with internal optics that expand the image circle of full-frame DSLR lenses for use on the Fuji medium format camera. Read more
The usually Apple-exclusive MacPhun software developer has announced that it will introduce PC versions of two of its most popular applications. Both Aurora HDR and Luminar should be available for the Windows operating system by the autumn of this year. Read more
Sony's newest G Master telephoto zoom, announced alongside the a9, is the first of the company's FE lenses to reach 400mm natively. We had one in California and photographed horses, portraits, and landscapes - check out how it did. Read more
Garmin has entered the 360-camera market with the VIRB, which captures 5.7K video at 30p as well as 15MP stills. Read more
German media reports that the founders of the company behind the Panono 360-degree ball camera have filed for bankruptcy at a court in Berlin. Read more
With a claimed 800 new custom parts, Microsoft's updated Surface Pro comes with the latest Kaby Lake processors, better battery life, a new hinge, plus the Surface Pen is updated as well. Read more
DW Photo is attempting to resurrect the Hy6 medium format camera, though the legal tangles of its development may stop it being branded Rolleiflex.
The Kodak EKTRA, the company's 'camera first' smartphone, is now available to purchase in the United States. Read more
Apple and Nokia have settled their years-old patent dispute. Apple will make an undisclosed payment to Nokia and sign a licensing agreement related to digital health products with the Finnish company.
David Gibson, one of Britain's best known street shooters, shares all.
Photographers from the SKYGLOW project travelled 150k miles and took 3 million photos in increasingly rare locations: those without light pollution.
The world's fastest 200mm was produced for 16 years. In that time, only 8000 were made.
Photokina, the biennial photo industry trade show in Cologne, Germany, has announced that it will become an annual event beginning in 2018, and expand its focus to additional areas of imaging technology. Read more
No mic socket? No problem. In this video, Daniel Peters at Photo Gear News shows you how to make a lapel microphone using just a smartphone and a pair of earbuds.
How does the iPhone 7 Plus stack up against the Arri Alexa cinema camera? Watch this short video to find out.
Canon Australia's video series "The Lab" is designed to make photographers experiment and think outside the box. In the latest video a group of photographers create images based on their sense of taste.
The GH5 is expected to get a firmware update this summer to support 400Mbps internal recording. NewsShooter explores what memory cards you'll need to make it work.
Microsoft's new Surface Pro offers Intel's latest processor generation and improved battery life.
Riding a mountain bike downhill is dangerous enough in daylight, but potentially lethal at night. Which is where drones come in.
Rumors abound that Canon (and maybe Nikon) may produce a mirrorless camera based using their existing DSLR mount. Does this guarantee immediate great lens choice or a perpetually second-rate experience? Read more
According to rumors, the next camera from Nest will be able to capture 4K video, though that resolution will be only used for 'virtual' pan and tilt functions.
Boundary's Prima 'fully modular' backpack is expandable to 30L and has a removable camera case and tablet sleeve. Early Kickstarter backers can get one for $189.
Stanley Greene captured 'brutally honest' photographs in the war zones of the Middle East, Chechnya and Georgia. He was also one of the few African-American photographers working internationally.
Owners of Leica M cameras that suffer from peeling CCDs will be able to claim a free repair in the future so long as the camera was purchased within five years of the fault becoming apparent, the company has announced. Read more