The Raw and the cooked: pulling apart Sony's Raw compression
|Raw compression won't be apparent in every image, but there are circumstances in which it risks undermining what should be a great image. Photo by Rishi Sanyal|
Update: September 15, 2015: Sony has announced that the a7S II will have the option to shoot uncompressed Raw, with firmware being rolled out for some existing models, including the a7R II used here, later.
A Raw file is a Raw file, right? Well, not exactly. Lately, there's been a lot of talk (and a lot of anger) about the compression Sony uses in its Raw files. Compressed Raw files aren't uncommon, but they're usually compressed in a way that retains all the original 'raw' data from the sensor. Or, at least, all the information that's visually meaningful.
Instead Sony has, for several years now, chosen to apply non-optional lossy compression to its Raw output. This isn't likely to be an issue for many users, since the impact is generally quite small, but as the company shows its ambitions in the high end market, with the launch of the a7 series and a dedicated support system for professionals, the impact on image quality deserves a little scrutiny.
What exactly is going on with Sony's Raw files, and what might the potential impact be? The compression system has been investigated and detailed by Iliah Borg and Alex Tutubalin, and we've tried to distill their findings. It's important to keep this in perspective, though: in many circumstances you won't see this impact or encounter the limitations it can impose.
Sony's compression process has two parts, and each of the two aspects have a different impact on the Raw files. The first step applies a compression curve to the data. This is a bit like a tone curve and is used to map the 14 bits of captured data down to an 11-bit space.
Although this part of the process is lossy, a well-designed compression curve has little to no impact on image quality. This is because although shot noise makes up a decreasing proportion of the captured data in bright regions, the actual magnitude of the noise increases. This means it doesn't make sense to retain all the information about bright regions of the images, since a lot of that information will just be recording the subtleties of the noise.
Ideally, then, you can get away with a lot of compression in bright tones so long as you preserve increasing amounts of information at the dark end of the file. Unlike the (optional) compression curve Nikon uses, Sony's doesn't fully exploit this phenomenon, meaning that some useful data is lost, as well as some of the noise. This will, theoretically, reduce the dynamic range available in the files.
Sony's Raw compression then has a second stage, where the image is divided up into a series of 16 pixel stripes for each color channel. Rather than recording a separate value for each of these pixels, the Sony system records the brightest and darkest value in each stripe, and a series of simple notes about how all the other pixels vary from those extremes. These notes are recorded using fewer bits than it would take to record the actual pixel values and it's this step that appears to cause most of the problems.
When there's not much difference between the brightest and darkest pixel, the system is able to describe the scene pretty well. However, as soon as you have a big gap between bright and dark, the 7-bit values used to note the differences aren't sufficient to precisely describe the original image information.
This imprecise recording of the original image data leads to artifacts in stripes around high-contrast edges in the photos. These errors can become even more pronounced if you increase the brightness or contrast in those regions when processing the files.
14-bit and 12-bit readout
To compound matters, several Sony cameras we've tested appear to switch their sensor read-out from 14 bits to 12 bits in certain modes: further reducing the amount of dynamic range that the camera ever captures, even before the effects of the compression process are brought into play. Continuous shooting, Bracketing and Bulb exposure modes will all push most Sonys down to 12-bit capture mode, which is then subjected to the two stage compression.
This only really matters at base ISO however, since that's the only time you're likely to capture more dynamic range than a 12-bit file can accurately store.
These approaches probably made sense in consumer-grade cameras, back in the days where processing power and storage space came at a significant premium. However, on a camera as expensive as the a7R II, which is likely to be used for quality critical shooting, it's hard to justify clumsy compression that can, depending on the image, throw away data you were expecting to have access to.
Overall, the effects of this compression aren't often visually significant. Their impact should mostly be understood as a reduction in processing latitude, since it tends only to be when you push and pull the Raw files that the missing data becomes visible. The compression curve throws away more shadow data than would be ideal: reducing dynamic range. There's a further reduction if you shoot in a mode that drops the camera into 12-bit readout mode. Meanwhile, the localized compression of tonal differences only has an impact near high-contrast edges.
|And it's not just heavily-pushed images that start to reveal the lost data: this is a straight out-of-camera JPEG file (shot with DRO Auto), still showing stripes extending from a high-contrast edge. It looks still worse in a gently processed Raw conversion.
It's worth noting, though, that the impact of this process is lessened in the company's higher pixel-count bodies, since a 32-pixel stripe will be a smaller proportion of the a7R II's 40MP images than it would in a 12MP image from the a7S.
It's quite possible to shoot for years and never notice the impact of these design choices Sony has made, but they do add up to mean that you can't access the full capability of the camera. Most people shoot Raw precisely because they want to preserve the maximum possible processing latitude and keep their creative options open. Raw compression isn't the end of the world by any means, but it throws away a little bit of a camera's capability, which might be a little hard to swallow if you've paid multiple thousands of dollars for a cutting-edge camera.
Sony's new 12-24mm F2.8 GM is the widest fast aperture zoom for full frame. Based on our tests it's a worthy recipient of Sony's 'GM' moniker.
Chris and Jordan took the new Sony 12-24mm F2.8 GM to Calgary's eclectic Ingelwood neighborhood. From record stores to spice shops, find out what got their attention when it was time to go wide.
The six prize-winning photographs and four honorable mentions were narrowed down from more than 6,000 entries captured across North America.
Though Thunderbolt 4 remains at 40Gb/s, its minimum requirements include dual 4K monitor support, faster external drive speeds and more.
You can now use compatible Fujifilm cameras with video conferencing software on macOS hardware without the need of a dedicated capture card.
The Epson V600 remains one of the most popular flatbed film scanners on the market. Revisit our review of this affordable and (mostly) easy-to-use option and see how its output compares to local lab scans.
Canon's mirrorless EOS R5 comes with a ton of features and capability stemming from its design inside and out. Come along with us on a guided tour of Canon's new high-end, high-megapixel camera and check it out for yourself.
Announced alongside the EOS R5, the R6 offers a lot of the same technology but in a more affordable, slightly more enthusiast-focused model. Take a closer look.
Alongside the EOS R5 and R6, Canon has announced a brace of lenses, all in the short to long telephoto range. Filling out the 'long' end are one L-series zoom, and two innovative primes.
Alongside a trio of telephoto lenses, Canon also announced a new 85mm this week. The RF 85mm F2 Macro IS STM is a compact, affordable alternative to the pro-oriented 85mm F1.2L.
The EOS R5 has been a long time coming – we knew it had 8K and we knew it had an AF joystick. But now that's it's here, what is it really like to use? Find out in our initial review based on hands-on time with the camera.
The R6 doesn't promise quite such headline-grabbing specs as its big brother, but it still packs a punch, whether you shoot stills, video or both.
Think you've read everything there is to know about the new Canon cameras? Chris and Jordan share eight important things you may have missed from today's Canon EOS R5 and R6 announcements.
We've been shooting around with the new Canon EOS R6. Initial impressions of image quality are positive, and out-of-camera JPEGs appear similar to that of the gold award-winning Canon EOS-1D X III. Have a look for yourself.
Canon has officially released the long-awaited EOS R5, the company's top-end full-frame mirrorless camera. Featuring a new 45MP CMOS sensor, Dual Pixel AF II system, 8K video capture and 20 fps bursts, this is the RF-mount camera we've been waiting for.
Although the Canon EOS R6 doesn't have the 45MP sensor and 8K video capture of the higher-end R5, it's still an incredibly capable camera with specs that outshine similarly priced peers.
The Canon RF 100-500mm F4.5-7.1L IS USM is the company's first super-zoom lens for RF-mount. Despite a relatively slow aperture range, it's very versatile, offering five stops of stabilization, weather-sealing and compatibility with Canon's new teleconverters.
Canon's RF 85mm F2 Macro IS STM is an inexpensive telephoto prime lens with a minimum focus distance of just 0.35m (14") and a 0.5x magnification. When attached to the new R5 and R6, it offers a whopping eight stops of shake reduction.
Canon has announced a pair of super-telephoto fixed-aperture primes. The 600mm and 800mm use diffractive optics to keep their size and weight down. They'll also be compatible with new 1.4x and 2x RF teleconverters.
Canon has announced a new small-footprint inkjet photo printer, the imageProGraf Pro-300. it will produce prints up to 13 x 19" and it goes on sale later this month for $900. A new textured photo paper will also arrive in July.
The new compression standard is set to reduce video file sizes by half to save space and speed-up transmission, paving the way for more portable 8K footage.
Sony recently confirmed plans to launch a successor to the video-centric a7S II. We don't even know the name of the camera, but Jordan already has a feature wish list for the new 'a7S III' – and it doesn't include 8K.
The Profot B10 is the first studio flash system that can be used when shooting with an iPhone camera.
The Pixii camera is an interesting little rangefinder camera that features a 12MP APS-C sensor and lacks a rear LCD display, opting instead to pair with your mobile device, which can be used to view and transfer images.
Sirui is launching an Indiegogo campaign for a wide-angle answer to its existing 50mm F1.8 anamorphic lens. The 35mm APS-C lens will come in a Micro Four Thirds mount with adapters for other systems.
Sony has added a 12-24mm F2.8 to its top-shelf 'G Master' series of lenses. It's the widest constant F2.8 zoom currently offered for full-frame, with a hefty price tag to match: it will sell for $3000 when it ships in mid-August.
Take a look at the view from Sony's new ultra-wide F2.8 zoom – we paired it with the a7R IV for some initial shooting.
Canon's EOS-1D X Mark III is one of the best DSLRs ever made. With fast burst speeds, great video quality and impressive autofocus, the 1D X III is equal parts cinema rig and sports shooter. Find out how it fares against steep competition in our full review.
Nikon Rumors is reporting that Nikon will announce successors to its Z6 and Z7 camera systems by the end of the calendar year.
Canon says the event, set to take place at 14:00 CEST in two days on July 9, will be its 'biggest product launch yet.'