Google reveals how to simulate shallow DOF from a single mobile camera
Google has published an 18-page study fully detailing the synthetic depth-of-field technology that makes its single-camera Portrait Mode possible. Google introduced its evolved Portrait Mode feature on the Pixel 2 and Pixel 2 XL, though neither smartphone model has the dual-camera hardware typically required to produce this effect.
The in-depth paper shows a degree of openness unusual for the smartphone and camera industries. Smartphones with a single camera produce images where everything is generally in focus. Dual-camera phones paired with a stereo algorithm get around this limitation by matching points in images from both cameras to determine depth within the captured scene. Having acquired that depth data, some pixels can be selectively blurred to produce the shallow DOF effect, Google explained in a blog post last year.
Achieving this same effect using only a single camera is difficult. Some mobile camera apps attempt to simulate a shallow DOF by separating an image's pixels into two layers, isolating the foreground, and then blurring the remaining pixels; this is called semantic segmentation. The lack of depth data, however, means the software doesn't know how much blur to apply to any arbitrary object in the scene. The results can often be lackluster or unrealistic, without the gradual optical blur expected of objects receding into the distance.
That's where Google's "authentic defocus" technology comes in. The Pixel 2 smartphones utilize the semantic segmentation method for images taken with the front-facing camera, but they also use a stereo algorithm for images taken with the rear camera... despite there only being a single lens. Google provided an overview of how it achieves that on its AI blog in October.
There are advantages to Google's technology versus using a second camera, including reducing the space taken up by the imaging module, reduced power consumption, and helping keep costs down.
Put simply, Google repurposes its dual-pixel auto focus hardware utilized increasingly in mobile cameras for fast AF. Each pixel on the sensor is split into two photodiodes; the left- and right-looking (or up- and down-looking) photodiodes essentially establish two perspectives of the scene with a ~1mm stereo baseline. A burst of images are aligned and averaged to reduce noise, and a stereo algorithm computes a depth map from the two perspectives. This simulates the data that would be provided by two physical cameras next to each other, enabling Google's software to determine the depth of every point within the captured scene.
There's a lot more to Google's approach, including even advantages over traditional optics - for example in its choice to force a larger depth-of-field around the focus plane to ensure a sharp subject, something impossible to achieve optically. The study also points out that there are advantages to Google's technology versus using a second camera, including reducing the space taken up by the imaging module, reduced power consumption, and helping keep costs down.
Read the full PDF here.
|The Lone Photographer by ed rader|
from My Best Photo of the Week
|_ERN9064 by ernesto juarez|
from Shoot yourself ! (with your camera)
|Neighbourhood Watch by Stevie Boy Blue|
from Zoo trip ~ Cute...
A new report from The Verge claims Instagram is currently testing a feature that allows users to re-share posts to their own account feeds.
GoPro has announced its HERO7 camera lineup. The updated action cameras feature new HyperSmooth and TimeWarp modes, as well as improved video and photo specs.
The latest Samsung midrange smartphone offers a super-wide-angle lens in its triple-camera setup.
The Sony 24mm F1.4 is the latest lens to join the company's premium G Master lineup. We've been shooting with one for a couple of days - here's what you need to know.
Apple released iOS 12 a few days ago and some iPhone X users are less than happy with how the new operating system has made their phones look.
Camera bag manufacturer Lowepro has introduced mark II backpacks for its ProTactic AW range with models that are said to feature an improved handling experience as well as a collection of accessories that can be attached to the outside.
Canon has announced its latest superzoom camera, the PowerShot SX70 HS. Compared to the SX60 that came before it, the SX70 has the same lens but offers a higher resolution EVF, 4K video capture and support for Canon's new CR3 Raw format.
Cosina has announced its eighth lens designed specifically for Sony's E-mount system. The Voigtlander 21mm F3.5 lens is due out October 2018.
Sony has taken the wraps off of its new 24mm F1.4 GM full-frame lens, which the company claims is the lightest in its class. Despite its fast aperture, the 24mm F1.4 is remarkably light, weighing just 445 grams (15.7 ounces). The lens will set you back $1400 when it ships next month.
In this episode of DPReview TV we take a look at Sony's brand new 24mm F1.4 GM lens, a desirable focal length for many photographers. How does it perform? Chris and Jordan give us their first impressions.
We've had a little time to shoot with Sony's new wide/fast prime, both close to home and on the water in San Francisco. Check out our initial sample images.
Fujifilm released a firmware upgrade for its X-T3 mirrorless camera that addresses issues with distortion compensation and the mechanical lock on SD cards.
The app's algorithms have been trained using using 200 million cropping data points from real photographers.
Thanks to a software update, the Loupedeck+ editing console can now be used for video editing.
British photographic engineer MTF Services is claiming the world’s first third-party lens adapters for the new Nikon Z system with a collection of four units designed to allow cinema lenses to be mounted on the mirrorless full frame bodies.
Think Tank Photo has updated its line of heavy-duty rain covers and introduced a new, compact version for emergency situations.
The X-T3 is our first opportunity to analyze what's likely to be Fujifilm's next generation image sensor. Take a look at how it performs next to the competition in our studio test scene.
Canon's new normal is seriously sharp wide open. After shooting with it for a few days, we've prepared a gallery of real-world sample images.
Nikon will cease offering Brazil-based customer service and technical support, though the company stresses that it will still offer technical assistance and warranty repairs for valid warranties.
Two years ago, CatLABS of JP announced a plan to save Packfilm from the dead. Now, it's announced it's giving up its efforts to better focus its resources elsewhere.
The GoPro Fusion is designed to make it easy to capture 360-degree video and stills. We took it out recently on a typically hot Seattle summer day to see what it can do.
We've got our hands on a full-production Nikon Z7 camera and have updated our gallery with additional samples.
A new Kickstarter campaign seeks funding for Chroma Chrono, a programmable RGB camera flash that emits multiple colors during long exposures.
Think Tank Photo has launched a new lineup of six dual-access, water-resistant protective lens cases it calls Lens Case Duo.
Canon and Nikon finally entered the full-frame mirrorless market this summer with the brand-new RF and Z mounts. Now that we've had some time with the cameras, we wanted to revisit our earlier predictions and take stock.
The devices' camera specs look pretty much identical to last year's iPhone X but under the hood a number of important improvements have been made.
Blackmagic Design has announced the public beta of its new Blackmagic RAW video codec. The company says the new format combines the benefits of shooting Raw video with the ease of use and smaller file sizes usually associated with non-Raw video files.
Serif, the company behind the Affinity suite, has announced the latest update for its mobile Photoshop competitor Affinity Photo for iPad.
The Atomos Ninja V external video recorder and monitor will be ready to ship at the end of this month. The 5.2in Ninja V is designed to provide a smaller option, while still offering many of the features of the larger 7-inch models.
Having shot with the camera, spoken to Canon and read the tea leaves, here's what DPR Technical Editor Richard Butler thinks the EOS R tells us about Canon and the RF's mount's future.