Google's Night Sight allows for photography in near darkness
Google's latest Pixel 3 smartphone generation comes with the company's new Night Sight feature that allows for the capture of well-exposed and clean images in near darkness, without using a tripod or flash. Today Google published a post on its Research Blog, explaining in detail the computational photography and machine learning techniques used by the feature and describing the challenges the development team had to overcome in order to capture the desired image results.
Night Sight builds on Google's multi-frame-merging HDR+ mode that was first introduced in 2014, but takes things a few steps further, merging a larger number of frames and aiming to improve image quality in extremely low light levels between 3 lux and 0.3 lux.
One key difference between HDR+ and Night Sight are longer exposure times for individual frames, allowing for lower noise levels. HDR+ uses short exposures to provide a minimum frame rate in the viewfinder image and instant image capture using zero-shutter-lag technology. Night Sight waits until after you press the shutter button before capturing images which means users need to hold still for a short time after pressing the shutter but achieve much cleaner images.
The longer per-frame exposure times could also result in motion blur caused by handshake or moving objects in the scene. This problem is solved by measuring motion in a scene and setting an exposure time that minimizes blur. Exposure times also vary based on a number of other factors, including whether the camera features OIS and the device motion detected by the gyroscope. In addition to varying per-frame exposure, Night Sight also varies the number of frames that are captured and merged, 6 if the phone is on a tripod and up to 15 if it is handheld.
An interesting problem with very dim scenes is how to properly white balance them, given the often dim or strongly colored light sources present in dark scenes. To solve this, Google took a learning-based approach to white balance, training their AWB algorithm to discriminate between poor and well-white balanced images. They did this by hand-correcting images from the Pixel 3 camera using its traditional AWB and using these corrected images to train the algorithm to suggest appropriate color shifts to achieve a more neutral output. The results are impressive (see below).
|Google's learning-based white balance approach yields pleasing colors in this nighttime cityscape, more pleasing than this same shot taken without Night Sight. Photo: Rishi Sanyal|
Frame alignment and merging are additional challenges that you can read all about in detail on the Google Research Blog. Of particular importance: Night Sight feeds the 15 frames into its 'Super Res' pipeline, which means you won't just get low noise images, you'll also get far more detail thanks to the ability to resolve inter-pixel detail and forego demosaicing altogether (which also means far less aliasing and moiré). Not just in nighttime shots but even in the daytime, if you enable Night Sight.
Our science editor Rishi Sanyal also had a closer look at Night Sight and the Pixel 3's other computational imaging features in this article.
Apple has updated its professional video editing app Final Cut Pro X to version 10.4.6. The update brings full 64-bit support, a new feature that helps convert older formats and much more.
Tonight's episode of NBC's Tonight Show, hosted by Jimmy Fallon, was filmed entirely on Samsung's flagship smartphone the Galaxy S10+.
Camera Bits has released the long-awaited update to its photo ingestion software in the form of Photo Mechanic 6.
SmugMug Films has shared its latest film, Streets in Mind, which takes a look at the life and work of London-based street photographer Alan Schaller.
We were in Japan earlier this month for the annual CP+ show in Yokohama, where we sat down with senior executives from several camera and lens manufacturers, among them Nikon.
Sony has released firmware version 5.0 for its flagship mirrorless camera, the a9. The update brings AI-driven autofocus modes, an improved menu structure and other updates.
Night Sight, Portrait Mode and (surprisingly) wide-angle selfie mode are features that we're currently loving about the Pixel 3's camera.
The Auschwitz Museum has asked visitors to be more respectful after an upsurge of pictures posted on social media showing people posing on the train tracks that lead to the main gate.
This week Chris and Jordan take the new Leica Q2 for a spin, and while most of us in the Northern Hemisphere are welcoming spring, they head even farther north than usual to visit ice castles. Because #Canada.
Harvard is facing a lawsuit over profiting from 19th century daguerreotypes that captured the portrait of a slave and his daughter on a South Carolina plantation.
From the detailed textures in rural landscapes to the incredible lighting inside futuristic buildings, the photorealism of Unreal Engine 4 is blurring the lines between fiction and reality...you know...aside from the spaceship.
Facebook has sent out emails to affected users requesting they change their passwords following a discovery that over 20K Facebook employees had access to 600 million passwords.
We've added Panasonic's new Lumix S1 and S1R full-frame mirrorless cameras to three of our buying guides. If you're looking for a quick summary of each model, then have a read.
YouTube channel Photoshop Cafe has shared a video detailing ten tips and tricks you can do to both fix and speed up Photoshop when it's running slow and sluggish.
It's not going to be the banger of the year, but it'll get a few laughs.
DJI has confirmed its drones won't be affected by the GPS 2019 week rollover.
Dogfish Head Craft Brewery has teamed up with Kodak to release a beer that's capable of doubling as a film developer.
The Diana Instant Square is a retro-inspired camera with manual controls that's fun to shoot in good light, but largely unpredictable in its operation.
Residents of a Paris street plagued by Instagrammers, selfie takers and music video crews are asking the city government for a weekend and evening ban to give them some peace.
The adapter plugs into the Osmo Pocket's USB Type-C port and features a 3.5mm TRS jack to plug in various external microphones.
Checkout allows Instagram users to select products for purchase and make payments directly in the app.
GauGAN as it's known, can create photorealistic images from basic drawings using the power of artificial intelligence.
The EOS RP is Canon's latest full-frame mirrorless camera, with diminutive dimensions and a diminutive price. Find out how it stacks up and get our thoughts in our early review.
Montana judge Dana L. Christensen has ruled the Republican National Committee did not infringe upon the copyright of photographer Erika Peterman after they took a photo from a Democratic candidate's Facebook page without permission and altered it to use in a derogatory promotional mailer.
Nikon has launched updates for three of its programs to address various bugs and glitches that could cause crashes and unwanted results.
LEE Filters has launched the LEE100, its next-generation filter holder that improves the design and looks in all the right places.
With the arrival of some much-needed sunshine and final production firmware for the Panasonic S1, we've been able to get outside and really start putting the camera through its paces.
Importing, culling and tagging photos is about to get a whole lot faster and look a whole lot better with the impending arrival of Photo Mechanic 6.
On its own, the FTZ adapter retails for $250 and when bundled it dropped the cost to just $150. Now, Nikon is offering it for free with all Z6, Z7 purchases in the United States.
Profoto said it spoke with Godox back at Photokina 2018 and continues to contact Godox in an effort to stop it from marketing its V1 light.