This NVIDIA algorithm copies the artistic style of one photo onto another
Struggling with stylistic consistency, or wanting to transpose the style of your best picture onto the rest of your Instagram feed? Thanks to a group of scientists at Cornell University, you can now do just that with surprisingly accurate and realistic results.
The team created an algorithm for graphics card company NVIDIA that lifts the stylistic characteristics of one picture and drops them onto a completely different image with startling precision. The algorithm is called FastPhotoStyle, and it's capable of transferring the coloration, drama and atmosphere of one picture and making an entirely different frame look as though it was taken at the same time even if the subject matter is totally unrelated.
According to the developers, the goal of photorealistic image style transfer is:
...to change the style of a photo to resemble that of another one. For a faithful stylization, the content in the output photo should remain the same, while the style of the output photo should resemble the one of the reference photo. Furthermore, the output photo should look like a real photo captured by a camera.
There are programs already invented to do this, but the inventors of this algorithm claim that what already exists is slow, and doesn't produce realistic results anyhow.
FastPhotoStyle is different, they say, because it uses a smoothing process after the initial whitening and Coloring Transfer step—or PhotoWCT step. This smoothing step tries to ensure that neighboring pixels receive similar styling and, by using what they call Matting Affinity, individual areas of the image can be subjected to slightly different treatment. This is what helps the algorithm produce such realistic looking results.
Another major difference is that this program reportedly operates as much as 60x faster than existing algorithms.
The code can be downloaded from NVIDIA's GitHub for anyone to use under Creative Commons license (BY-NC-SA 4.0), and a user manual download is included on the page. If you're brave, you can read the full technical paper as well.
Technical Paper Abstract:
A Closed-Form Solution to Photorealistic Image Stylization
Photorealistic image style transfer algorithms aim at stylizing a content photo using the style of a reference photo with the constraint that the stylized photo should remains photorealistic.
While several methods exist for this task, they tend to generate spatially inconsistent stylizations with noticeable artifacts. In addition, these methods are computationally expensive, requiring several minutes to stylize a VGA photo. In this paper, we present a novel algorithm to address the limitations.
The proposed algorithm consists of a stylization step and a smoothing step. While the stylization step transfers the style of the reference photo to the content photo, the smoothing step encourages spatially consistent stylizations. Unlike existing algorithms that require iterative optimization, both steps in our algorithm have closed-form solutions.
Experimental results show that the stylized photos generated by our algorithm are twice more preferred by human subjects in average. Moreover, our method runs 60 times faster than the state-of-the-art approach.
Apple has updated its professional video editing app Final Cut Pro X to version 10.4.6. The update brings full 64-bit support, a new feature that helps convert older formats and much more.
Tonight's episode of NBC's Tonight Show, hosted by Jimmy Fallon, was filmed entirely on Samsung's flagship smartphone the Galaxy S10+.
Camera Bits has released the long-awaited update to its photo ingestion software in the form of Photo Mechanic 6.
SmugMug Films has shared its latest film, Streets in Mind, which takes a look at the life and work of London-based street photographer Alan Schaller.
We were in Japan earlier this month for the annual CP+ show in Yokohama, where we sat down with senior executives from several camera and lens manufacturers, among them Nikon.
Sony has released firmware version 5.0 for its flagship mirrorless camera, the a9. The update brings AI-driven autofocus modes, an improved menu structure and other updates.
Night Sight, Portrait Mode and (surprisingly) wide-angle selfie mode are features that we're currently loving about the Pixel 3's camera.
The Auschwitz Museum has asked visitors to be more respectful after an upsurge of pictures posted on social media showing people posing on the train tracks that lead to the main gate.
This week Chris and Jordan take the new Leica Q2 for a spin, and while most of us in the Northern Hemisphere are welcoming spring, they head even farther north than usual to visit ice castles. Because #Canada.
Harvard is facing a lawsuit over profiting from 19th century daguerreotypes that captured the portrait of a slave and his daughter on a South Carolina plantation.
From the detailed textures in rural landscapes to the incredible lighting inside futuristic buildings, the photorealism of Unreal Engine 4 is blurring the lines between fiction and reality...you know...aside from the spaceship.
Facebook has sent out emails to affected users requesting they change their passwords following a discovery that over 20K Facebook employees had access to 600 million passwords.
We've added Panasonic's new Lumix S1 and S1R full-frame mirrorless cameras to three of our buying guides. If you're looking for a quick summary of each model, then have a read.
YouTube channel Photoshop Cafe has shared a video detailing ten tips and tricks you can do to both fix and speed up Photoshop when it's running slow and sluggish.
It's not going to be the banger of the year, but it'll get a few laughs.
DJI has confirmed its drones won't be affected by the GPS 2019 week rollover.
Dogfish Head Craft Brewery has teamed up with Kodak to release a beer that's capable of doubling as a film developer.
The Diana Instant Square is a retro-inspired camera with manual controls that's fun to shoot in good light, but largely unpredictable in its operation.
Residents of a Paris street plagued by Instagrammers, selfie takers and music video crews are asking the city government for a weekend and evening ban to give them some peace.
The adapter plugs into the Osmo Pocket's USB Type-C port and features a 3.5mm TRS jack to plug in various external microphones.
Checkout allows Instagram users to select products for purchase and make payments directly in the app.
GauGAN as it's known, can create photorealistic images from basic drawings using the power of artificial intelligence.
The EOS RP is Canon's latest full-frame mirrorless camera, with diminutive dimensions and a diminutive price. Find out how it stacks up and get our thoughts in our early review.
Montana judge Dana L. Christensen has ruled the Republican National Committee did not infringe upon the copyright of photographer Erika Peterman after they took a photo from a Democratic candidate's Facebook page without permission and altered it to use in a derogatory promotional mailer.
Nikon has launched updates for three of its programs to address various bugs and glitches that could cause crashes and unwanted results.
LEE Filters has launched the LEE100, its next-generation filter holder that improves the design and looks in all the right places.
With the arrival of some much-needed sunshine and final production firmware for the Panasonic S1, we've been able to get outside and really start putting the camera through its paces.
Importing, culling and tagging photos is about to get a whole lot faster and look a whole lot better with the impending arrival of Photo Mechanic 6.
On its own, the FTZ adapter retails for $250 and when bundled it dropped the cost to just $150. Now, Nikon is offering it for free with all Z6, Z7 purchases in the United States.
Profoto said it spoke with Godox back at Photokina 2018 and continues to contact Godox in an effort to stop it from marketing its V1 light.