This NVIDIA algorithm copies the artistic style of one photo onto another
Struggling with stylistic consistency, or wanting to transpose the style of your best picture onto the rest of your Instagram feed? Thanks to a group of scientists at Cornell University, you can now do just that with surprisingly accurate and realistic results.
The team created an algorithm for graphics card company NVIDIA that lifts the stylistic characteristics of one picture and drops them onto a completely different image with startling precision. The algorithm is called FastPhotoStyle, and it's capable of transferring the coloration, drama and atmosphere of one picture and making an entirely different frame look as though it was taken at the same time even if the subject matter is totally unrelated.
According to the developers, the goal of photorealistic image style transfer is:
...to change the style of a photo to resemble that of another one. For a faithful stylization, the content in the output photo should remain the same, while the style of the output photo should resemble the one of the reference photo. Furthermore, the output photo should look like a real photo captured by a camera.
There are programs already invented to do this, but the inventors of this algorithm claim that what already exists is slow, and doesn't produce realistic results anyhow.
FastPhotoStyle is different, they say, because it uses a smoothing process after the initial whitening and Coloring Transfer step—or PhotoWCT step. This smoothing step tries to ensure that neighboring pixels receive similar styling and, by using what they call Matting Affinity, individual areas of the image can be subjected to slightly different treatment. This is what helps the algorithm produce such realistic looking results.
Another major difference is that this program reportedly operates as much as 60x faster than existing algorithms.
The code can be downloaded from NVIDIA's GitHub for anyone to use under Creative Commons license (BY-NC-SA 4.0), and a user manual download is included on the page. If you're brave, you can read the full technical paper as well.
Technical Paper Abstract:
A Closed-Form Solution to Photorealistic Image Stylization
Photorealistic image style transfer algorithms aim at stylizing a content photo using the style of a reference photo with the constraint that the stylized photo should remains photorealistic.
While several methods exist for this task, they tend to generate spatially inconsistent stylizations with noticeable artifacts. In addition, these methods are computationally expensive, requiring several minutes to stylize a VGA photo. In this paper, we present a novel algorithm to address the limitations.
The proposed algorithm consists of a stylization step and a smoothing step. While the stylization step transfers the style of the reference photo to the content photo, the smoothing step encourages spatially consistent stylizations. Unlike existing algorithms that require iterative optimization, both steps in our algorithm have closed-form solutions.
Experimental results show that the stylized photos generated by our algorithm are twice more preferred by human subjects in average. Moreover, our method runs 60 times faster than the state-of-the-art approach.
|Dubai by Nilesh Trivedi|
|Hummingbird Tight by Dennis Bayer|
from -Vivid Purple- (in Full Colours Only)
Today, at Adobe MAX 2018, Adobe previewed Photoshop CC on iPad, a full-featured, desktop-class version of Photoshop for iOS.
The weather and has most definitely taken a turn toward fall here, and our shooting opportunities have followed suit. We brought the Canon RF 35mm F1.8 along to a harvest festival of sorts and a few of our usual haunts.
Pennsylvania Governor Tom Wolf has signed House Bill 1346 into effect, which imposes a fine upwards of $300 to drone operators who invade the privacy or harm the physical wellbeing of citizens.
Sigma is a company in flux, but CEO Kazuto Yamaki is undaunted by the upcoming prospect of developing lenses for eight lens mounts. The challenge will be keeping the company's identity along the way.
If you've been meaning to convert all of your old photos, video, and audio to digital formats, but simply lack the time or willpower to get through it all, a new service from Kodak will help you get the job done.
Almost all new cameras include impressive video features, but for the best results you'll often need an off-camera recorder. Chris and Jordan take a look at the brand new Ninja V from Atomos, and explain why it might just be one of the most useful tools you can add to your camera.
Collect allows you to transform 360-degree into a more easily digestible format by transforming it into directed traditional videos.
Sick of using your plain ol' keyboard to edit your photos in Lightroom and Photoshop? TourBox is hoping to expedite your post-production workflow using a clever combination of dials, buttons, and knobs.
Bag and accessory manufacturer Hex has launched two bags as part of its latest collection: the Clamshell Backpack and DSLR Sling.
Crank out instant photos with Holga Digital's new analog printer, currently being funded on Kickstarter.
We got some hands-on time with Leica's new S3 medium format camera, which boasts a new higher-res sensor as well as other improvements.
Luna Display started its life as a crowdfunding project on Kickstarter. Now, it's available to purchase directly online.
We sat down with the Google Pixel camera team to learn about key new camera features on the Pixel 3 and Pixel 3 XL, and an explanation of the sophisticated software advancements that power them.
A lawsuit filed on Tuesday claims the cameras in Apple's iPhone 7 Plus and newer dual-camera models infringe on a patent that was granted in 2003.
Nikon's Coolpix P1000 has moved the zoom needle from 'absurd' to 'ludicrous,' with an equivalent focal length of 24-3000mm. So far, it's a fun camera to shoot with – if a bit over the top.
Like the LG V40 ThinQ the A9 combines a super-wide-angle, regular wide-angle and tele camera, but adds a depth-sensor to the mix as well.
The FAA has issued a warning to drone pilots in anticipation of disaster response following Hurricane Michael, noting that fines for interfering with emergency operations can exceed $20,000.
According to a report from Fortune, Apple acquired Danish masking technology startup Spektral in December 2017 for "more than $30 million."
Insta360's latest model comes with a range of features that allow for the creation of unique action cam footage.
The Photogrip can be used as a camera grip, mini tripod or phone stand and comes with a detachable remote.
At a time when manufacturers are adding triple and even quad-cameras to their flagship smartphones, Google is sticking with one main camera. But given the sophistication of the company's computational efforts, we think it's the right approach for now.
DPReview is hiring! We're seeking three Software Development Engineers at a range of experience levels to join our Seattle-based team.
The University of Dayton Research Institute created a video detailing what damage is caused when a drone strikes the wing of an airplane.
Lenovo's upcoming high-end smartphone will be the first model to feature four cameras on the back.
The Pixel 3 and Pixel 3 XL offer a second front-facing camera and a host of improved computational features such as digital zoom based on super-resolution capture, better depth mapping and a fill-light effect for low light portraits.
Canon has ported a large chunk of its Digital Photo Professional (DPP) Raw processing software's feature set to iOS and launched the DPP Express app.
The Panasonic LX100 II offers a higher-resolution sensor over its predecessor, but it's the addition of a touchscreen that makes the Mark II so gosh-darn enjoyable to shoot with. We've got some fresh samples from Panasonic's new premium compact camera.
Sony has announced a new "Alpha Female" program, a creator-in-residence opportunity that will award six-month grants to five female filmmakers and photographers.
The new 490, 492 and 492LCD are targeted at amateur photographers and come with a 4kg/8.82lbs payload.
The Fujifilm X-T3 exhibits very good image quality overall, with excellent detail and color at low ISOs. Further analysis of the high ISO performance confirms it falls behind its predecessor, though.