53

Image style AI can convert paintings to photographs

Apps that convert your photos into paintings, with styles ranging from Monet to Lichtenstein, have been all the rage recently, with Prisma being one of the most popular offerings. A research team at UC Berkeley has now developed an artificial intelligence algorithm that can do the reverse – and much more. 

Using 'image style transfer' their AI can convert Monet's impressionist paintings into a much more realistic image that comes close to photo quality, giving you a good idea of the scene that the French painter was looking at when he put down his easel and started to paint. In addition, the system is capable of changing winter to summer in an image of a Yosemite scene, turn apples into oranges and horses into zebras. Of course it's also possible to transform regular photos into Monets, Van Goghs or Cezannes.

The research project uses so-called 'unpaired data' which, in the words of the project team means that 'we have knowledge of the set of Monet paintings and of the set of landscape photographs. We can reason about the stylistic differences between those two sets, and thereby imagine what a scene might look like if we were to translate it from one set into another.'

To achieve that the relationships between similar styles had to be coded in a way that can be understood by a computer and then the AI had to be trained using large number of photos from Flickr and other sources. In a final step the quality of the results was checked by both humans and machines before fine-tuning of the algorithms. 

On some occasions the results are still far from perfect but overall the AI is impressively good at transferring styles from one image to another. More information is available on the project's GitHub page.

View Comments (53)

Comments

All (53)
Most popular (7)
Editors' picks (0)
DPR staff (0)
Oldest first
dtibi

waiting for AI algorithm that converts crap phone photos to award winnig photos

10 months ago
dtibi

bathroom selfies to fashion photos

10 months ago
Barty L
Barty L

When everyone is producing award-winning photos it's the crap ones that will stand out and actually win the awards. Just bide your time.

10 months ago
DaveClark

Just buy a $10,000 Leica. Problem solved.

10 months ago
Sir Nick of High Point

Finally, someone figured out how to change a horse into a zebra. Turn out the lights everyone. We can all go home now :)

10 months ago*
PowerG9atBlackForest
PowerG9atBlackForest

Will there be a Google plug-in for recognizing people? So, I could indentify, say, Mona Lisa as Mrs. Mona Lisa from Pisa?

10 months ago
Gimli son of Gloin
Gimli son of Gloin

Great!
*throws camera on the floor*
Just great. Now even a god damn bot has more artistic skill than me.

10 months ago
EssexAsh

Using 'image style transfer' their AI can convert Monet's impressionist paintings into a much more realistic image that comes close to photo quality, giving you a good idea of the scene that the French painter was looking at when he put down his easel and started to paint.
--------

And thus completely and utterly missing the point of what Monet and impressionism was all about. Awesome for every artistic ignoramus out there. I'm sure they'll sell loads

10 months ago
jorg14
jorg14

Art is like religion, everyone thinks they have the right answer.

10 months ago
HeyItsJoel
HeyItsJoel

Religion is like art, everyone has their own interpretation of it.

10 months ago
AbrasiveReducer

I'll stick with art.

10 months ago
belle100
belle100

I suppose one can use this technology to refine a low res photo to a high res one.

10 months ago
AstroStan

That is already being done with this type of AI. Not only facial recognition but even astronomical analysis of small blurry galaxies. But one must be careful when interpreting such images as they are as much (or more) invention as discovery.

10 months ago
Ann Chaikin
Ann Chaikin

Wonder what it would do with Picasso or Dali. Could be interesting.

10 months ago
SETI
SETI

Also Edvard Munch...

10 months ago
Alec
Alec

And Malevich's black square.

10 months ago
Willgo

Malevich's black square would be a revelation. It would show that he was really painting a black cat in a coal cellar, but didn't have either ISO a zillion or a torch handy.

10 months ago
(unknown member)

The best application for this might be to convert painted old movie backdrops into more photographic looking backgrounds. But, I'd expect the same reaction to doing that as colourizing black and white movies.

10 months ago
lescrane

what a waste of effort to prove a point. At least it's not government funded.

10 months ago
technotic

Are you against progress in general?

10 months ago
HeyItsJoel
HeyItsJoel

Forgive him. He wasn't breastfed.

10 months ago
Irkiosan

Very impressing!

10 months ago
mxx
mxx

Did you mean impressive or depressing?

10 months ago
Irkiosan

Hahaha, both I guess.

10 months ago
OliverGlass

I did a painting of a dot, and this transformed it into a photo of a dot. I can now apply vintage filter effects. Awesome! :)

10 months ago
Ruy Penalva
Ruy Penalva

Supposedly only when the painter has a photographic stile.

10 months ago
ashyu

Perhaps more bizarre is when Google used their image recognition ANNs "in reverse" to generate images by "asking" what a trained ANN saw in various images. The results were artificially-generated surreal scenes that don't exist anywhere in real life or in any prior artwork, but were based on what the ANN was originally trained to recognize.
Gallery here: https://goo.gl/photos/fFcivHZ2CDhqCkZdA
Full article here: https://research.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html

10 months ago
Ab Latchin

errr. this app should be called... we removed the brushstrokes from paintings :)

10 months ago
tom graham

yes indeed, no hint of brush strokes. Looks like bad out of focus photo. Hurts my eyes, can not look at them. And yes, went to the web site and that just made them look worse.

10 months ago*
Satyaa

"giving you a good idea of the scene that the French painter was looking at when he put down his easel" - really? The algorithms are built and biased by today's programmers. They 'learn' from today's published photographs online. I doubt that anyone would be able to interpolate what the painter saw back then.

On the other hand, this would have been useful to produce images of how different people saw alternate facts after certain big event on Jan 20th in the US this year!

10 months ago
Alec
Alec

What if it were used on more paintings? http://imgur.com/a/UhN8V

10 months ago
Alex Permit
Alex Permit

Agree with PropaPH & Triplet Perar. These transformations don't look at all like photographs. They look like really bad art.

10 months ago
PropaPH

Dr Frankenstein, I presume?

10 months ago
Triplet Perar
Triplet Perar

I have some bad news for those geeks that think they will be able "to see" scenes as the old painters saw them. If you think that is possible, then you know nothing about painting. Most of the time painters would *invent* and move objects around the scenes, exaggerating and diminishing proportions, painting in colours that ever were there, and adding things, people and elements that never were there too. In words of an art historian, art is about "raising above the severe limitations of the subject, and exploring the unknown possibility".

10 months ago
DanielFjall
DanielFjall

Nobody claimed they would either.

10 months ago
Satyaa

Triplet Perar: That sounds like lot of PP to me!

10 months ago
Neuropsychology
Neuropsychology

Triple, how dare you call me a geek; it's nerd sir! :P

10 months ago
Dan Tong

You need to go to the research web site to see that this approach can do a lot more than just attempt to go from artistic stylized image to photo realistic source. The potential uses are far wider and may be of serious interest to photographers and other visual artists.

https://junyanz.github.io/CycleGAN/

In fact it sounds great to me and I am looking forward to the future when it may be available as a commercial (and affordable) plug-in for Photoshop etc. or an independent graphics program.

10 months ago*
Clyde Thomas

Can you sue a robot?

"...AI had to be trained using large number of photos from Flickr and other sources..."

- "using"
means they used photos.

- "training"
means they used the photos for training purposes.

- "large number"
means a clever lawyer could file massive class action law suit for copyrights violation.

I say... Pictures, or it didn't happen.

10 months ago*
Sidath Senanayake

I highly doubt there is a lawsuit here. Have a read of the Flickr terms and conditions and you'll realise that you're allowing your photos to be used via the Flickr API for things such as this and much more.

The only way you could prevent this sort of usage is to make your photos private.

10 months ago
Clyde Thomas

Yeah I was kinda joking man.

10 months ago
schmoldovia
schmoldovia

I really would like to see a PICASSO or FEININGER converted to a photo. Then converting it back to a painting back to a photo ... at least funny!!!

Maybe we also could find out who really was MonaLisa if we have a "photo".

10 months ago
Mark Turney
Mark Turney

No it doesn't. By definition, the end result is an image manipulation and most definitely NOT a photograph .... which would have been captured by a photographer with a camera. With this implication, an artist could paint a dragon flying overthe forest, "convert" it using this garbage, then say "look what I photographed!"

It's great software for the UFO hunters I guess!

10 months ago
Henrik Herranen

I like to draw cats and handbags.
https://affinelayer.com/pixsrv/

My tests (most ideas mine, some blatantly lifted from others):
http://www.iki.fi/leopold/tmp/170223_Kissa/

10 months ago*
JackM
JackM

If you click through to the project, you'll see the results are not convincing at all.

10 months ago
Eric Calabros
Eric Calabros

True about most of AI hype titles these days

10 months ago
dlb41

Where does it all end.

10 months ago
left eye

At the beginning.

10 months ago
Walter

Non ending..life is a circle... The joke of course are the app creators who try to fool people into thinking they are artists. What next?...old masters in wine glasses....? :)

10 months ago*
left eye

This news is 2 days too late.

10 months ago
Flowchart
Flowchart

my thoughs exactly

10 months ago
stevie wonder
stevie wonder

Can't wait to see what Mona Lisa really looked likeπŸ˜³πŸ™ˆπŸ˜œ

10 months ago
HeyItsJoel
HeyItsJoel

That's a maaaaan, baby!

10 months ago

Latest in-depth reviews

Fujifilm X-H1 First Impressions Review

The Fujifilm X-H1 is a top-of-the-range 24MP mirrorless camera with in-body stabilization and the company's most advanced array of video capabilities. We've been shooting with one for a while now and have put together a gallery, a sample video and some preliminary analysis.

Panasonic Lumix DC-GX9 First Impressions Review and Sample Gallery

Panasonic's Lumix DC-GX9 is a rangefinder-style mirrorless camera that offers quite a few upgrades over its predecessor, with a lower price tag to boot. We've spent the weekend with the GX9 and have plenty of thoughts to share, along with an initial set of sample photos.

Panasonic Lumix DC-ZS200/TZ200: First Impressions Review

Panasonic's new premium compact boasts a 24-360mm equiv. F3.3-6.4 zoom lens, making it the longest reaching 1"-type pocket camera on the market. We spent a little time with it; read our first impressions.

Review: Google Pixel 2 is the best smartphone for stills photographers

The Pixel 2 pushes the envelope in terms of what can be accomplished with computational photography. Find out why it earns our highest ever score for a smartphone camera in our full review.

Panasonic Lumix DC-GH5S Review

The Panasonic GH5S is best understood as an even more video-centric variant of the GH5. We've tested it in a range of circumstances to see whether the video improvements are worth the loss of stabilization.

Latest buying guides

Buying Guide: Best cameras for landscapes

Landscape photography isn't as simple as just showing up in front of a beautiful view and taking a couple of pictures. Landscape shooters have a unique set of needs and requirements for their gear, and we've selected some of our favorites in this buying guide.

Buying Guide: Best cameras for parents

Quick. Unpredictable. Unwilling to sit still. Kids really are the ultimate test for a camera's autofocus system. We've compiled a short list of what we think are the best options for parents trying to keep up with young kids, and narrowed it down to one best all-rounder.

Buying Guide: Best cameras over $2000

If you're a serious enthusiast or working pro, the very best digital cameras on the market will cost you at least $2000. That's a lot of money, but generally speaking these cameras offer the highest resolution, the best build quality and the most advanced video specs out there, as well as fast burst rates and top-notch autofocus.

Buying Guide: Best cameras for video

Video features have become an important factor to many photographers when choosing a new camera. Read on to find out which cameras we think are best for the videophile.

Buying Guide: Best cameras for sports and action

Are you a speed freak? Hungry to photograph anything that goes zoom? Or perhaps you just want to get Sports Illustrated level shots of your child's soccer game. Keep reading to find out which cameras we think are best for sports and action shooting.