54

Image style AI can convert paintings to photographs

Apps that convert your photos into paintings, with styles ranging from Monet to Lichtenstein, have been all the rage recently, with Prisma being one of the most popular offerings. A research team at UC Berkeley has now developed an artificial intelligence algorithm that can do the reverse – and much more. 

Using 'image style transfer' their AI can convert Monet's impressionist paintings into a much more realistic image that comes close to photo quality, giving you a good idea of the scene that the French painter was looking at when he put down his easel and started to paint. In addition, the system is capable of changing winter to summer in an image of a Yosemite scene, turn apples into oranges and horses into zebras. Of course it's also possible to transform regular photos into Monets, Van Goghs or Cezannes.

The research project uses so-called 'unpaired data' which, in the words of the project team means that 'we have knowledge of the set of Monet paintings and of the set of landscape photographs. We can reason about the stylistic differences between those two sets, and thereby imagine what a scene might look like if we were to translate it from one set into another.'

To achieve that the relationships between similar styles had to be coded in a way that can be understood by a computer and then the AI had to be trained using large number of photos from Flickr and other sources. In a final step the quality of the results was checked by both humans and machines before fine-tuning of the algorithms. 

On some occasions the results are still far from perfect but overall the AI is impressively good at transferring styles from one image to another. More information is available on the project's GitHub page.

Comments

All (54)
Most popular (7)
Editors' picks (0)
DPR staff (0)
Oldest first
dtibi

waiting for AI algorithm that converts crap phone photos to award winnig photos

3 weeks ago
dtibi

bathroom selfies to fashion photos

3 weeks ago
Barty L
Barty L

When everyone is producing award-winning photos it's the crap ones that will stand out and actually win the awards. Just bide your time.

3 weeks ago
DaveClark

Just buy a $10,000 Leica. Problem solved.

3 weeks ago
Sir Nick of High Point

Finally, someone figured out how to change a horse into a zebra. Turn out the lights everyone. We can all go home now :)

3 weeks ago*
PowerG9atBlackForest
PowerG9atBlackForest

Will there be a Google plug-in for recognizing people? So, I could indentify, say, Mona Lisa as Mrs. Mona Lisa from Pisa?

3 weeks ago
Gimli son of Gloin
Gimli son of Gloin

Great!
*throws camera on the floor*
Just great. Now even a god damn bot has more artistic skill than me.

3 weeks ago
EssexAsh

Using 'image style transfer' their AI can convert Monet's impressionist paintings into a much more realistic image that comes close to photo quality, giving you a good idea of the scene that the French painter was looking at when he put down his easel and started to paint.
--------

And thus completely and utterly missing the point of what Monet and impressionism was all about. Awesome for every artistic ignoramus out there. I'm sure they'll sell loads

3 weeks ago
jorg14
jorg14

Art is like religion, everyone thinks they have the right answer.

3 weeks ago
HeyItsJoel
HeyItsJoel

Religion is like art, everyone has their own interpretation of it.

3 weeks ago
AbrasiveReducer

I'll stick with art.

3 weeks ago
belle100
belle100

I suppose one can use this technology to refine a low res photo to a high res one.

3 weeks ago
AstroStan

That is already being done with this type of AI. Not only facial recognition but even astronomical analysis of small blurry galaxies. But one must be careful when interpreting such images as they are as much (or more) invention as discovery.

3 weeks ago
Ann Chaikin
Ann Chaikin

Wonder what it would do with Picasso or Dali. Could be interesting.

3 weeks ago
SETI

Also Edvard Munch...

3 weeks ago
Alec
Alec

And Malevich's black square.

3 weeks ago
Willgo

Malevich's black square would be a revelation. It would show that he was really painting a black cat in a coal cellar, but didn't have either ISO a zillion or a torch handy.

3 weeks ago
OlyPent
OlyPent

The best application for this might be to convert painted old movie backdrops into more photographic looking backgrounds. But, I'd expect the same reaction to doing that as colourizing black and white movies.

3 weeks ago
lescrane

what a waste of effort to prove a point. At least it's not government funded.

3 weeks ago
technotic

Are you against progress in general?

3 weeks ago
HeyItsJoel
HeyItsJoel

Forgive him. He wasn't breastfed.

3 weeks ago
Irkiosan

Very impressing!

3 weeks ago
mxx
mxx

Did you mean impressive or depressing?

3 weeks ago
Irkiosan

Hahaha, both I guess.

1 week ago
OliverGlass

I did a painting of a dot, and this transformed it into a photo of a dot. I can now apply vintage filter effects. Awesome! :)

3 weeks ago
Ruy Penalva
Ruy Penalva

Supposedly only when the painter has a photographic stile.

3 weeks ago
ashyu

Perhaps more bizarre is when Google used their image recognition ANNs "in reverse" to generate images by "asking" what a trained ANN saw in various images. The results were artificially-generated surreal scenes that don't exist anywhere in real life or in any prior artwork, but were based on what the ANN was originally trained to recognize.
Gallery here: https://goo.gl/photos/fFcivHZ2CDhqCkZdA
Full article here: https://research.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html

3 weeks ago
Ab Latchin

errr. this app should be called... we removed the brushstrokes from paintings :)

3 weeks ago
tom graham

yes indeed, no hint of brush strokes. Looks like bad out of focus photo. Hurts my eyes, can not look at them. And yes, went to the web site and that just made them look worse.

3 weeks ago*
Satyaa

"giving you a good idea of the scene that the French painter was looking at when he put down his easel" - really? The algorithms are built and biased by today's programmers. They 'learn' from today's published photographs online. I doubt that anyone would be able to interpolate what the painter saw back then.

On the other hand, this would have been useful to produce images of how different people saw alternate facts after certain big event on Jan 20th in the US this year!

3 weeks ago
Alec
Alec

What if it were used on more paintings? http://imgur.com/a/UhN8V

3 weeks ago
Alex Permit
Alex Permit

Agree with PropaPH & Triplet Perar. These transformations don't look at all like photographs. They look like really bad art.

3 weeks ago
PropaPH

Dr Frankenstein, I presume?

3 weeks ago
Triplet Perar
Triplet Perar

I have some bad news for those geeks that think they will be able "to see" scenes as the old painters saw them. If you think that is possible, then you know nothing about painting. Most of the time painters would *invent* and move objects around the scenes, exaggerating and diminishing proportions, painting in colours that ever were there, and adding things, people and elements that never were there too. In words of an art historian, art is about "raising above the severe limitations of the subject, and exploring the unknown possibility".

3 weeks ago
DanielFjall
DanielFjall

Nobody claimed they would either.

3 weeks ago
Satyaa

Triplet Perar: That sounds like lot of PP to me!

3 weeks ago
Neuropsychology
Neuropsychology

Triple, how dare you call me a geek; it's nerd sir! :P

3 weeks ago
Dan Tong

You need to go to the research web site to see that this approach can do a lot more than just attempt to go from artistic stylized image to photo realistic source. The potential uses are far wider and may be of serious interest to photographers and other visual artists.

https://junyanz.github.io/CycleGAN/

In fact it sounds great to me and I am looking forward to the future when it may be available as a commercial (and affordable) plug-in for Photoshop etc. or an independent graphics program.

3 weeks ago*
Clyde Thomas

Can you sue a robot?

"...AI had to be trained using large number of photos from Flickr and other sources..."

- "using"
means they used photos.

- "training"
means they used the photos for training purposes.

- "large number"
means a clever lawyer could file massive class action law suit for copyrights violation.

I say... Pictures, or it didn't happen.

3 weeks ago*
Sidath Senanayake

I highly doubt there is a lawsuit here. Have a read of the Flickr terms and conditions and you'll realise that you're allowing your photos to be used via the Flickr API for things such as this and much more.

The only way you could prevent this sort of usage is to make your photos private.

3 weeks ago
Clyde Thomas

Yeah I was kinda joking man.

3 weeks ago
schmoldovia
schmoldovia

I really would like to see a PICASSO or FEININGER converted to a photo. Then converting it back to a painting back to a photo ... at least funny!!!

Maybe we also could find out who really was MonaLisa if we have a "photo".

3 weeks ago
Mark Turney
Mark Turney

No it doesn't. By definition, the end result is an image manipulation and most definitely NOT a photograph .... which would have been captured by a photographer with a camera. With this implication, an artist could paint a dragon flying overthe forest, "convert" it using this garbage, then say "look what I photographed!"

It's great software for the UFO hunters I guess!

3 weeks ago
Henrik Herranen

I like to draw cats and handbags.
https://affinelayer.com/pixsrv/

My tests (most ideas mine, some blatantly lifted from others):
http://www.iki.fi/leopold/tmp/170223_Kissa/

3 weeks ago*
Sdaniella
Sdaniella

it's bad enough photoshophanatics are generating clickbait pics and videos ...

this process only adds power to creative-visual-distortion mills ...

3 weeks ago
JackM
JackM

If you click through to the project, you'll see the results are not convincing at all.

3 weeks ago
Eric Calabros
Eric Calabros

True about most of AI hype titles these days

3 weeks ago
dlb41

Where does it all end.

3 weeks ago
left eye

At the beginning.

3 weeks ago
Walter

Non ending..life is a circle... The joke of course are the app creators who try to fool people into thinking they are artists. What next?...old masters in wine glasses....? :)

3 weeks ago*
left eye

This news is 2 days too late.

3 weeks ago
Flowchart
Flowchart

my thoughs exactly

3 weeks ago
stevie wonder
stevie wonder

Can't wait to see what Mona Lisa really looked like😳🙈😜

3 weeks ago
HeyItsJoel
HeyItsJoel

That's a maaaaan, baby!

3 weeks ago