147

This neural network turns smartphone snaps into 'DSLR-quality photos'

Researchers at ETH Zürich have developed an AI-powered system that can turn your measly smartphone snapshots into images that look like they were recorded with a full-blown DSLR... or so they claim.

The project is called 'DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks' and part of the abstract on the project home page reads as follows:

Despite a rapid rise in the quality of built-in smartphone cameras, their physical limitations—small sensor size, compact lenses and the lack of specific hardware—impede them to achieve the quality results of DSLR cameras. In this work we present an end-to-end deep learning approach that bridges this gap by translating ordinary photos into DSLR-quality images.

Of course, the term "DSLR-quality images" could mean many things, but it looks like the software is currently focusing on sharpness, color and tonality. This is in contrast with what smartphone manufacturers tend to refer to as "DSLR-quality images" and what they try to replicate with 'Portrait' mode photos: depth-of-field, or rather the lack of it.

To create the software the team started by by training a deep learning system by feeding it photos taken of the same scene using a smartphone camera and a DSLR. This approach worked well but could only improve the quality for the specific smartphone in question. A more sophisticated second version only needs to see two sets of images from different cameras to understand how to apply the image quality from one to the other; in other words: you can feed any photo into the system and apply the image quality of a target camera to it.

The results still need some fine-tuning on occasions—for example, some of the sample shots on display show color casts or a loss of detail after going through the process. However, test images tend to be better exposed and more vibrant. The most obvious improvements can be achieved with smartphone cameras on older or lower-tier devices though.

The scientist hope to eventually use their neural network for modifying the shooting conditions rather than the image quality of the camera. For example, you could turn a photo that was taken on a rainy day into one captured in bright sunshine... for many photographers this might be just a step to far.

If you want to try the current version yourself, you can do so on phancer.com.

Comments

All (147)
Most popular (15)
Editors' picks (0)
DPR staff (0)
Oldest first
MPS Photographic

They have taken bland smart-phone photos and turned them into brighter bland smart-phone photos.

3 weeks ago
Tristan Jones

So they took poo and turned it into mushy poo....
A toddler can do that...

3 weeks ago*
ardp

It did a sort of HDR work, added noise and actually reduces sharpness.
Even on their demo images, it just burns out the sky in an attempt to brighten the image. Google's "im feeling lucky" button on Picasa does better.

More work to be done :)

3 weeks ago
Dean Mindock

I tried it on a couple of jpeg shots. It does a reasonable job but nothing I can't do better in Photoshop. It is fairly quick and I think it could help photo journalists get somewhat optimized shots back to home office quickly when time matters. I got to think though that as time passes the AI algorithms will become a lot smarter as that is an inherent feature of that field. IOW, it will only get a lot better. I can see where Adobe might be looking into this, just to stay competitive.

3 weeks ago
DarnGoodPhotos

Sounds like they are trying to create a smarter "Enhance" button.

3 weeks ago
onlooker

Take underexposed photo, up the brightness and saturation. ETH Zürich must be a research powerhouse.

3 weeks ago
Jane79
Jane79

It actually is. It's ranked as the 5th best university in the world for the subjects of engineering and technology and among the top 10 universities in the world overall. This project is not representative and has probably been picked up by DPR only because of ETH Zurich's significance in the world of engineering.

3 weeks ago
onlooker

"has probably been picked up by DPR only because of ETH Zurich's significance in the world of engineering." -- Yes, I am sure DPR regularly follows universities according to their research ranking. You were kidding, right?

3 weeks ago
UllerellU

I will tell you clearly, today the quality of the cameras of the mobile phones is ... bad, the dynamic range, the performance in low light, depth of bits, they are still compact consumer sensors with a few tiny lens A lot of software, crappy simulations, smoke screens, it's over.

Although the imgen quality was exceptional, it was like the Hasselblad X1D.

I would not want it, photography is not just about results, a big part of photography is the shooting experience, enjoy the experience, discover things around you that if you did not have the camera in your hand you would not discover, have all the controls by hand, physical controls, a good viewer, feel that you are the one who handles the exposure, who compensates for the highlights, qualifies or intensifies them according to their needs, who chooses the right white balance ....

I do not want to shoot with a stupid touch screen, where is the enjoyment in that.

3 weeks ago
Ultan

Here's a better one:
https://arxiv.org/abs/1612.07919
"EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis" by Mehdi S. M. Sajjadi, Bernhard Scholkopf and Michael Hirsch of the Max Planck Institute for Intelligent Systems, 30 July 2017

They multiply the resolution by 4 (linearly, it appears)and get photorealistic results from having trained their neural net to reconstruct perceptually similar textures to the ones found in its large and diverse training set of photos. This includes high-contrast, sharp features as well as subtle textures. The program size is about 3MB and it is fast, 10-60ms depending on picture complexity. The results are stunning quality, and it should be able to do even better with upcoming application to video.

3 weeks ago*
Blaklynx01

Soon every damn photo of a scene will look the same because AI will think, "This photo looks like the Eiffel Tower. I'll apply the 'training' photos I have and make it look great".

Now person 2 has a similar photo and their same AI will say, "This photo looks like the Eiffel Tower. I'll apply the 'training' photos I have and make it look great".

Now person 3 has a similar photo and their same AI will say, "This photo looks like the Eiffel Tower. I'll apply the 'training' photos I have and make it look great".

How many millions will have the same AI? Every photo will look the same / similar.

That's great. I can't wait to be bored to death.

3 weeks ago
fred_rdt

Actually, the results presented in their paper are "stunning quality". Is the software available for download, so we can check with our own pictures?

3 weeks ago
Ultan

"Is the software available for download...?"
Fred:
http://webdav.tuebingen.mpg.de/pixel/enhancenet/
(which is a good link to see the results without downloading a PDF as well)
see link: " Pre-trained model (zip)"

3 weeks ago
jdu_sg

And now there's no more photography, it's all photo-realism.

3 weeks ago
Mac McCreery
Mac McCreery

Great to see progress- I just like handling a DSLR. I find it far more satisfying than using a phone.

3 weeks ago
KrampusClaus

and guess a MUCH MUCH MUCH better photo using NO AI or scaring Elon Musk with use of AI !!!!

I have seen many times ppl using a tablet to do selfies !! Now that is FUNNY !

i personally am scared more of human UNintelligence - as in DJ TRUMP the orange ass-face clown who knows nothing !!

3 weeks ago
wabre

Can't believe so many negative comments on here. How do you think your so fantastic DSLR or mirrorless became that great if it wasn't for people putting an amazing amount of time and ideas to get where we are. You think your DSLR is great, then give those who made the transit from film to digital some credit. Who knows what these guys of ETH might achieve..!

3 weeks ago
Michael Long

And many don't seem to realize just how much computational image processing is being done with their "high-end" DSLR or mirrorless camera. Or do they think their images just jump off that Bayer color mosaic directly onto their flash cards?

Seriously, with the rise of multi-sensor, multi-lens AI-driven computational camera systems, I suspect most of today's mid-level cameras are headed for the dustbin within the next decade.

3 weeks ago
The Squire
The Squire

ENHANCE!

3 weeks ago
Albert Valentino

Another nail in the coffin 😐

3 weeks ago
Jane79
Jane79

Learning what does NOT work is an achievement in itself. :-)

3 weeks ago
Blaklynx01

Want 'DSLR' quality? Then use a bloody DSLR. It's that simple.

Want MF quality? Then use MF.

Want smart phone quality? Then use a smart phone.

Want smart phone quality that'll pretend to be DSLR quality? Don't take photos.

3 weeks ago*
wabre

It's with people like you that progress, evolution, ideas etc wouldn't have a chance. I hope not too many people see it this way.

3 weeks ago
Blaklynx01

wabre: haha, "people like you". I'm not anti-progress, anti-idea as you so eloquently state.

If people are happy to begin with less quality, then by all means they can go for it. Crap in is crap out. Always has been and always will be.

If all things are equal in the software and AI world, then the better hardware systems will always trump. Let's not pretend that physics don't exist. My first post implies; use the proper hardware for best results to begin with; I said nothing more and nothing less. Anything after that is a bonus (yep, includes software and evolutionary/revolutionary ideas).

If hardware doesn't matter, then let's all go back to 1Mp compact cameras, save a ton of money and weight, save the world's raw materials, save the environmental impacts from constant consumerism (must always buy the latest and greatest) and call it a day.... and let 'AI' software take care of the rest.

Let us all live in a world of 'fakeness'...

3 weeks ago
JordanAT
JordanAT

I'm okay with how good my camera phone is. Now what I need is an AI that can upgrade the person taking the picture.

3 weeks ago
M Lammerse
M Lammerse

I would believe them right away if they would use the comparison with mirrorless... ;-)

3 weeks ago
GinoSVK
GinoSVK

This AI should visit the ophthalmology clinic as soon as possible.

3 weeks ago
PredatorsPrey

Has anyone of you tried it out and got a result?

I've uploaded a photo taken with my zte smartphone and just get the moving cubes animation during the last 2 hours. Ok, the photo is technically garbage and doesn't have a good compensation but that's why I've choosen it - to see what it can do.

3 weeks ago
Conrad567

What kind of DSLR are they talking about here! A Canon 3D? A Nikon 1D? Or a Nikon D850?

3 weeks ago
TMHKR

Well, as long as there are no "light leak emulation" and similar hipster "artistic" cr4p...

3 weeks ago
Duckie

LOL!

3 weeks ago
Artem Holstov

Right, so now the correct white balance and exposure are called 'dslr quality'.

3 weeks ago
harry

I am very happy with what my new iPhone 8 Plus's cameras and software can do now, especially the highly touted HDR.

3 weeks ago
wildbild
wildbild

Photoshop version 1 yielded better results with simple sliders. Bizarre claims.

3 weeks ago
Duckie

Good to keep digging up new topics, be it agreeable or not.

Everyday people visit your web site in the hope of seeing something new. This insatiable demand for news is very hard to satisfy.

3 weeks ago
AmateurSnaps

April 1st really comes around quicker each year.

3 weeks ago
Wye Photography
Wye Photography

This is what happens when scientists/programmers (ale drinking pizza eating beard stroking Linux using geeks whose idea of a meaningful relationship is watching an episode of TBBT with a pizza) do some really cool things for photographers (gear orientated sharpness-resolution obsessive compulsives).

Scientist/programmer: We did this really cool thing for you.
Photographer: Yeah, but it looks 5h1t.

3 weeks ago
otto k

Show some respect, we don't all have beards!

3 weeks ago
The Squire
The Squire

This is awesome! Im going to upload some of my DSLR photos to get the AI to create some medium-format Hasselblad quality images!

3 weeks ago
otto k

Considering the sensor size difference you're going to get large format quality! 8x10 here I come!

3 weeks ago
otto k

And then I re-uploaded the already enhanced image and it became even more enhancier! I'm currently at 4x6 mile sensor quality and I'm just getting started! Only thing is all the mirror slap shot from such a huge mirror makes the photo so blurry you cannot see anything in it... Should have gone with emulating MILC at the start...

3 weeks ago
TMHKR

@otto k

Just don't go to burst mode with that mirror, we've had enough earthquakes by now.

3 weeks ago
dopravopat
dopravopat

Checked the site and the pictures. Some looked bad both "before" and "after". And tragically, the "before" picture looked in some cases better than the "after" picture.

3 weeks ago
bobogdan78

They shouldn't have labeled them and let us guess which is 'before' and which is 'after'.

3 weeks ago
moontide

I tried a photo but preferred the original, the photo was brightened too much for my taste. Probably works better on poor quality image.

3 weeks ago
Edmond Leung

I think the most effective way to get a better looking photo from the smartphone is:
1. Take the photo at it's maximum resolution; and turn on HDR;
2. Goto Photoshop, downsize the photo to approximately 1/4 of the original size; then adjust the curves and others.
But the lens and its hardware limitations are the deadlock for improvement.

3 weeks ago
RegisS

Ok, I tried the phancer upload/download. I uploaded a reasonably well exposed Android phone shot of ocean waves pounding a rocky shore. The 'enhanced' photo has saturation and brightness cranked way up. The churning wave detail is completely blown out white. The saturation boost colorized the black volcanic rock, and added some green tint to the blue sky. Foreground foliage color is barfogenic. Clearly NOT performing as described.

3 weeks ago
badi

Well... this can be seen from almost all the samples presented. You can do all that with just some curves and adjustment (and to my taste, almost always, the originals look nicer).

3 weeks ago
Couscousdelight

My Bullsh**-O-Meter is beeping.

3 weeks ago
PaulSnowcat
PaulSnowcat

Modern science is all about creating algorythmes that can do something in a mediocre to bad grade and call it "professional results". I call this stupidness and useless efforts. If you WANT to be a pfotographer - taking pictures, composing them, overcoming technical limitationa is not a burden. If it IS a burden for you then snapshots are good enough for you.

3 weeks ago
Mark Turney
Mark Turney

Amen brother!

3 weeks ago
Boissez

The thing is, the potential for automation/AI/machine learning to improve evolves on an exponential basis, while our meat-based monkey brains hardly evolve at all. It is only a matter of time before it eventually supercedes us at most tasks.

Hell, software like grip.tools has already put many pros out of a job - and that's just the top of iceberg.

3 weeks ago
adengappasami

The more people supply foto into the site the better the engine will become. Currently its not increasing DR but more like increasing levels and saturation at pixel level. the next iteration can control clipping highlights while fixing DR.

Lots of potential and almost feels like what snapseed can already do.

3 weeks ago
KoolKool

digital is limitless...you never know! :)

3 weeks ago
TMHKR

Year 2025: Photography pixel art! Exposure that lasts a couple of weeks! :)

EDIT: months!

3 weeks ago*
SandySandy

I saw the photos on their site and noticed some vefy strange things:
1. All the photos have merely been brightened. The dynamic Range hasn't been improved.
2. Most skies have highlight clipping after usage of this processing.
3. There is a curious colour change in many photos where a certain range of tonality (mid range mostly), has been erroneously deleted.
4. I suspect that None of the Programmers have used a DSLR in their lives. Also they would not be able to judge Dynamic Range in photos.
5. These guys also haven't heard of or used the Google Pixel. That's why they have no idea about the various kinds if photo-stacking. Maybe they should check out XDA-Developers.com first.
6. Good intentions, but very poor understandig of what needs to be achieved.
7. AI is a word used by every pedigree of idiot nowadays. They should just Ban It. Like using the 'N' word. Because people have no idea what it means.

3 weeks ago
adengappasami

i hope u have the pedigree to comment on AI. else its as gassy as it can get

3 weeks ago
RegisS

Pedigree? Look at the site. Upload a photo. SandySandy has it right.

3 weeks ago
Wye Photography
Wye Photography

That's what happens when programmers do "things" for photographers.

3 weeks ago
SandySandy

"i hope u have the pedigree to comment on AI. else its as gassy as it can get."

I know a mutt when i see one. You're a mutt. So much for pedigree.

2 weeks ago
micsmics

In a smartphone, the camera is a super-compact camera (disappears from the market).
The mirrorless cameras, instead of becoming similar to the Panasonic GM1, are aimed at the size of the nikon 850 )
And programmers do not optimize the code, but are engaged in imitation (bad taste)

3 weeks ago
FLruckas
FLruckas

Just have an app that uses the GPS to let you chose which of the many great pictures of where you are that were all already taken that you like and then automatically download the ones you choose to your phone, and upload them to your social media as if you took them.
Why bother even composing anymore?
:=)

3 weeks ago*
adengappasami

now that is google now :)

3 weeks ago
arqomx
arqomx

it's like making an AI for calculating bullet tradjectory based on elevation, gravity pull, wind speed and direction, humidity etc so a sniper can't miss his/her shot (s).. in the end it's just a tool to help a person make a better, calculated decision.. experiences and skills still matters..

3 weeks ago
sh10453
sh10453

Just thinking out loud.
Why don't they use extrapolation?
That way they won't need two sets of images.
I believe extrapolation is used by some industrial grade printers, where they can print a gorgeous 36" X 24" (3 feet X 2 feet) print from just a 2 megapixel image.

3 weeks ago
GreatOceanSoftware

Great. Just what I need. A camera bag full of phones.

3 weeks ago
Tonio Loewald

Soon we can have ML powered cameras that remove noise, paint in extra resolution, and add detail to shadows. Years ago there was a fractal compression algorithm that could scale compressed images beyond their original size with inferred detail. The compression algorithm never gained popularity but several commercial products use the technique to upscale photos to higher resolutions.

3 weeks ago
Michael Stubbs

There seem to be a bunch of scientists out there utterly determined to remove all human skill from life whether that be driving or taking amazing photographs.

3 weeks ago
tangbunna

this one is color algorithm that will calculate the dynamic range of a photo. no human can do. unless manual extra paint job

3 weeks ago
Michael Stubbs

A good photographer would have used his eye in combination with a light meter to do that before pressing the shutter, would he not? That is why I carry graduated filters around with me.

3 weeks ago
LFPCPH
LFPCPH

Yes - skills are disappearing. Nobody can make a good flint axe any more.

3 weeks ago
TMHKR

You forgot handwriting skills... I will mock millennials with my super ultra artistic detailed cursive handwriting!

3 weeks ago
aegarcia

A phone can make calls, surf the internet and incidentally takes pictures that satisfies a lot of people that is, a jack of all trades. A phone picture will get published every now and then, not all the time. To take really good quality photos one needs a good camera and the know how, the logic is simple.

3 weeks ago
Tom_A

Whether we like this effort or not, it is an example of “algorithmic imaging” becoming ever more important.

3 weeks ago
aegarcia

I fully agree with NJOcean. Comparing photos taken by my mobile phone with those from my Lumix Tz110 to see if the latter can be left home, I enlarge them in a 4k 27' monitor. The result convinced me I cannot do without my Point and Shot. It would be a miracle if a software can improve mobile photos without an improvement of the lens as well.

3 weeks ago
snowrunner

Sorry, not impressed. I only see adjustments for exposure, contrast and perhaps colour tint. Every image editing software you can get for free on the internet can do the same, it's usually called 'auto correct'. You can't beat the laws of optics and physics, though. If you want 'DSLR-quality' images, you'll have to buy yourself a DSLR. No disrespect to the ETH Zürich, sometimes even very smart people do something nonsensical.

3 weeks ago*
aegarcia

I'm a newcomer to DP review and I think I,ve click your Website every single day since you begun publishing. I,ve done the same with Photoblog but I'll say it now, you guys are better and faster at giving photographic information and so decided to register.

3 weeks ago
aegarcia

I've got a desktop capable of showing 4k photos with a 4k monitor, of course. I'm a keen amateur. I begun taking pictures in 1949 with a Bownie Kodak. I've have had all kinds of cameras since then. Right now I'm happy with a Pentax K3, a Lumix Tz110, and a Tz60. Waiting for the next generation of cameras in 2018 to replace my equipment. I'm also into video with Powerdirector 15, Power2go ll, and Power DVD 17.
In short I know what I'm doing and to compare Samsung or IPhone photos against about 14mp pictures from a 24mp DSLR, which I have done so many times, is just not on. The difference is gigantic.

3 weeks ago
Total: 85, showing: 1 – 50
« First‹ Previous12Next ›Last »