142

This neural network turns smartphone snaps into 'DSLR-quality photos'

Researchers at ETH Zürich have developed an AI-powered system that can turn your measly smartphone snapshots into images that look like they were recorded with a full-blown DSLR... or so they claim.

The project is called 'DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks' and part of the abstract on the project home page reads as follows:

Despite a rapid rise in the quality of built-in smartphone cameras, their physical limitations—small sensor size, compact lenses and the lack of specific hardware—impede them to achieve the quality results of DSLR cameras. In this work we present an end-to-end deep learning approach that bridges this gap by translating ordinary photos into DSLR-quality images.

Of course, the term "DSLR-quality images" could mean many things, but it looks like the software is currently focusing on sharpness, color and tonality. This is in contrast with what smartphone manufacturers tend to refer to as "DSLR-quality images" and what they try to replicate with 'Portrait' mode photos: depth-of-field, or rather the lack of it.

To create the software the team started by by training a deep learning system by feeding it photos taken of the same scene using a smartphone camera and a DSLR. This approach worked well but could only improve the quality for the specific smartphone in question. A more sophisticated second version only needs to see two sets of images from different cameras to understand how to apply the image quality from one to the other; in other words: you can feed any photo into the system and apply the image quality of a target camera to it.

The results still need some fine-tuning on occasions—for example, some of the sample shots on display show color casts or a loss of detail after going through the process. However, test images tend to be better exposed and more vibrant. The most obvious improvements can be achieved with smartphone cameras on older or lower-tier devices though.

The scientist hope to eventually use their neural network for modifying the shooting conditions rather than the image quality of the camera. For example, you could turn a photo that was taken on a rainy day into one captured in bright sunshine... for many photographers this might be just a step to far.

If you want to try the current version yourself, you can do so on phancer.com.

View Comments (142)

Comments

All (142)
Most popular (15)
Editors' picks (0)
DPR staff (0)
Oldest first
MPS Photographic

They have taken bland smart-phone photos and turned them into brighter bland smart-phone photos.

3 months ago
Tristan Jones

So they took poo and turned it into mushy poo....
A toddler can do that...

3 months ago*
ardp

It did a sort of HDR work, added noise and actually reduces sharpness.
Even on their demo images, it just burns out the sky in an attempt to brighten the image. Google's "im feeling lucky" button on Picasa does better.

More work to be done :)

3 months ago
Dean Mindock

I tried it on a couple of jpeg shots. It does a reasonable job but nothing I can't do better in Photoshop. It is fairly quick and I think it could help photo journalists get somewhat optimized shots back to home office quickly when time matters. I got to think though that as time passes the AI algorithms will become a lot smarter as that is an inherent feature of that field. IOW, it will only get a lot better. I can see where Adobe might be looking into this, just to stay competitive.

3 months ago
DarnGoodPhotos

Sounds like they are trying to create a smarter "Enhance" button.

3 months ago
onlooker

Take underexposed photo, up the brightness and saturation. ETH Zürich must be a research powerhouse.

4 months ago
Jane79
Jane79

It actually is. It's ranked as the 5th best university in the world for the subjects of engineering and technology and among the top 10 universities in the world overall. This project is not representative and has probably been picked up by DPR only because of ETH Zurich's significance in the world of engineering.

3 months ago
onlooker

"has probably been picked up by DPR only because of ETH Zurich's significance in the world of engineering." -- Yes, I am sure DPR regularly follows universities according to their research ranking. You were kidding, right?

3 months ago
UllerellU

I will tell you clearly, today the quality of the cameras of the mobile phones is ... bad, the dynamic range, the performance in low light, depth of bits, they are still compact consumer sensors with a few tiny lens A lot of software, crappy simulations, smoke screens, it's over.

Although the imgen quality was exceptional, it was like the Hasselblad X1D.

I would not want it, photography is not just about results, a big part of photography is the shooting experience, enjoy the experience, discover things around you that if you did not have the camera in your hand you would not discover, have all the controls by hand, physical controls, a good viewer, feel that you are the one who handles the exposure, who compensates for the highlights, qualifies or intensifies them according to their needs, who chooses the right white balance ....

I do not want to shoot with a stupid touch screen, where is the enjoyment in that.

4 months ago
Ultan

Here's a better one:
https://arxiv.org/abs/1612.07919
"EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis" by Mehdi S. M. Sajjadi, Bernhard Scholkopf and Michael Hirsch of the Max Planck Institute for Intelligent Systems, 30 July 2017

They multiply the resolution by 4 (linearly, it appears)and get photorealistic results from having trained their neural net to reconstruct perceptually similar textures to the ones found in its large and diverse training set of photos. This includes high-contrast, sharp features as well as subtle textures. The program size is about 3MB and it is fast, 10-60ms depending on picture complexity. The results are stunning quality, and it should be able to do even better with upcoming application to video.

4 months ago*
Blaklynx01

Soon every damn photo of a scene will look the same because AI will think, "This photo looks like the Eiffel Tower. I'll apply the 'training' photos I have and make it look great".

Now person 2 has a similar photo and their same AI will say, "This photo looks like the Eiffel Tower. I'll apply the 'training' photos I have and make it look great".

Now person 3 has a similar photo and their same AI will say, "This photo looks like the Eiffel Tower. I'll apply the 'training' photos I have and make it look great".

How many millions will have the same AI? Every photo will look the same / similar.

That's great. I can't wait to be bored to death.

3 months ago
fred_rdt

Actually, the results presented in their paper are "stunning quality". Is the software available for download, so we can check with our own pictures?

3 months ago
Ultan

"Is the software available for download...?"
Fred:
http://webdav.tuebingen.mpg.de/pixel/enhancenet/
(which is a good link to see the results without downloading a PDF as well)
see link: " Pre-trained model (zip)"

3 months ago
jdu_sg

And now there's no more photography, it's all photo-realism.

4 months ago
Mac McCreery
Mac McCreery

Great to see progress- I just like handling a DSLR. I find it far more satisfying than using a phone.

4 months ago
KrampusClaus

and guess a MUCH MUCH MUCH better photo using NO AI or scaring Elon Musk with use of AI !!!!

I have seen many times ppl using a tablet to do selfies !! Now that is FUNNY !

i personally am scared more of human UNintelligence - as in DJ TRUMP the orange ass-face clown who knows nothing !!

4 months ago
wabre

Can't believe so many negative comments on here. How do you think your so fantastic DSLR or mirrorless became that great if it wasn't for people putting an amazing amount of time and ideas to get where we are. You think your DSLR is great, then give those who made the transit from film to digital some credit. Who knows what these guys of ETH might achieve..!

4 months ago
Michael Long

And many don't seem to realize just how much computational image processing is being done with their "high-end" DSLR or mirrorless camera. Or do they think their images just jump off that Bayer color mosaic directly onto their flash cards?

Seriously, with the rise of multi-sensor, multi-lens AI-driven computational camera systems, I suspect most of today's mid-level cameras are headed for the dustbin within the next decade.

4 months ago
The Squire
The Squire

ENHANCE!

4 months ago
Albert Valentino

Another nail in the coffin 😐

4 months ago
Jane79
Jane79

Learning what does NOT work is an achievement in itself. :-)

4 months ago
Blaklynx01

Want 'DSLR' quality? Then use a bloody DSLR. It's that simple.

Want MF quality? Then use MF.

Want smart phone quality? Then use a smart phone.

Want smart phone quality that'll pretend to be DSLR quality? Don't take photos.

4 months ago*
wabre

It's with people like you that progress, evolution, ideas etc wouldn't have a chance. I hope not too many people see it this way.

4 months ago
Blaklynx01

wabre: haha, "people like you". I'm not anti-progress, anti-idea as you so eloquently state.

If people are happy to begin with less quality, then by all means they can go for it. Crap in is crap out. Always has been and always will be.

If all things are equal in the software and AI world, then the better hardware systems will always trump. Let's not pretend that physics don't exist. My first post implies; use the proper hardware for best results to begin with; I said nothing more and nothing less. Anything after that is a bonus (yep, includes software and evolutionary/revolutionary ideas).

If hardware doesn't matter, then let's all go back to 1Mp compact cameras, save a ton of money and weight, save the world's raw materials, save the environmental impacts from constant consumerism (must always buy the latest and greatest) and call it a day.... and let 'AI' software take care of the rest.

Let us all live in a world of 'fakeness'...

4 months ago
JordanAT
JordanAT

I'm okay with how good my camera phone is. Now what I need is an AI that can upgrade the person taking the picture.

4 months ago
M Lammerse
M Lammerse

I would believe them right away if they would use the comparison with mirrorless... ;-)

4 months ago
GinoSVK
GinoSVK

This AI should visit the ophthalmology clinic as soon as possible.

4 months ago
PredatorsPrey

Has anyone of you tried it out and got a result?

I've uploaded a photo taken with my zte smartphone and just get the moving cubes animation during the last 2 hours. Ok, the photo is technically garbage and doesn't have a good compensation but that's why I've choosen it - to see what it can do.

4 months ago
Conrad567

What kind of DSLR are they talking about here! A Canon 3D? A Nikon 1D? Or a Nikon D850?

4 months ago
TMHKR

Well, as long as there are no "light leak emulation" and similar hipster "artistic" cr4p...

4 months ago
Duckie

LOL!

3 months ago
Artem Holstov

Right, so now the correct white balance and exposure are called 'dslr quality'.

4 months ago
harry

I am very happy with what my new iPhone 8 Plus's cameras and software can do now, especially the highly touted HDR.

4 months ago
wildbild
wildbild

Photoshop version 1 yielded better results with simple sliders. Bizarre claims.

4 months ago
Duckie

Good to keep digging up new topics, be it agreeable or not.

Everyday people visit your web site in the hope of seeing something new. This insatiable demand for news is very hard to satisfy.

4 months ago
AmateurSnaps

April 1st really comes around quicker each year.

4 months ago
Wye Photography
Wye Photography

This is what happens when scientists/programmers (ale drinking pizza eating beard stroking Linux using geeks whose idea of a meaningful relationship is watching an episode of TBBT with a pizza) do some really cool things for photographers (gear orientated sharpness-resolution obsessive compulsives).

Scientist/programmer: We did this really cool thing for you.
Photographer: Yeah, but it looks 5h1t.

4 months ago
otto k

Show some respect, we don't all have beards!

4 months ago
The Squire
The Squire

This is awesome! Im going to upload some of my DSLR photos to get the AI to create some medium-format Hasselblad quality images!

4 months ago
otto k

Considering the sensor size difference you're going to get large format quality! 8x10 here I come!

4 months ago
otto k

And then I re-uploaded the already enhanced image and it became even more enhancier! I'm currently at 4x6 mile sensor quality and I'm just getting started! Only thing is all the mirror slap shot from such a huge mirror makes the photo so blurry you cannot see anything in it... Should have gone with emulating MILC at the start...

4 months ago
TMHKR

@otto k

Just don't go to burst mode with that mirror, we've had enough earthquakes by now.

4 months ago
dopravopat
dopravopat

Checked the site and the pictures. Some looked bad both "before" and "after". And tragically, the "before" picture looked in some cases better than the "after" picture.

4 months ago
bobogdan78

They shouldn't have labeled them and let us guess which is 'before' and which is 'after'.

4 months ago
moontide

I tried a photo but preferred the original, the photo was brightened too much for my taste. Probably works better on poor quality image.

4 months ago
Edmond Leung

I think the most effective way to get a better looking photo from the smartphone is:
1. Take the photo at it's maximum resolution; and turn on HDR;
2. Goto Photoshop, downsize the photo to approximately 1/4 of the original size; then adjust the curves and others.
But the lens and its hardware limitations are the deadlock for improvement.

4 months ago
RegisS

Ok, I tried the phancer upload/download. I uploaded a reasonably well exposed Android phone shot of ocean waves pounding a rocky shore. The 'enhanced' photo has saturation and brightness cranked way up. The churning wave detail is completely blown out white. The saturation boost colorized the black volcanic rock, and added some green tint to the blue sky. Foreground foliage color is barfogenic. Clearly NOT performing as described.

4 months ago
badi

Well... this can be seen from almost all the samples presented. You can do all that with just some curves and adjustment (and to my taste, almost always, the originals look nicer).

4 months ago
Couscousdelight

My Bullsh**-O-Meter is beeping.

4 months ago
PaulSnowcat
PaulSnowcat

Modern science is all about creating algorythmes that can do something in a mediocre to bad grade and call it "professional results". I call this stupidness and useless efforts. If you WANT to be a pfotographer - taking pictures, composing them, overcoming technical limitationa is not a burden. If it IS a burden for you then snapshots are good enough for you.

4 months ago
Mark Turney
Mark Turney

Amen brother!

4 months ago
Boissez

The thing is, the potential for automation/AI/machine learning to improve evolves on an exponential basis, while our meat-based monkey brains hardly evolve at all. It is only a matter of time before it eventually supercedes us at most tasks.

Hell, software like grip.tools has already put many pros out of a job - and that's just the top of iceberg.

4 months ago
adengappasami

The more people supply foto into the site the better the engine will become. Currently its not increasing DR but more like increasing levels and saturation at pixel level. the next iteration can control clipping highlights while fixing DR.

Lots of potential and almost feels like what snapseed can already do.

4 months ago
KoolKool

digital is limitless...you never know! :)

4 months ago
TMHKR

Year 2025: Photography pixel art! Exposure that lasts a couple of weeks! :)

EDIT: months!

4 months ago*
SandySandy

I saw the photos on their site and noticed some vefy strange things:
1. All the photos have merely been brightened. The dynamic Range hasn't been improved.
2. Most skies have highlight clipping after usage of this processing.
3. There is a curious colour change in many photos where a certain range of tonality (mid range mostly), has been erroneously deleted.
4. I suspect that None of the Programmers have used a DSLR in their lives. Also they would not be able to judge Dynamic Range in photos.
5. These guys also haven't heard of or used the Google Pixel. That's why they have no idea about the various kinds if photo-stacking. Maybe they should check out XDA-Developers.com first.
6. Good intentions, but very poor understandig of what needs to be achieved.
7. AI is a word used by every pedigree of idiot nowadays. They should just Ban It. Like using the 'N' word. Because people have no idea what it means.

4 months ago
adengappasami

i hope u have the pedigree to comment on AI. else its as gassy as it can get

4 months ago
RegisS

Pedigree? Look at the site. Upload a photo. SandySandy has it right.

4 months ago
Wye Photography
Wye Photography

That's what happens when programmers do "things" for photographers.

4 months ago
SandySandy

"i hope u have the pedigree to comment on AI. else its as gassy as it can get."

I know a mutt when i see one. You're a mutt. So much for pedigree.

3 months ago
micsmics

In a smartphone, the camera is a super-compact camera (disappears from the market).
The mirrorless cameras, instead of becoming similar to the Panasonic GM1, are aimed at the size of the nikon 850 )
And programmers do not optimize the code, but are engaged in imitation (bad taste)

4 months ago
FLruckas
FLruckas

Just have an app that uses the GPS to let you chose which of the many great pictures of where you are that were all already taken that you like and then automatically download the ones you choose to your phone, and upload them to your social media as if you took them.
Why bother even composing anymore?
:=)

4 months ago*
adengappasami

now that is google now :)

4 months ago
arqomx
arqomx

it's like making an AI for calculating bullet tradjectory based on elevation, gravity pull, wind speed and direction, humidity etc so a sniper can't miss his/her shot (s).. in the end it's just a tool to help a person make a better, calculated decision.. experiences and skills still matters..

4 months ago
sh10453
sh10453

Just thinking out loud.
Why don't they use extrapolation?
That way they won't need two sets of images.
I believe extrapolation is used by some industrial grade printers, where they can print a gorgeous 36" X 24" (3 feet X 2 feet) print from just a 2 megapixel image.

4 months ago
GreatOceanSoftware

Great. Just what I need. A camera bag full of phones.

4 months ago
Tonio Loewald

Soon we can have ML powered cameras that remove noise, paint in extra resolution, and add detail to shadows. Years ago there was a fractal compression algorithm that could scale compressed images beyond their original size with inferred detail. The compression algorithm never gained popularity but several commercial products use the technique to upscale photos to higher resolutions.

4 months ago
aegarcia

A phone can make calls, surf the internet and incidentally takes pictures that satisfies a lot of people that is, a jack of all trades. A phone picture will get published every now and then, not all the time. To take really good quality photos one needs a good camera and the know how, the logic is simple.

4 months ago
Tom_A

Whether we like this effort or not, it is an example of “algorithmic imaging” becoming ever more important.

4 months ago
aegarcia

I fully agree with NJOcean. Comparing photos taken by my mobile phone with those from my Lumix Tz110 to see if the latter can be left home, I enlarge them in a 4k 27' monitor. The result convinced me I cannot do without my Point and Shot. It would be a miracle if a software can improve mobile photos without an improvement of the lens as well.

4 months ago
snowrunner

Sorry, not impressed. I only see adjustments for exposure, contrast and perhaps colour tint. Every image editing software you can get for free on the internet can do the same, it's usually called 'auto correct'. You can't beat the laws of optics and physics, though. If you want 'DSLR-quality' images, you'll have to buy yourself a DSLR. No disrespect to the ETH Zürich, sometimes even very smart people do something nonsensical.

4 months ago*
aegarcia

I'm a newcomer to DP review and I think I,ve click your Website every single day since you begun publishing. I,ve done the same with Photoblog but I'll say it now, you guys are better and faster at giving photographic information and so decided to register.

4 months ago
aegarcia

I've got a desktop capable of showing 4k photos with a 4k monitor, of course. I'm a keen amateur. I begun taking pictures in 1949 with a Bownie Kodak. I've have had all kinds of cameras since then. Right now I'm happy with a Pentax K3, a Lumix Tz110, and a Tz60. Waiting for the next generation of cameras in 2018 to replace my equipment. I'm also into video with Powerdirector 15, Power2go ll, and Power DVD 17.
In short I know what I'm doing and to compare Samsung or IPhone photos against about 14mp pictures from a 24mp DSLR, which I have done so many times, is just not on. The difference is gigantic.

4 months ago
cdembrey

More throwing red-meat at the Lions 8-D Can you say beyond click-bait ???

4 months ago
Total: 84, showing: 1 – 50
« First‹ Previous12Next ›Last »

Latest in-depth reviews

Fujifilm X-H1 First Impressions Review

The Fujifilm X-H1 is a top-of-the-range 24MP mirrorless camera with in-body stabilization and the company's most advanced array of video capabilities. We've been shooting with one for a while now and have put together a gallery, a sample video and some preliminary analysis.

Panasonic Lumix DC-GX9 First Impressions Review and Sample Gallery

Panasonic's Lumix DC-GX9 is a rangefinder-style mirrorless camera that offers quite a few upgrades over its predecessor, with a lower price tag to boot. We've spent the weekend with the GX9 and have plenty of thoughts to share, along with an initial set of sample photos.

Panasonic Lumix DC-ZS200/TZ200: First Impressions Review

Panasonic's new premium compact boasts a 24-360mm equiv. F3.3-6.4 zoom lens, making it the longest reaching 1"-type pocket camera on the market. We spent a little time with it; read our first impressions.

Review: Google Pixel 2 is the best smartphone for stills photographers

The Pixel 2 pushes the envelope in terms of what can be accomplished with computational photography. Find out why it earns our highest ever score for a smartphone camera in our full review.

Panasonic Lumix DC-GH5S Review

The Panasonic GH5S is best understood as an even more video-centric variant of the GH5. We've tested it in a range of circumstances to see whether the video improvements are worth the loss of stabilization.

Latest buying guides

Buying Guide: Best cameras for landscapes

Landscape photography isn't as simple as just showing up in front of a beautiful view and taking a couple of pictures. Landscape shooters have a unique set of needs and requirements for their gear, and we've selected some of our favorites in this buying guide.

Buying Guide: Best cameras for parents

Quick. Unpredictable. Unwilling to sit still. Kids really are the ultimate test for a camera's autofocus system. We've compiled a short list of what we think are the best options for parents trying to keep up with young kids, and narrowed it down to one best all-rounder.

Buying Guide: Best cameras over $2000

If you're a serious enthusiast or working pro, the very best digital cameras on the market will cost you at least $2000. That's a lot of money, but generally speaking these cameras offer the highest resolution, the best build quality and the most advanced video specs out there, as well as fast burst rates and top-notch autofocus.

Buying Guide: Best cameras for video

Video features have become an important factor to many photographers when choosing a new camera. Read on to find out which cameras we think are best for the videophile.

Buying Guide: Best cameras for sports and action

Are you a speed freak? Hungry to photograph anything that goes zoom? Or perhaps you just want to get Sports Illustrated level shots of your child's soccer game. Keep reading to find out which cameras we think are best for sports and action shooting.