Previous news story    Next news story

DxO launches DxOMark Mobile - device IQ analysis as used on Connect

By dpreview staff on Oct 9, 2012 at 20:38 GMT

DxO Labs has announced 'DxOMark Mobile' - its assessment of mobile phone image quality that will support mobile device reviews on connect.dpreview.com. In its first group of tests, DxO has found that the best contemporary mobile devices (in this case the Nokia 808 PureView) will out-perform a 5-year-old high-end compact camera. Though, given the 808's sensor is newer and significantly larger, the question has to be why it doesn't out-perform the compact by a bigger margin.

And when it comes to video, the Samsung Galaxy SIII will trump last year's Canon PowerShot S100. DxOMark Mobile will analyze 14 aspects of mobile imaging and produce a final score that will be comparable to existing DxOMark figures. We'll be reporting DxOMark Mobile figures alongside our overall scores in connect reviews.


Press Release

Mobile phone image quality surpasses that of 2007 compact camera models, according to DxOMark Mobile

DxOMark now measures the camera image quality for cellphones, smartphones, and digital tablets

October 9, 2012 - DxOMark, the website of reference for camera and lens image quality measurements, announces the launch of new measurements dedicated to camera image quality data for cellphones, smartphones, and digital tablets.

Drawing on DxO Labs' expertise in measuring the image quality of conventional cameras and lenses, DxO Mark Mobile aims to help consumers analyze and compare the still and video performance of mobile devices by freely providing them with its industrial-quality scientific measurements.

Mobile image quality is now superior to that of 5-year-old compact cameras

Mobile phone cameras have become a key feature for a large number of consumers: in 2011, more than one quarter of all photos were taken with a mobile phone. This trend has accelerated in 2012 - which may end up being the year when mobile photography surpasses traditional amateur compact camera photography.

Mobile phone image quality has also made considerable progress - for example, the image quality of the Nokia 808 PureView, in first place in the DxOMark Mobile rankings for still photos, is superior to that of a 5-year-old compact camera. And as for video, the best mobile tested in this category, the Samsung Galaxy SIII, outperforms the Canon Powershot 100.

"Of the dozen mobile devices that we have already tested, some have already attained a significant level of image quality," said Frédéric Guichard, Chief Scientific Officer for DxO Labs. "However, the increase in sensor resolution and the miniaturization of pixels has introduced new problems well-known to photographers, such as digital noise. Our detailed test results will allow consumers to analyze all of the intrinsic qualities of mobile cameras."

Innovative, standards-compliant DxOMark Mobile measurement technology

DxOMark experts have developed innovative testing protocols based on both objective and perceptual measurements. DxOMark Mobile measures many attributes of mobile camera image quality such as color, exposure, texture, noise, and autofocus, in both still photo and video modes. In all, DxOMark Mobile analyzes 14 attributes, assigning each one a score between 1 and 100. An overall DxOMark score lets users rank and make comparisons between camera-equipped mobile devices, as well as between mobile cameras and compact cameras.

"In addition to measurements made in the laboratory under perfectly-controlled environmental and lighting conditions, we also conduct perceptual analyses of shots taken in the field," commented Guichard. "We reproduce the everyday conditions that people commonly encounter when using these devices so as to reliably analyze and report on their image quality."

DxOMark Mobile's fully-reproducible testing protocols conform to international standards established by the Camera & Imaging Products Association (CIPA), the lnternational Imaging Industry Association (I3A), and the Camera Phone Image Quality (CPIQ) working group.

"With its continuously-enriched database, DxOMark Mobile aims to become the reference source for information about mobile camera image quality, and to develop partnerships with many players in the mobile photography market," said Jérôme Ménière, CEO of DxO Labs.

The first reviews based on DxOMark Mobile measurements will be published in the next few weeks.

Comments

Total comments: 32
keeponkeepingon
By keeponkeepingon (Oct 11, 2012)

In the iphone 5 tests you state that all images are normalized for testing to
8mp.

If resolution is one of your 14 measured parameters this would seem very unfair to large megapixel phones with a decent lens.

Is this a DXO lab mobile testing standard or a dpreview choice? If the later then what the heck?

0 upvotes
Peksu
By Peksu (Oct 11, 2012)

There are long articles on the DxO site showing the formulas for normalization. Normalization means that a camera with the same per-pixel performance but higher resolution will have higher score. Without normalization that would not be the case, the resolution would only be an extra parameter amongst others.

0 upvotes
jxh
By jxh (Oct 11, 2012)

This is an interesting and useful area.

My choice of a phone is not controlled by it's camera quality any more than my choice of a car is based on its sound system. However, a reasonable phone cam (even if it lacks long zoom ranges, large sensor or other niceties) satisfies the need to have a camera available whenever you happen to need one.

But a phone that needs to fit in the pocket has limitations that a decent point and shoot does not. Zoom size, external mechanical controls, even body size and shape.

0 upvotes
falconeyes
By falconeyes (Oct 10, 2012)

I am curious to learn how DxO manages to overcome the hurdle imposed by different JPG engines. This made their RAW ratings pretty objective and made them refrain from doing tests based on JPG. It will be interesting to see how they deal with that problem.

1 upvote
Peksu
By Peksu (Oct 11, 2012)

If the phones only output jpg, why should something else (some hypothetical raw) be compared? That would have zero significance to anyone in real life.

1 upvote
gwlaw99
By gwlaw99 (Oct 13, 2012)

Because a camera with a lot of noise reduction will get a better score even though it makes the images soft

0 upvotes
Henry M. Hertz
By Henry M. Hertz (Oct 10, 2012)

and please notice how dpreview (once more) used a nikon camera as top product on a chart..... and of course canon cameras to show the worse.

because there is no nikon compact camera that scored worse then the S100 or G5....

this has become typicall for dpreview lately.

0 upvotes
Simon Joinson
By Simon Joinson (Oct 11, 2012)

if you look carefully you'll see this is (i) a press release (ii) a press release from DxO - not dpreview (iii) an graphic produced by another company - DxO for a press release. Nothing to do with us.

2 upvotes
john Clinch
By john Clinch (Oct 15, 2012)

Firstly I don't see the pictures in the graphic as important

But the release is about a new website called Digital Photography Review Connect. So you can hardly say it has nothing to do with you.

0 upvotes
sir_bazz
By sir_bazz (Oct 10, 2012)

Good job DXO

Always good to have more solid performance data.

2 upvotes
Gothmoth
By Gothmoth (Oct 10, 2012)

another meaningless DXO nonsense....

yes nonsense because DXO tests mean nothing in real life.

0 upvotes
Shamael
By Shamael (Oct 12, 2012)

I agree with you, I look at the shot a camera produces in jpeg and in RAW if there is any, and I make up my mind. I care a f... what DXO says, tests or thinks. The pictures of the latest cameras shown on this site have proven that DXO is just a religion and in all I saw until now, there is nothing in any D600, A99, RX1, or NEX-6, that could make me lift from my best part and move to a camera shop. In these days, some body specs like OMD or A99 could make me change, but certainly not the images they deliver, since all of them are stuck on a certain IQ level at the moment. The one that makes the best shot, DP2M, is so limited in it's use that the price you pay for is an insult, the most pricey one you can buy is same crappy built as other's and is a bigger insult. So, on the end, check the shots posted online and make your choice. Between 450 and 1300$, you get a top notch picture in these days, all that goes beyond is for pro's or wanabe geeks only.

0 upvotes
bradleyg5
By bradleyg5 (Oct 10, 2012)

I don't have a smart phone guys, I have an iPad and a really slow phone. I need to get a smart phone eventually, I'm totally gonna make my buying decision based off what they say.

I mean, the Nokia 404 has Symbian and I just can't deal with that. But an Android phone that does the best video. Maybe that i can deal with...

I'll lose all my apps though, I kinda want an iPhone 5 but the smaller camera sensor makes me hate them, you can't convince me it's even better than the iPhone 4s. Finally DxO mark is going to have proof! Take these smart phone guys to task over their fanciful claims. Long live progress! Better photography for all!

1 upvote
AnHund
By AnHund (Oct 10, 2012)

Pure speculation that the Samsung should outperform the Canon S100 in my opinion.

2 upvotes
Boerseuntjie
By Boerseuntjie (Oct 10, 2012)

Actually Canon sensors score bad all the time ;) but I will agree with you no way the Samsung outperforms the S100

0 upvotes
le_alain
By le_alain (Oct 10, 2012)

"the image quality of the Nokia 808 PureView, in first place in the DxOMark Mobile rankings for still photos, is superior to that of a 5-year-old compact camera"

such conclusion (comparing a canon G5) confising image quality is not sensor
Make a A3 print of both ...
I don't mind any review based on DXO tests

0 upvotes
DafO
By DafO (Oct 10, 2012)

Great - as a photography geek the photography ability is a major consideration when looking for a new phone, but usual tech sites only give it a fleeting look.

1 upvote
Dan Tong
By Dan Tong (Oct 10, 2012)

This is very interesting and practically useful. Good Standards make it far easier to make intelligent comparisons and choices.

1 upvote
Rage Joe
By Rage Joe (Oct 10, 2012)

Shouldn't we start, this time again, whining about how smart phones are never going to have real cameras. And then laugh.

1 upvote
Dennis Linden
By Dennis Linden (Oct 10, 2012)

nervously

3 upvotes
Benarm
By Benarm (Oct 9, 2012)

Without actual samples these DxO scores are pretty useless. Hopefully they will provide something like DPR's studio comparison photos.

4 upvotes
Simon Joinson
By Simon Joinson (Oct 10, 2012)

like this? http://connect.dpreview.com/post/1343713337/apple-iphone-5-review?page=7

2 upvotes
abolit
By abolit (Oct 10, 2012)

agree 100% I don't care how DxO measures the sensor I look at my shots and I can see if it's good or bad. Never trusted DxO much.

Comment edited 19 seconds after posting
1 upvote
AnHund
By AnHund (Oct 10, 2012)

The 808 is clearly much, much better than iPhone 5 if looking at one of the user provided links: http://sdrv.ms/QOzrHB

But then again the 808 is still very, very far from the IQ of for instance a Canon S100.

So even though phone cameras have become a lot better they are not that good yet.

0 upvotes
Henry M. Hertz
By Henry M. Hertz (Oct 10, 2012)

if DXO says phones are better then this is a fact!

0 upvotes
keeponkeepingon
By keeponkeepingon (Oct 11, 2012)

> like this? http://connect.dpreview.com/post/1343713337/apple-iphone-5-review?page=7

Simon: are those the full resolution samples from each phone or are the phones with bigger sensors hacked down to 8mp by dpreview? If the later I would suggest your studio scene is mostly useless, especially when comparing resolution.

Why not give us the actual SOC files from the phones? If they are not SOC pictures
it's amazing that all of the files are within .1mb of each other.....

0 upvotes
d3xmeister
By d3xmeister (Oct 9, 2012)

I think this is great, I was wondering when will it happen. The first results are actually not that surprising.

0 upvotes
raincoat
By raincoat (Oct 9, 2012)

they have time now to test mobile phone cameras, but no time to fix bugs in their paid software DOP or add more modules to DOP?

2 upvotes
Mssimo
By Mssimo (Oct 9, 2012)

I think they provide the tools and Dpreview provides the staff.

0 upvotes
Detail Man
By Detail Man (Oct 9, 2012)

DxOReview ... featuring controlled-exposure, pixel-resolution-normalized, and non-Adobe-massaged Studio Comparison Tool RAW image-samples for the purpose of readers actually being able to make reliable comparisons between RAW-recording cameras !

Comment edited 2 minutes after posting
2 upvotes
keeponkeepingon
By keeponkeepingon (Oct 11, 2012)

This word RAW it does not mean what you think it means....

(how can a RAW file be a pixel resolution normalized jpeg. does not compute...)

0 upvotes
mpgxsvcd
By mpgxsvcd (Oct 9, 2012)

Say it ain't so. DxO Mark and Dpreview teaming up? Is the world coming to an end?

All kidding aside. You guys do a great job.

Comment edited 24 seconds after posting
5 upvotes
Total comments: 32