Previous news story    Next news story

DxOMark introduces 'Perceptual MPix' score for lens sharpness

By dpreview staff on Dec 17, 2012 at 16:00 GMT

DxOMark has introduced a new metric of lens sharpness that it's calling the 'Perceptual MegaPixel'. This is designed to give a simple, easily-understandable indication of how a lens impacts on the resolution of the camera with which it's paired. The measurement is based on standard MTF (Modulation Transfer Function) testing, but also takes into account the sensitivity of the human visual system to detail. The data can be explored over at the DxOMark website, www.dxomark.com.

Our lens reviews are prepared in partnership with DxOMark, using test data shot in its labs. Do you think the 'Perceptual Megapixel' would be a useful number to include in our reviews? Let us know in the comments below.

Press release:

DxOMark introduces the Perceptual Megapixel as a new unit for measuring the perceived sharpness of lenses

A simple way to quantify the impact of lens sharpness on camera resolution

December 17, 2012 - DxOMark, the website of reference for measuring the image quality of digital cameras and lenses, announces the launch of the Perceptual Megapixel, its new measure of lens sharpness. Expressed in units already well-known to the general public, the Perceptual Megapixel provides photographers with a means of instantly understanding the performance of a lens by quantifying its impact on the perceived resolution of the camera with which it is coupled.

A measurement correlated to human vision

Designed by DxOMark experts, Perceptual Megapixel quantifies the sharpness "perceived" by the photographer while taking into account all the characteristics and defects of the lens and the camera.

"This new unit of measure replaces the MTF scores that we have been publishing up until now. Perceptual Mpix weights MTF measurements with the sensitivity to detail of the human visual system, thus providing a score for the sharpness perceived by our eyes. We think this new measurement will be more understandable and practical for photographers who use our website," explained Frédéric Guichard, Chief Scientific Officer for DxO Labs.

An average of 45% of all megapixels lost

The Perceptual Megapixel measurement allows for making numerous comparisons among cameras and for the first time quantifies the impact of lens sharpness on camera resolution. Thus the perceived resolution for a camera can be significantly different from the native resolution depending on the lenses with which it is coupled.

For example, when coupled with a Canon EOS 5D Mark II camera with a 21.1 Mpix sensor, a Sigma 35mm F1.4 DG HSM lens has a Perceptual Mpix score of 17.2, whereas a lens such as the Carl Zeiss Distagon T* 35mm f/1.4 ZE has a Perceptual Mpix score of 15.2.

The example above is based on data from DxOMark's database of test reults for more than 2,700 camera and lens combinations. These tests reveal that, on average, about 45% of the resolution is lost due to lens defects.

"The Perceptual Megapixel measurement allows photographers to step away from manufacturers' announcements about resolution and to answer a key question when the time comes to change equipment: would it be better to buy a new camera or a new lens?" observed Nicolas Touchard, VP Marketing, Image Quality Evaluation, for DxO Labs.

This new metric introduced by DxOMark aligns with the standardization work currently being undertaken by the IEEE, the International Imaging Industry Association (I3A), and the ISO - and in which DxO Labs actively participates. Perceptual Mpix is also based on the scientific findings of CNES (the French space agency) pertaining to the optimization of digital acquisition, notably in the field of satellite imagery.

More than 2,700 camera/lens combinations available

Freely available to all, DxOMark has been designed to serve the entire photographic community, from serious amateur photographers to photo experts and journalism specialists. It has a threefold mission: to provide objective measurements of digital cameras and lenses; to analyze camera and lens performance; and to provide a means for making meaningful comparisons.

DxOMark now has a database of more than 2,700 available camera/lens combinations. Dozens of DxO Labs engineers and technicians in six DxO laboratories perform these measurements, taking a photo every 10 seconds. Testing protocols are entirely reproducible and conform to the international standards established by the Camera & Imaging Products Association (CIPA), the International Imaging Industry Association (I3A), and the Camera Phone Image Quality (CPIQ) group.

Comments

Total comments: 182
12
Sdaniella
By Sdaniella (Dec 27, 2012)

none of their tests account for optimal jpeg engine setup (default vs custom) where default settings are clearly NOT optimal for lit conditions being shot in, and such non-optimal 'defaults' VARIES widely from model to model (within a mfr) as well as VARY widely from other mfrs (mfr-to-mfr).

their approach is as flawed as the assumption everyone just shoots in 'default' modes; which is preposterous.

Comment edited 1 minute after posting
0 upvotes
mauro paillex
By mauro paillex (Dec 25, 2012)

Brain masturbations! A good picture is a good picture! Even with a 2 mp camera!!

0 upvotes
castleofargh
By castleofargh (Dec 25, 2012)

i don't understand all the rage around this. dxo never said the biggest number was the best lens. you guys do that and then argue about how wrong it is!
dxo tries to give us an idea about how sharp a lens can feel.
i for one don't think the sharpest lens is the best. just like i don't believe a tilt-shift lens is the best lens.
how fast it focuses, how smooth is the bokeh, does it have image stabilization ability? this will make me decide what lens is ultimately best for me.
but i don't see the need to spit on this sharpness information, all perceptual it can be.
information is always a good thing.

0 upvotes
Ronj2
By Ronj2 (Dec 24, 2012)

All this does not explain why experts could not tell the difference from prints made from a 14mp P&S camera and a Mediun format digital one. Let see some actual 100% crops that show the difference between a 6mp lens and 16mp lens through a 24mp sensor.

http://www.luminous-landscape.com/reviews/kidding.shtml

0 upvotes
Ramjager
By Ramjager (Dec 20, 2012)

Great article however a point or two.
DXO seem o only concern themselves with base ISO.
Fine if you ONLY shoot landscape..however in the world where many of these cameras get used higher ISO performance is a better judgement point.
People who shoot sport,any action or journos make as large a group if not larger Han landscape only guys.
Yet DXO mark applies to base ISO giving extremely misleading results that apply to a minute amount of he community.
Not to mention no ratings or AF performance,build,battery use etc.
The DXO lens ratings are pathetic at best.
IE the Nikon and Canon 400/2.8's rate poorly really poorly compared to a consumer grade Sigma zoom.
Sorry but there lens test are invalid,inaccurate and imply wrong.
A one stringed test for sensors and in accurate tests on lenses.
Like many others I pay no attention to DXO for good reasons.

0 upvotes
chicco47
By chicco47 (Dec 19, 2012)

I have a lot of doubts whether the DxO scores based on scientific lab tests have any practical value; in same cases they are in contrast with practical results.

0 upvotes
schneesturm
By schneesturm (Dec 20, 2012)

That's pretty much like saying, it doesn't have any practival value if you shoot with high-end FF gear or a point-and-shoot. And in fact, if you are just taking pictures for the internet it's likely even true.

0 upvotes
rfsIII
By rfsIII (Dec 19, 2012)

The more I ponder it, the more I think that lens sharpness is such an infinitesimally small part of photography. Yes, there are bad lenses, but they're much less common than in the olden days.
A camera is much more than a light-tight box with a hole at one end that you shoot one time to evaluate forever. It's a system that is used over time and that's what's missing in the DXO system--time. What really needs a one-number evaluation score is customer service, repair prices, how well the equipment stays calibrated and aligned over time and in relation to the knocks and bumps of daily photography, availability and quality of accessories like microscope adapters and remote releases, owner satisfaction, worldwide dealer network so that when I'm traveling I can easily replace lost parts of the kit, how good the images look on the wide array of printers, RIPs, monitors, and projection systems... There are so many factors that are more important than just "sharpness."

0 upvotes
peevee1
By peevee1 (Dec 19, 2012)

Reading their article, I did not find the mention at what aperture do they measure the sharpness? But let's assume it is the best a given lens can do at any aperture (ignoring DoF and light requirements). But what about zoom - at what focal length this is measured?

Otherwise the scores themselves are useful just to confirm the stupidity of sensor megapixel wars of last many years, with most lenses giving only 4-6 P-Mpix of optical resolution at best.

0 upvotes
sotirius
By sotirius (Dec 20, 2012)

How is it then that MF lenses are so much better?

0 upvotes
schneesturm
By schneesturm (Dec 20, 2012)

They are not only measuring most common apertures and focal lengths (like 2.8, 4.0, 5.6 ..., 35 mm, 50 mm ...) but also different camera bodys. By analysing their data you might find out, that most SLR-lenses indeed provide more than 12 P-Mpix (depending on aperture, focal length and camera sensor of course).

0 upvotes
Abhijith Kannankavil
By Abhijith Kannankavil (Dec 19, 2012)

One more number to ignore. Nothing more.

1 upvote
mblackmagico
By mblackmagico (Dec 21, 2012)

I see you don't do any printing.

4 upvotes
JonB1975
By JonB1975 (Dec 19, 2012)

Having read this article and the comments below I have come to this conclusion:

I will continue to completely ignore DxO labs.

3 upvotes
Sergi Gabriel
By Sergi Gabriel (Dec 19, 2012)

Very strange for me information from DXOMark that Canon lens 70-200 IS 2.8L is better then 70-200 IS 2.8L v.2. It is reason why information from DXO is only information, not more.

Comment edited 2 minutes after posting
0 upvotes
AlephNull
By AlephNull (Dec 19, 2012)

Canon does not make a 20-70 lens (maybe you mean 24-70?), but the 24-70 doesn't have IS. Or do you mean 70-200?

Your comment just doesn't make sense.

1 upvote
peevee1
By peevee1 (Dec 20, 2012)

If the P-mpix score is taken just at one specific focal length, what about all the others?

0 upvotes
mblackmagico
By mblackmagico (Dec 21, 2012)

Read the report, the v.1 is higher res on full frames, but lower res on APS-C sensors. This is not so strange being that it has an extra stop of stabilization. If you used both youd know this to be true. And DP Reviews tests show the same results. Your just mad that canon ripped you off.

0 upvotes
subwave
By subwave (Dec 19, 2012)

It would help a bit, if DxO would release some sample images with different PMpix. Maybe the same image with 4, 8, 12 and 16 PMpix.

Comment edited 25 seconds after posting
0 upvotes
ozturert
By ozturert (Dec 19, 2012)

Excellent. Now you know which lenses will mostly get the highest "Perceptual MPix" scores (hint: Nikon). Just a guess :)

0 upvotes
Cliff5200
By Cliff5200 (Dec 19, 2012)

One more thing to DXO,,,, Even though this is a great metric, you guys need to do a better job explaining it. Look at the comments below, people are confused. Even the name "Perpetual Megapixel" sounds like you guys are hawking snake oil. Why not "relative megapixel" or "resolved megapixel"? Someone on your staff has been watching too much Star Trek.

After ranting on the people that commented below I tried to have an open mind and re-read your press release. You are not presenting the greatest benefit (the relative ranking across cameras with different sensor sizes) of this metric first, you are presenting techno-babble first. I guess I can understand all of the confusion below.,,,,,,,,,

Try this:

1. First explain the real world benefit of the metric to the semi-technical average reader. Explain the intended usage (for the average user).

2. Second go into technical details for people that are interested.

Comment edited 2 times, last edit 4 minutes after posting
2 upvotes
peevee1
By peevee1 (Dec 20, 2012)

"Perceptual", not "perpetual".

0 upvotes
Cliff5200
By Cliff5200 (Dec 19, 2012)

And BTW, before any of you technical wanabees get a chance to say anything, do you know what a relative test or test methodology is?? It is a method of ranking or grouping subjects (in this case camera and lens combinations) by order of performance relative to each other. It is for comparison purposes. To be valid the test must be consistently performed for all subjects in the set and the test parameters (resolution in this case) must be of interest to the reader. I trust that DXO has enough experience to run the test consistently. I don't need to agree with every aspect of the test methodology for the test to be worthwhile on a relative basis!

This is not a test that you can compare the absolute results of with another testing organization (like SLR Gear) unless they where to exactly reproduce the test setup and methodologies used by DXO. (and they don't need to do that).

DXO has tested 2700 camera/lens combinations so this is a good enough cross section to be very worthwhile.

1 upvote
Cliff5200
By Cliff5200 (Dec 19, 2012)

To DXO - Thanks, this is great! It allows a simple and quick relative comparison between camera and lens combinations in different formats (FX, DX, 4-3, etc). This is something that has been previously time consuming to do. Again - THANKS FOR THIS DXO!

To ALL YOU KNUCKLE HEADS AND HATERS - It is good to know the total resolution different camera and lens combinations will produce. This is a simple metric that does that. OF COURSE YOU NEED TO TAKE OTHER TESTS INTO ACCOUNT, but this gives you a good relative ranking to get started!

This metric is a great general reality check. Knuckle head camera snobs and camera manufacturers WAKE UP! I bet you haters spent crazy money on a superwide (or whatever lens you overpaid for) and just hate that the Samyang 14-2.8 (or other inexpensive lens like a 50-1.8 or 35-2) is so good! Well don't hate DXO, it was your purchasing decision!

After looking at this I think I will replace my DX camera with an FX camera and a set of inexpensive primes.

2 upvotes
jdeigert
By jdeigert (Dec 19, 2012)

"After looking at this I think I will replace my DX camera with an FX camera and a set of inexpensive primes."
I agree, and so would Canon and Nikon. This is good timing with the newly released 6D and D600

0 upvotes
SebaF80
By SebaF80 (Dec 19, 2012)

Well, to have a simple (and single) value scoring a system is a great think. You can easily say a 3mt tree is higher than a 2mt one, but not all the thinks can be so semplified.

We all don't know what algoritm DxO has used to score the lensed paired with a specific camera but let me say that the number of megapixel itself is not a scientific metric.

If they wanted to semplify the MTF to only one figure they could use another approach: to calculate the area of the region delimited by the curve as we do in maths with the "definite integral".
We can surely state the higher the MTF curve the better the system is.
The only perplexity at reading the MTF is if is better an MTF shape or a different but the area (only one number) considers all the single contributes.

What it seems to me is the perceived megapixel is a "camera centric" way to understand how much its sensor is loosing paired to a lens, but cannot state 2 system with same Perceptual MPix behaves the same. A crucial point.

0 upvotes
SebaF80
By SebaF80 (Dec 19, 2012)

From a different point of view, using "Perceptual MPix" you can understand how much your camera is getting worse but what to do to improve your system?
Is it better to change the lens, the camera or both?

And what about ISO noise, sensor sensitivity and all the parameters you have to consider to "improve" your results, like c.a., distorsion, vignetting, etc, ... ?

This is not clear to me ... I prefer to examine the MTF ....

0 upvotes
jonikon
By jonikon (Dec 18, 2012)

No. DxOmark's lens test results are already bewildering, and not very helpful in choosing a lens for me at least. DxOmark' rates most lenses closer to poor than any test site I have ever seen and gives the impression that one would have to pay at least $2000 to get a decent lens for their Nikon D3200!

I much prefer SLRgear.com's interactive blur index graphics for an indication of what one can expect from a lens in the real world. DxOmark is for those few who like to t contemplate the String Theory of the universe, but not for those interested in doing photography in the real world.

0 upvotes
SebaF80
By SebaF80 (Dec 18, 2012)

Oh my God, what an headache!
Well, to summarize in just one figure how the optical system will reproduce colors, tones and how will proof to haze, low light and all the serious conditions we always should evaluate before planning to upgrade our gears is, in my opinion, like to try describing how comfortable is a car only knowning its maximum speed. This make no sense.

Yes, it's a parameter we can take into consideration but megapixel is nothing a part the number of the simplest "electronics eye" of the system.
You cannot say "this system is better/worse than" just comparing how much "eyes" are sampling your scene.
Because a professional 12Mpxel system fore sure outperforms a more "densel" consumer camera.

And what if my camera is taking a "12 Perceptual Mpx" score? Is it good, average or bad?

I don't think you can say, as you have to deal with how light transitions (tones and colors) are reproduced.
MTF is not simple but a single value is too poor.

0 upvotes
AbrasiveReducer
By AbrasiveReducer (Dec 18, 2012)

Just skimming but any system of measurement that shows a Sigma lens outperforming a Zeiss lens, even Cosina-Zeiss, seems pretty suspect.

0 upvotes
kjbkix
By kjbkix (Dec 18, 2012)

check out the resolution tests of the new Sigma 35 - it's blowing away everything in its FL class

0 upvotes
ashwins
By ashwins (Dec 19, 2012)

To understand how good that Sigma lens is, please read on objective review by an unbiased site:

http://www.lenstip.com/index.php?test=obiektywu&test_ob=359

0 upvotes
AP7
By AP7 (Dec 18, 2012)

The definition and interpretation of terms: Acutance in Field Map (Acutance Map) and Profiles (Acutance Profiles) are not clear in dxomark website.

Both show % acutance as a function of field position for different focal lengths and apertures. While first shows in color code, the latter shows in % value from 50% to 90%.

Acutance is defined as follows:

http://www.dxomark.com/index.php/About/Glossary/A-B-C-D

My concern is how do we interpret the curves? For example, the acutance values in % at 35mm and f/5.6 are 68.6%, 67.6%, 65% and 61.4% at the center, 1/3 field, 2/3 field and corner respectively. The measurements are done on Canon 7D with 18 MP sensor (5184 pixel x 3456 pixel) and Canon EF 35mm f/2 lens.

Then, what does 68.6% mean? Does it mean that P-Mpix is 68.6% of 18MP at centre for that focal length and aperture combo? or perceptible horizontal resolution is 68.6% of 5184 pixels or what? The interpretation should be made clear with enough examples.

Comment edited 1 minute after posting
1 upvote
AP7
By AP7 (Dec 18, 2012)

@DxO Labs: Do you know the answer of the above question? If yes, could you explain?

1 upvote
AP7
By AP7 (Dec 20, 2012)

Interesting enough, even DxO Labs does not know the answer of what they are doing. Crazy metric, I believe!

0 upvotes
Artistico
By Artistico (Dec 18, 2012)

Um... wouldn't many of those 45% of megapixels lost due to "lens defects" (wouldn't it be better to call this "optical limitations"?) be due to the megapixels lost in Bayer interpolation?

0 upvotes
mosswings
By mosswings (Dec 18, 2012)

Not a lot, really. A bit of web-searching yields this: Bayer filters apparently don't affect luminance resolution that much. Color resolution is where Bayer interpolation takes its hits; it has to use color-luminance correlations to interpolate the color data.

1 upvote
Mr Fartleberry
By Mr Fartleberry (Dec 18, 2012)

This all smells like Pop Photo's "There is no bad lens" SQF baloney.

1 upvote
lensez
By lensez (Dec 18, 2012)

DXO ignored Einstein's recommendation: "Make everything as simple as possible, but not simpler."

1 upvote
Ronj2
By Ronj2 (Dec 18, 2012)

Well, if folks start to put faith in this measuring system it could put pressure on lens manufactures to produce better lenses across the board now that high MP sensors are becoming the norm. Why would we want to use a lens that only resolves 12mp or less on a 24 or 36 MP camera unless we just want bigger files without any real increased detail. So far there does not appear to be a single lens according to DXO that resolves 24-36mp. 18-24mp would seem to be plenty for any DSLR camera at this point with the best lenses.

Time wil tell.

0 upvotes
mosswings
By mosswings (Dec 18, 2012)

But lenses would have to get hugely better and far more expensive to do that. We've already gone through one or two iterations in the digital age to keep lens designs up with sensor advancements. Those were fairly easy.
Also remember that you still can see improvements with higher resolution sensors, it's just that they get less and less. At the 16MP point in APS-C, we're clearly seeing the limitations of cheaper kit zooms. At the 24MP point, midrange lenses are beginning to struggle a bit. But, agreed, I have a hard time seeing how this market will survive if everyone has to put a $3000 lens on their $500 entry-level camera body to pursue all of that body's high resolution.

0 upvotes
Reilly Diefenbach
By Reilly Diefenbach (Dec 18, 2012)

I think DXO is really out in left field with this 45% stuff.

0 upvotes
James O'Neill
By James O'Neill (Dec 18, 2012)

Firstly lenses are normally measured in 1 dimensional lines per mm - but we normally deal with sensors in pixels in two dimensions, not just width or height.

0 upvotes
James O'Neill
By James O'Neill (Dec 18, 2012)

Aargh DPR editing
Secondly you if the lens has the same linear res as the sensor you get half that linear res in the output. If it has twice the resolution you get 2/3 the res in the output, to get if has 9x the resolution you get 9/10ths.
With a 6000 x 4000 (24MP) Sensor, to get 5000x3000 15MP out you'd a lens which was 3 times the resolution of the sensor. In other words a 12000x18000 (216MP) lens. What the DXO numbers tell you is what many of us worked out long ago - you have to add a LOT of pixels to notice any increase in resolution. Another way to look at it is if you have 24MP lens you need a 216MP sensor to get a 15MP of resolution in the output

0 upvotes
John_I
By John_I (Dec 18, 2012)

Yes! I find the original analyses from DXO incomprehensible although very thorough I am sure. Something that brings the evident science in producing data to a better interpretation for me would be valuable! John at JBIPix.com

0 upvotes
ProfHankD
By ProfHankD (Dec 18, 2012)

"Perceptual MP" is a perfectly reasonable simplified way of expressing the standard line pairs per mm measurements. It suffers all the imprecision, etc., that any simplified metric suffers, but it sure would be nice to see fixed-lens compact cameras marked with "perceptual MP" rather than sensor pixel count or, worse, maximum interpolated pixel count. There are lots of cheap cameras with 5MP sensors, lenses that resolve more like 2MP, and a sticker on the box saying 12MP.

What I don't like is the claim that "45% of the resolution is lost due to lens defects." In my testing of over 100 lenses, it is very rare that image defects are caused by "lens defects." Resolution is just one of many design criteria; most lenses that resolve poorly are not "defective" -- they're designed giving more weight to some other attribute(s). It's a bit of bravado for DxO to claim lenses that score poorly on their simplified metric are defective. ;-)

0 upvotes
Simon97
By Simon97 (Dec 18, 2012)

Measurebaters rejoice!

1 upvote
gsum
By gsum (Dec 18, 2012)

Ha, Ha - just what I was thinking.
These measurements tell you precisely nothing about the way in which a camera/lens combination deals with e.g. random data at near pixel level (such as distant foliage) or whether slightly out of focus areas are rendered as mush.
Real world high res examples are more effective as they cover many more aspects of image quality than mtf charts etc.
Anyway carry on discussing - it's a good way of passing the time when the weather is no good for photography.

0 upvotes
Infared
By Infared (Dec 18, 2012)

I have learned to not pay any attention to ANY information that DxO disperses.
If you have any self-dignity you will take YOUR power back by getting yourself and your photos off Facebook AND Instagram, too!

Comment edited 3 minutes after posting
2 upvotes
mememe
By mememe (Dec 18, 2012)

Hows that tinfoil hat working out for you?

6 upvotes
AbrasiveReducer
By AbrasiveReducer (Dec 18, 2012)

I use Facebook to disperse my personal information to the widest possible audience and it seems to be working.

0 upvotes
Henry M. Hertz
By Henry M. Hertz (Dec 18, 2012)

people who can´t read a MTF chart should not bother with this measurement at all.

it´s like giving a school kid only one school grade.... averaged over all school subjects.

how can you say the kid is good in math?
it could be he is a genius in math but bad in all other subjects.

a MTF chart on the other hand will tell you way more about the lens and it´s performance.

how is a f1.2 lens (that is worse wide open) compared to a lens that starts at f4? how is the sharpness from f1.2 to f4 averaged in the single number?

this DXO test is flawed as every benchmark that only gives one number.

they should offer the charts (MP for given focal lenght and fstop) for center and border sharpness.... and stick the single number where the sun don´t shine.

Comment edited 13 minutes after posting
7 upvotes
Chekr
By Chekr (Dec 18, 2012)

Dear DXO,

Henry M. Hertz has decreed that us plebeians will not use this measurement, please remove it from your site immediately!

Kind regards,

Person who cannot read a MTF chart.

5 upvotes
rrccad
By rrccad (Dec 18, 2012)

agreed - way to oversimplified - same with the camera summary. the number cannot take into account how a person shoots - are they more interested in landscape (where the lens is stopped down and border to border is more important than wide open center performance?)

now if they did this by corner, center, and by aperture - we'd have a winner.

Comment edited 1 minute after posting
0 upvotes
Wing2
By Wing2 (Dec 18, 2012)

Even MTF is not telling the whole story - lens distortion, color, bokeh, IS performance.... and many more are available from MTF

To further reduce this to a single number is not very scientific to me

1 upvote
Andy Crowe
By Andy Crowe (Dec 18, 2012)

Sharpness varies widely across most lenses' frame and aperture values so I'm not sure how you can compress all that into a single figure?

2 upvotes
Henry M. Hertz
By Henry M. Hertz (Dec 18, 2012)

the single number is uninteresting for me.... but the graph shows how the lens performs on different appertures and focal lengths.

only problem is it does not differentiate between center and border sharpness.

the problem with DXO single number is that it gives one number you have to judge the performance on. to much is missing that is interesting for the photographer. how have they averaged between border and center sharpness for example?
does a lens with excellent center sharpness but very bad borders is judged better then a lens with good center sharpness and good border sharpness?

a car with 100 ps that weights 1 ton and has a cw value of 0,8 does not reach the speed of a car with 100 ps, weights 1 ton and has a cw value of 0.3.
so just giving the PS value does not say much.

another point is that every test of the EF 70-200mm f2.8 II IS shows this lens performes better then the old model.. only DXO says it does not.

and samyang lenses are better then zeiss... well yes.

Comment edited 3 times, last edit 8 minutes after posting
2 upvotes
marike6
By marike6 (Dec 18, 2012)

@Henry Hertz

Totally agree about the single number. But the Samyang 35 1.4 has produced similar amazing results at other test sites including Lenstip and Photozone. Not every Zeiss lens is automatically head and shoulders above the competition and Samyang has a few spectacular lenses.

0 upvotes
Henry M. Hertz
By Henry M. Hertz (Dec 18, 2012)

i admit i have to take a closer look but samyang seemed to have a lot of lenses at the top.

that is not my experience at all....

Comment edited 2 times, last edit 14 minutes after posting
0 upvotes
duartix
By duartix (Dec 18, 2012)

My opinion from the Samyang 7.5mm FE I use on the Panasonic GH2 is that it has stellar resolution. Most of the comparisons I've seen to the Panasonic 8mm FE will tell you the same.
Samyang doesn't have a name (yet) but if you forget AF, their glass has definitely something to say.
Anyway I support your conclusions on the single number. I don't go to DxOMark's site for their score, I go there for their graphs which I believe are invaluable. However, if they are condensing their lens results on a single figure these are useless for me.

0 upvotes
Henry M. Hertz
By Henry M. Hertz (Dec 18, 2012)

well i admit it IS hard to believe for me that cheap samyang lenses perform better then zeiss with all their decades of experience in construction and materials.

Comment edited 2 times, last edit 47 seconds after posting
0 upvotes
duartix
By duartix (Dec 18, 2012)

Cheap? By all means no!
The m43 version is MF and it still costs ~300€.
It's not surprised it performs noticeably better than the Panasonic 8mm, what is surprising is that the later is almost 3x more expensive when all it adds is AF and automatic correction (which the Samyang almost doesn't need).

0 upvotes
Henry M. Hertz
By Henry M. Hertz (Dec 18, 2012)

cheap when you compare it to zeiss glass....

0 upvotes
Eric Hensel
By Eric Hensel (Dec 18, 2012)

...so you're saying Zeiss glass is overrated, and overpriced?

1 upvote
The A-Team
By The A-Team (Dec 18, 2012)

I don't understand all the hate. Good job, DXO. I like the new measurement, and I'm sure TONS of people will find it more useful/comprehensible than MTF charts.

2 upvotes
Henry M. Hertz
By Henry M. Hertz (Dec 18, 2012)

well nikon looks bad in this test so it must be correct..... lol

0 upvotes
duartix
By duartix (Dec 18, 2012)

This time they have gone a step too far in oversimplifying.
I would more easily understand a P-Mpix graph / per Aperture.
This figure doesn't tell you how consistent a lens is (from center to corner) nor how it performs at each aperture.
This system paves way for a F/1.0 lens that is absolute garbage at fast apertures but performs good at F/4.0 to be the king of P-Mpixland.
And what is worse: from what I've gather it's not telling how good it performs across the focal range.

Comment edited 7 minutes after posting
0 upvotes
Henry M. Hertz
By Henry M. Hertz (Dec 18, 2012)

if you look at the provided chart they do offer values for focal lenghts and f-stops.

but they need two charts.... one for center sharpness one for border sharpness.

and the single number is just crap.

1 upvote
duartix
By duartix (Dec 18, 2012)

Ah... thanks!
Finally found some charts. That's the kind of information that's very useful.
I won't give a rat about it magical number though. And no numbers will tell me anything about the lens character either...

0 upvotes
Leonard Shepherd
By Leonard Shepherd (Dec 18, 2012)

The 4th reply from Mosswings is part right on the money. Those who understand optics know when the sensor and lens each has equal resolution the image resolution (what we are interested in) is 42% lower than either measured in isolation.
Confirming the average loss is 45% is no more than reinventing the wheel!
I see no advantage in relating lens performance to human vision.
The human eye perceives time at 1/60th of a second. With electronic flash we can record movement much shorter and with a tripod much longer than 1/60, creating visual movement impressions impossible without using a camera.
Many use wide angle and telephoto lenses to create visual angles of view the unaided fixed angle eye cannot perceive. In macro we can record detail much too small for the eye to see.
D and O have come up with a formula to make themselves look better than others. The reality is they give the impression of not having a clue as to what most photographers think photography is about.

2 upvotes
Dr_Jon
By Dr_Jon (Dec 18, 2012)

Actually it's a sum of squares job, you lose just under 30% from two identical resolution devices. Hence if the lens and camera can both resolve 21MP you'd get 14.8MP of resolution in theory, although I doubt DXO calculate it that way.

0 upvotes
Olibaer
By Olibaer (Dec 18, 2012)

How great is that! Now I can downsample all my huge 16 MPix files to 6 MPix without losing any quality since the lens isn't any better anyway... or is it?

0 upvotes
EdLu
By EdLu (Dec 18, 2012)

@mosswings - Interesting analogy! But I can't get your equation to work - I think the final result needs to be the reciprocal.

0 upvotes
mosswings
By mosswings (Dec 18, 2012)

sorry - the exponent should be (-0.5), not 0.5.
I think that when introducing a new figure of merit, it is incumbent upon the proposer to establish theoretical bounds for that F.O.M. to guide the user. If my analogy is correct, we should not expect any current or future imaging chain to yield more than about 75% of the lowest resolution element of that chain. Lenses are going to have a hard time keeping up with sensor resolution at reasonable price points. A 2:1 lens:sensor resolution ratio would be beyond the point of diminishing returns.

0 upvotes
EdLu
By EdLu (Dec 18, 2012)

I agree with what you are saying, but 75% sounds like we are losing a lot. It is just a technical number. Consider how good the image quality is with modern equipment, and how much better it is than a few decades ago.

0 upvotes
mosswings
By mosswings (Dec 18, 2012)

We are, but think a bit further: physics is physics. We have ALWAYS been losing something. Lens performance has evolved at a slower rate than sensor resolution, but started higher. 10 years ago at the start of the digital age lenses were way better than the 1-2 MP sensors of the era. So we saw most of the huge leaps in sensor resolution. A generation or two ago, things began slowing down. Lens designs started improving to stay ahead, to the extent that they were able. But they haven't been able to stay as far ahead of the sensor as they started off being. Lenses are becoming the limiting factor and will be so in the future.

Comment edited 55 seconds after posting
0 upvotes
rfsIII
By rfsIII (Dec 18, 2012)

So little of photography has to do with lens sharpness. In the real world, focus speed and especially focus accuracy are just as much if not more important determinants of overall picture quality. And at what distance do they do their testing?
I'm intrigued by their assertion that they can measure the interaction between camera and lens, however. That would be a great service for we readers of DPReview.
But to echo the question of another poster, how many samples of a lens and of a camera are they going to test to deliver this number?

Lensrentals dot com has permanently raised the bar in that respect; no reputable tester can look at just one sample of one lens or one camera and say they have lived up to their responsibilities to their readers.

2 upvotes
marike6
By marike6 (Dec 18, 2012)

Tons of lens testing websites including Photozone, and Lesntip test more than one sample of the same lens in case there is a de-centering defect in a given lens. Lensrentals is not unique in this.

0 upvotes
TN Args
By TN Args (Dec 18, 2012)

Oh good, now the dpreview lens tests, with their partner dxomark, can be summarized with one PMP number for each model of camera body it fits, in a drop down list.

So this is where dpreview was heading with their new lens testing regime!

Or, dpreview can ignore the PMP number and properly describe the lens' performance under various situations. Which is it to be? It can't be both, they will often contradict!

Also, does a lens' PMP number vary with aperture? And zoomed focal length? Or is it some kind of average?

Dxo dxoes it again!

1 upvote
BoatGuy
By BoatGuy (Dec 18, 2012)

If they haven't profile my lenses, for example the Sony E 16-50, then I don't care.

DxO is all promise and hype, short on delivery.

0 upvotes
skyfotos
By skyfotos (Dec 18, 2012)

Sharpness is a subjective assessment (i.e. racking a lens in and out of focus and judging when it seems to be sharpest). Resolution is an objective measurement and free from subjective assessment.

0 upvotes
mosswings
By mosswings (Dec 18, 2012)

This measure is sort of like a bandwidth calculation for chained electronic amplifiers - for a 2 amp system, the final BW is (1/(B1)^2+1/(B2)^2)^0.5 where B1 and B2 are the amps' respective bandwidths.
Using this analogy, one can see that if the resolution of the lens (B1) is equal to the resolution of the sensor (B2), then the final resolution will be 0.707 B. If the resolution of the sensor is twice that of the lens, or vice versa, the final resolution will be 0.89*the lower resolution.
So even the very best of today's lenses which approximately equal the resolution of today's best sensors cannot be expected to yield more than about 70% of the effective resolution of today's sensors. One would have to make a lens that is at least TWICE as good as the sensor to see most of the resolution of the sensor.
Of course, this is just an analogy, but it indicates what sort of effective resolutions we're still dealing with in this high-MP age. Food for thought.

0 upvotes
James O'Neill
By James O'Neill (Dec 18, 2012)

@mosswings
What I understood was it was 1/final res = 1/lens res + 1/sensor res (no need to square and then square root them)
And this assumes res is defined in lines/pairs per unit distnace, at the same MTF throughout.
If lens res = sensor res final res = 1/2 that.

Keep in mind that the bayer sensor effectively halves the resolution too

0 upvotes
Bob from MA
By Bob from MA (Dec 18, 2012)

I am against the new perceived megapixel system.
Even the old DXO lens-plus-camera system has serious flaws. To add a human vision assumption just makes it worse. I want to know what the lens can do under ideal conditions.
In the past 10 years, I have owned one printer and 4 DSLRs: The printer has been capable of 116 megapixels at 200 DPI on a 44x66 inch print. The cameras started with a 6 megapixel, then a 12 megapixel, a 21 megapixel, and now the 36 megapixel Nikon D800E. I want a lens that can make a good 44x66 inch print!
We will all keep our lenses for several generations of cameras. Tell us what lens to buy that will fit our camera of the future.

1 upvote
wmac
By wmac (Dec 18, 2012)

Perhaps I don't get it.

Human perception of what? A printed photo at a specific size? How if I print it on a huge banner?

I guess a normalized MTBF (i.e. percentage of identifiable pixels) would be more effective and understandable.

For example a 55% normalized MTBF on a 20M camera would mean 11M pixels are identifiable. And the percentage is very easy to understand.

0 upvotes
Plastek
By Plastek (Dec 18, 2012)

Even that sux.
- on what aperture?
- which focal lenght? (essential factor for zooms)
- how far was the testing board? (makes a huge difference for landscape and macro lenses)

You CAN'T summarize lens resolution or sharpness with a single number. It just doesn't make any sense.

2 upvotes
Drew Faber
By Drew Faber (Dec 18, 2012)

I like being able to compare different cameras easily!
I hope they post both scores... with DxO correction and without it.

0 upvotes
zodiacfml
By zodiacfml (Dec 18, 2012)

This is an awesome idea but something would be frowned upon by manufacturers and gear heads. I mean, with this new system one would easily see if an expensive lens is worth it on a moderate/low resolution camera or even those shooting in jpeg. This would also mildly moderate the MP race since the lens often limits the current MP levels we have today.
Sigma and Tamron would be recognized as true alternatives as they work good enough to those who buy them.
Can't wait for the lens reviews.

2 upvotes
rrccad
By rrccad (Dec 18, 2012)

but it doesn't. alot of the lower priced lenses and third party lenses work well stopped down against their more expensive brand name peers. so depending on what you shoot - you may be able to get away with a cheaper variant.

this won't tell you that, and you still have to go to slrgear or photozone.de to see how the lens performs in the way you are going to use it.

0 upvotes
Entropius
By Entropius (Dec 18, 2012)

There are a lot more things to a lens's performance than a one-dimensional measurement. What does this scheme make of a lens like the Panasonic 14/2.5, which is extremely sharp but has high chromatic aberration? What about the Canon 70-200/2.8 mk1 at 200/2.8, which has poor microcontrast but resolves a good amount of detail? The high sharpness and resolution, but bizarre bokeh, of the Zuiko 50-200?

What about a classic lens like my Olympus OM 50/1.8, which is surprisingly highly-resolving wide open in the middle of the frame (great for portraits) but without the high acutance of modern asphericals, and with a strong dropoff in sharpness toward the edge?

There's a whole lot more to a lens's performance than one number. That's why the old DPReview lens reviews were great, with the little checkerboards: you can see for yourself what the point spread function at different places in the frame is.

4 upvotes
Erik Magnuson
By Erik Magnuson (Dec 18, 2012)

Try actually looking at how DxO uses this number before ranting. There is a one number score (called the DxOMark score) that weights "sharpness, distortion, vignetting, transmission and chromatic aberration." This particular change only deals with the units in the "sharpness" area of their measurements.

2 upvotes
Donald Duck
By Donald Duck (Dec 18, 2012)

Sharpness at what aperture? Where - center, borders?

5 upvotes
Erik Magnuson
By Erik Magnuson (Dec 18, 2012)

It's a weighted average across the field for the "best" aperture. No they don't tell you the weighting used.

0 upvotes
random78
By random78 (Dec 18, 2012)

> It's a weighted average across the field for the "best" aperture.

Right. And for a zoom lens it is at whichever arbitrary focal length + aperture combination gave the best result for that lens. So a lens gets good score because at some particular combination of focal length and aperture, using some specific unknown weighting of center and corner sharpness it performs well . I wonder how could you consider it a useful measure despite the comparison being so arbitrary. As an example under this measure Canon 24-105mm f4 and Canon 18-55mm IS II have the same P-MPix Score on a 7D.

Comment edited 1 minute after posting
0 upvotes
Donald Duck
By Donald Duck (Dec 18, 2012)

There is more they do not tell me. How they get that number. Where is the human vision involved in all this. It is an documented metric. It is part of a trend to dumb down the data to please the average reader.

"The Perceptual Megapixel measurement allows photographers to step away from manufacturers' announcements about resolution and to answer a key question when the time comes to change equipment: would it be better to buy a new camera or a new lens?"

They have been testing lenses+cameras and reporting combined resolution even before this. My guess is that they use the inverse square law to blur further the measurements according to what they think the human vision does for a fixed size and viewing distance. Many people can understand what MTF-50, for example, is; but almost nobody would understand the new metric - especially when they do not even bother to explain it.

0 upvotes
S_Michaelsen
By S_Michaelsen (Dec 18, 2012)

There is more to lens than sharpness, allot more.

0 upvotes
jquagga
By jquagga (Dec 18, 2012)

No, I don't think this number would be useful in reviews. However since you're probably going to include it anyway, I'd appreciate LPI figures since those are somewhat more substantial.

1 upvote
pkincy
By pkincy (Dec 17, 2012)

A chap named Robert Parker made himself famous as well as likely very wealthy by taking the old English Prose out of play when fine wines were tasted and described. He reduced the tens of thousands of factors that make a wine either favorable or unfavorable into an easily understandable by the "wine novice want to be wine snob" arithmetic scale that is very similar to US grade school grading systems. From the mid 80's forward you could not sell a wine that fell below his 85 rating and you would quickly sell out of anything he rated over 90 pts.

None of this translated to whether or not you liked the wine but translated wonderfully well to the marketing types at wineries and fine wine retailers.

This Mpx thing strikes me as similar. If it takes off soon, no lens will ever be made or, if made, sold without a Mpx rating of over 80% of the Mpx of the camera body it is tested on. A little like the camera body Mpx war but worse.

Comment edited 20 seconds after posting
9 upvotes
Richard Murdey
By Richard Murdey (Dec 17, 2012)

There is something soul-crushing about this, like frat boys scoring women on a 0-10 scale.

14 upvotes
AbrasiveReducer
By AbrasiveReducer (Dec 18, 2012)

One of the best things about film photogrpahy was that it didn't attract the mathematicians the way digital does. We just took pictures and the more expensive stuff usually worked better. The zone system was about as complicated as things got.

0 upvotes
Timmbits
By Timmbits (Dec 17, 2012)

I'm very skeptical about their claims that some lenses "waste megapixels" (referring to the claims on their website). It just looks like some kind of a new metric that, if accepted (and that's a big unlikely if), then they get control over whoever wants to use that metric, because they will have to be mentioned, and get free publicity to sell their software.
I prefer to rely on independent metrics, so that I may choose my source conducting the tests. Only that way can I be sure of the quality and impartiality of the testing.

Comment edited 45 seconds after posting
3 upvotes
iAPX
By iAPX (Dec 17, 2012)

MPix is an interesting metric to quickly evaluate options, with a body+lens combo. I think it's relevant as a comparison metric for the mass.

My metrics are *NOT* a statistical average metric, nor what a lens could do on it's center, I don't care about it's better features, but instead it's worst characteristic, to be sure everything is correct and moreover homogenous.

Because I use the whole frame for my work, and what's matter for me is to have great quality from side to side, including corners, not a perfect quality on the center of the lens. Homogeneity is a better value for me than performance in some area (center usually), and would die for a great lens with great corners, few or no distorsion, conceived to let the center area work exactly as the corners or edges :)

EDIT: some typos (sorry my native language is french)

Comment edited 3 times, last edit 5 minutes after posting
1 upvote
iAPX
By iAPX (Dec 17, 2012)

I just looked at the Nikon 35mm f/1.8 and Tamron 17-50 f/2.8 stabilized, both used between f/8 and f/13 for photo studio, and their measure correlate with my subjective impression: in this case I prefer to use the Tamron and have global better performance.

Even the MPix on a D300 is better for the Tamron (8Mpix instead 7Mpix).

Sorry for not using incredible gears, but in real life, and hairtyler contests, it works pretty well with a great sharpness (BEFORE processing) that show on hairs.

0 upvotes
Total comments: 182
12