DXOMARK.com, did you notice

Started 8 months ago | Discussions thread
Anders W
Forum ProPosts: 17,529Gear list
Like?
Re: DXOMARK.com, did you notice
In reply to blue_skies, 8 months ago

blue_skies wrote:

Joel Stern wrote:

Not one m43 lens gets a good rating, no matter what body it was tested with. What's with that?

-- hide signature --

If i am typing on my iPad, please excuse any typos.

Hi Joel,

I was looking at this a while back, and noticed your question - here are my comments:

First, read the DxOMark's explanation: http://www.dxomark.com/Reviews/DxOMark-Score (and click on the reasons behind DxOMark's score), and also http://www.dxomark.com/Reviews/DxOMark-Score/DxOMark-Score-design

Consider this: (quoting)

A DxOMark Score of 10 is high enough for an excellent 30x20cm print. When doubling each dimension (hence multiplying the print surface by four), the DxOMark Score should ideally be multiplied by four also. Typical correspondences are indicated in the table below:

  • 10 = 30x20cm
  • 20 = 46x40cm
  • 40 = 60x40cm

Furthermore, they define the information capacity, of a camera, as

"However, for a given sensor, it is usually about twice the sensor pixel count."

This is something to consider: m43 cameras therefore have a maximum rating of 32Mpix, whereas 36Mp FF cameras have a maximum rating of 72Mpix. Big difference.

However, the highest rating lens (OTUS 55/1.4) only scores 45Mpix (62%), whereas the highest m43 lens (Oly 75/1.8) scores 27Mpix (84%). I read that as the smaller format performing admirably.

In addition, the typical high-end rating for FF hovers around 33Mpix, whereas for m43 it hovers around 23Mpix. If you then take into account the table above, 23Mpix to 33Mpix, the difference is not that dramatic. In fact, printers that upscale could bring this difference down even further.

The information capacity score and P-MPix are two different things. P-MPix is a measure of sharpness only and can theoretically reach a max value equal to the pixel count. The ínformation capacity score is measured on an open scale and aggregates a number of different properties. As already pointed out here,

http://www.dpreview.com/forums/post/52999723

some of these properties are essentially the same as those captured by DxO sensor measurements/scores, which implies a sort of double-counting, FF being rewarded twice for essentially the same thing.

Test Bias.

And they explain the test bias (paraphrasing):

a lens scores higher if it is faster, has a longer focal length, is mated to a higher resolving camera(sensor), amd works better when used wide open. DOF is not considered. Their Mpix score is a roll-up of weighted averages, and can be far below the 'best' result (usually at a stopped down aperture).

This makes a little bit of sense, for the reason that all lenses sharpen when stopped down, so a faster, more expensive lens may not hold a benefit at a stopped down aperture.

As a photographer, you'll want to know which is the best lens/camera combination for your purpose, and the DxOMark helps you analyzing this.

Filtering.

You should use the filters on the side-bar, e.g. only consider only m43 lenses, so you can see a relative rating of which lenses work better on which cameras. Of course, you will find f/1.4 lenses placing above f/1.8, and longer FL typically being above shorter FL, and primes outclassing zooms, but it gives you a quick indicator what to expect.

Resolution.

What was already mentioned, sensor resolution is the biggest contributor to the score: a 36Mp D800E outresolves a 24Mp D610, and the scores show this. The same lens (e.g. Sigma 60) on a 24Mp APS-C (Nex-7) scores above a 16Mp m43 (EM1).

When numbers are close (+/-10%), expect similar results: e.g the Sigma 60 mentioned above on a Nex-7 and a EM1 actually resolves almost the same amount of detail (both Nex-7 and EM1 have almost the same pixel pitch - the Nex-7 has a larger area, and less crop), that is, within the crop overlap area you'd see about identical details.

Lenses across formats.

Then, in another context, DxOMark is not rating adapted lenses. E.g. compare the Cosina Voigtlander 35mm f/1.4 Nokton (CV35) with the Zeiss M 35mm f/2.0 (ZM35). How to rate such lenses?

From personal experience, on the highest resolving APS-C Nex-7 (same as EM1), the ZM35 clearly places above the CV35 (more pop, more microcontrast, sharpest detail), even though it is a stop slower. But on the A7, the CV35 is actually sharper at the edges at f/2.0 than the ZM35, so it would flip the score in the favor of the CV35. Same lenses, different sensors.

  • (Note: technically this can all be explained in lp/mm and lp/ph, resolving power of the lens, resolving power of the sensor, and so on - The FF and APS-C in this case render 24Mp, but the FF has a much higher picture height ph, so it requires a lower lp/mm to render all detail than the smaller APS-C.)

Next: rendering

The test are dry, technical, scores, with some interpretation of color authenticity, according to the DxOMark comments. But a lens sometimes has 'character', or 'bokeh', or renders 'pleasingly'. This can very subjective, and is harder to quantify.

E.g. the ZM35 is very sharp, edge to edge, with a flat focal plane. It is said to have a 'clinical' look, whereas the CV35 has noticeable field curvature, but renders very nice with people subjects, it is said to have a 'warm' look. Neither lens is 'best', they do better at different tasks.

Lastly, your personal style

If your typical scene is not a low-light, slow shooting, opportunity, but an outdoors, bright light, the DxO scores mean very little. I mean, any lens, 10Mpix or 40Mpix, when stopped down, will most likely allow to print big. I think that the DxOMark scores can be misleading - you'd have to consider each lens in their table, and see the score at the optimal settings (e.g. f/8). It can have a much higher score.

Or, consider this: I am comparing (Nex lenses) the E35 with the E18200. The E35 prime gets a score of 21Mpix, whereas the E18200 gets a score of 13Mpix. Wow, you may go, the prime is sooo much better. But if you consider e.g. the sharpness score, you get 11P versus 7P. Much closer than 21 versus 13.

Furthermore, lenses are corrected through optical profiles, either in-camera or in-pp. This reduces some of the measurements and may not be properly reflected. A lens, after correction, may be as good as a (better) lens that needs no correction, after all.

Conclusion:

Start from your own perspective: which format is your camera, what focal length lens are you interested in, what type (prime/zoom) and how fast do you want the lens to be (=price)?

Given this premise, the DxOMark lens database will give you quick answers as to which lenses to consider: e.g. Oly, Pany, Sigma, and so on. Then the scores are useful.

But trying to qualify a lens by absolute ranking is silly. Sure, a D800E with OTUS 55/1.4 is a great combination, but it is large and bulky, and will set you back almost $7k.

And, by the way, the D800/E + OTUS 55/1.4 is not stabilized - is it really the best choice for low light?

Typical low-light scene, 150 lux, 1/60th exposure, from DxOMark's explanation page

-- hide signature --

Cheers,
Henry

 Anders W's gear list:Anders W's gear list
Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Olympus E-M1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS Panasonic Lumix G Vario 7-14mm F4 ASPH +21 more
Reply   Reply with quote   Complain
Post (hide subjects)Posted by
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark post MMy threads
Color scheme? Blue / Yellow