DXOMARK.com, did you notice

Started Jan 28, 2014 | Discussions
Anders W Forum Pro • Posts: 21,468
Re: DXO scores are like MTF charts

Ulric wrote:

Anders W wrote:

Ulric wrote:

Anders W wrote:

I see no principal reason why a 150 mm lens with an MFT image circle could not behave as a 300 mm lens with an FF image circle (at the same f-stop). What would that be?

If the comparison is performed at the same f-number across formats, the behaviour will not be the same for any non-flat scene.

I know. I just mentioned the comparison for which I'd expect pretty much the same design challenges and therefore pretty much the same performance (at any given level of ambition and until we enter serious diffraction territory for either format).

If, instead, we compare at equivalent f-stops, I'd expect FF to be ahead by an amount varying with the point at which we compare as follows: virtually nil at f/8 (MFT) versus f/16 (FF), barely noticeable at f/4 versus f/8, noticeable but not very significant at f/2 versus f/4, and pretty dramatic at f/1 versus f/2.

If the shutter speed and ambient light are also kept the same, interaction with the sensor performance should have an impact on the results. But I haven't been paying enough attention to the DxO lens tests to know if they do that or not.

I don't think they test sharpness at anything but lowest ISO with exposure appropriate for that ISO. They illuminate the target by means of flash by the way.

But for the total score (information capacity score) of a lens, they factor in its ability to accumulate light, which is a function of its max aperture (or rather actual transmission) and its image circle. In other words, the sensor size for which the lens is made enters the equation here, which amounts to a kind of "double-counting" since the same factor is taken into account when sensors are evaluated as well. If you just ignore the score and look at the individual measurements, that doesn't cause any problems though.

 Anders W's gear list:Anders W's gear list
Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Olympus E-M1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS Panasonic Lumix G Vario 7-14mm F4 ASPH +28 more
Ulric Veteran Member • Posts: 4,532
Re: DXO scores are like MTF charts

Anders W wrote:

Ulric wrote:

Anders W wrote:

Ulric wrote:

Anders W wrote:

I see no principal reason why a 150 mm lens with an MFT image circle could not behave as a 300 mm lens with an FF image circle (at the same f-stop). What would that be?

If the comparison is performed at the same f-number across formats, the behaviour will not be the same for any non-flat scene.

I know. I just mentioned the comparison for which I'd expect pretty much the same design challenges and therefore pretty much the same performance (at any given level of ambition and until we enter serious diffraction territory for either format).

If, instead, we compare at equivalent f-stops, I'd expect FF to be ahead by an amount varying with the point at which we compare as follows: virtually nil at f/8 (MFT) versus f/16 (FF), barely noticeable at f/4 versus f/8, noticeable but not very significant at f/2 versus f/4, and pretty dramatic at f/1 versus f/2.

If the shutter speed and ambient light are also kept the same, interaction with the sensor performance should have an impact on the results. But I haven't been paying enough attention to the DxO lens tests to know if they do that or not.

I don't think they test sharpness at anything but lowest ISO with exposure appropriate for that ISO. They illuminate the target by means of flash by the way.

But for the total score (information capacity score) of a lens, they factor in its ability to accumulate light, which is a function of its max aperture (or rather actual transmission) and its image circle. In other words, the sensor size for which the lens is made enters the equation here, which amounts to a kind of "double-counting" since the same factor is taken into account when sensors are evaluated as well. If you just ignore the score and look at the individual measurements, that doesn't cause any problems though.

So if I understand correctly, this image compares images shot at different lighting conditions, meaning that the results do not really translate to e.g. shooting in available light:

 Ulric's gear list:Ulric's gear list
Panasonic Lumix DMC-GF3 Olympus OM-D E-M5 Panasonic Lumix DMC-GM1 Olympus PEN-F Panasonic Lumix G 20mm F1.7 ASPH +13 more
Anders W Forum Pro • Posts: 21,468
Re: DXO scores are like MTF charts

Ulric wrote:

Anders W wrote:

Ulric wrote:

Anders W wrote:

Ulric wrote:

Anders W wrote:

I see no principal reason why a 150 mm lens with an MFT image circle could not behave as a 300 mm lens with an FF image circle (at the same f-stop). What would that be?

If the comparison is performed at the same f-number across formats, the behaviour will not be the same for any non-flat scene.

I know. I just mentioned the comparison for which I'd expect pretty much the same design challenges and therefore pretty much the same performance (at any given level of ambition and until we enter serious diffraction territory for either format).

If, instead, we compare at equivalent f-stops, I'd expect FF to be ahead by an amount varying with the point at which we compare as follows: virtually nil at f/8 (MFT) versus f/16 (FF), barely noticeable at f/4 versus f/8, noticeable but not very significant at f/2 versus f/4, and pretty dramatic at f/1 versus f/2.

If the shutter speed and ambient light are also kept the same, interaction with the sensor performance should have an impact on the results. But I haven't been paying enough attention to the DxO lens tests to know if they do that or not.

I don't think they test sharpness at anything but lowest ISO with exposure appropriate for that ISO. They illuminate the target by means of flash by the way.

But for the total score (information capacity score) of a lens, they factor in its ability to accumulate light, which is a function of its max aperture (or rather actual transmission) and its image circle. In other words, the sensor size for which the lens is made enters the equation here, which amounts to a kind of "double-counting" since the same factor is taken into account when sensors are evaluated as well. If you just ignore the score and look at the individual measurements, that doesn't cause any problems though.

So if I understand correctly, this image compares images shot at different lighting conditions, meaning that the results do not really translate to e.g. shooting in available light:

If you mean that they are shot with the same exposure (same total amount of light on the sensor), yes. I am pretty sure that the sharpness results are the very best the camera-lens combo can do at the respective f-stop aperture, which implies full exposure at base ISO.

As to the 45/1.8 specifically, be aware that DxO in all likelihood didn't (and doesn't) have a perfectly copy of that lens. Look at the sharpness profile for wide open (probably f/2 as well) versus f/2.8 versus f/4 and you'll see what I mean. According to DxO, sharpness falls rather than increases (especially at the edges IIRC) when the lens is stopped down to f/2.8. That shouldn't be the case for a healthy copy and certainly isn't for mine. The fact that they appear to have a so-so copy of course make the sharpness values suspect in general.

Based on the reports that I have seen here, there is considerable sample variation for this particular lens. Lenstip apparently got a good copy and I think their figures are pretty much in line with those you can expect when everything is in order. My own copy is good as well, so good that it matches what my 75/1.8 delivers if only I stop down the 45/1.8 an inkling more than the 75. The first copy I tried, however, was less than perfect, as you can see here:

http://www.dpreview.com/forums/post/42257923

 Anders W's gear list:Anders W's gear list
Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Olympus E-M1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS Panasonic Lumix G Vario 7-14mm F4 ASPH +28 more
Ulric Veteran Member • Posts: 4,532
Re: DXO scores are like MTF charts

Anders W wrote:

Ulric wrote:

Anders W wrote:

Ulric wrote:

Anders W wrote:

Ulric wrote:

Anders W wrote:

I see no principal reason why a 150 mm lens with an MFT image circle could not behave as a 300 mm lens with an FF image circle (at the same f-stop). What would that be?

If the comparison is performed at the same f-number across formats, the behaviour will not be the same for any non-flat scene.

I know. I just mentioned the comparison for which I'd expect pretty much the same design challenges and therefore pretty much the same performance (at any given level of ambition and until we enter serious diffraction territory for either format).

If, instead, we compare at equivalent f-stops, I'd expect FF to be ahead by an amount varying with the point at which we compare as follows: virtually nil at f/8 (MFT) versus f/16 (FF), barely noticeable at f/4 versus f/8, noticeable but not very significant at f/2 versus f/4, and pretty dramatic at f/1 versus f/2.

If the shutter speed and ambient light are also kept the same, interaction with the sensor performance should have an impact on the results. But I haven't been paying enough attention to the DxO lens tests to know if they do that or not.

I don't think they test sharpness at anything but lowest ISO with exposure appropriate for that ISO. They illuminate the target by means of flash by the way.

But for the total score (information capacity score) of a lens, they factor in its ability to accumulate light, which is a function of its max aperture (or rather actual transmission) and its image circle. In other words, the sensor size for which the lens is made enters the equation here, which amounts to a kind of "double-counting" since the same factor is taken into account when sensors are evaluated as well. If you just ignore the score and look at the individual measurements, that doesn't cause any problems though.

So if I understand correctly, this image compares images shot at different lighting conditions, meaning that the results do not really translate to e.g. shooting in available light:

If you mean that they are shot with the same exposure (same total amount of light on the sensor), yes. I am pretty sure that the sharpness results are the very best the camera-lens combo can do at the respective f-stop aperture, which implies full exposure at base ISO.

Which would mean 1.6x the exposure and 6.4x the total light for the D3X. What the result would be for equal exposure or equal total light is not possible to tell from these measurements. The implication being that these results depend on the assumption that light is unlimited. It doesn't invalidate the results because that is sometimes true, it just means that they are not generally applicable.

 Ulric's gear list:Ulric's gear list
Panasonic Lumix DMC-GF3 Olympus OM-D E-M5 Panasonic Lumix DMC-GM1 Olympus PEN-F Panasonic Lumix G 20mm F1.7 ASPH +13 more
Anders W Forum Pro • Posts: 21,468
Re: DXO scores are like MTF charts

Ulric wrote:

Anders W wrote:

Ulric wrote:

Anders W wrote:

Ulric wrote:

Anders W wrote:

Ulric wrote:

Anders W wrote:

I see no principal reason why a 150 mm lens with an MFT image circle could not behave as a 300 mm lens with an FF image circle (at the same f-stop). What would that be?

If the comparison is performed at the same f-number across formats, the behaviour will not be the same for any non-flat scene.

I know. I just mentioned the comparison for which I'd expect pretty much the same design challenges and therefore pretty much the same performance (at any given level of ambition and until we enter serious diffraction territory for either format).

If, instead, we compare at equivalent f-stops, I'd expect FF to be ahead by an amount varying with the point at which we compare as follows: virtually nil at f/8 (MFT) versus f/16 (FF), barely noticeable at f/4 versus f/8, noticeable but not very significant at f/2 versus f/4, and pretty dramatic at f/1 versus f/2.

If the shutter speed and ambient light are also kept the same, interaction with the sensor performance should have an impact on the results. But I haven't been paying enough attention to the DxO lens tests to know if they do that or not.

I don't think they test sharpness at anything but lowest ISO with exposure appropriate for that ISO. They illuminate the target by means of flash by the way.

But for the total score (information capacity score) of a lens, they factor in its ability to accumulate light, which is a function of its max aperture (or rather actual transmission) and its image circle. In other words, the sensor size for which the lens is made enters the equation here, which amounts to a kind of "double-counting" since the same factor is taken into account when sensors are evaluated as well. If you just ignore the score and look at the individual measurements, that doesn't cause any problems though.

So if I understand correctly, this image compares images shot at different lighting conditions, meaning that the results do not really translate to e.g. shooting in available light:

If you mean that they are shot with the same exposure (same total amount of light on the sensor), yes. I am pretty sure that the sharpness results are the very best the camera-lens combo can do at the respective f-stop aperture, which implies full exposure at base ISO.

Which would mean 1.6x the exposure and 6.4x the total light for the D3X. What the result would be for equal exposure or equal total light is not possible to tell from these measurements. The implication being that these results depend on the assumption that light is unlimited. It doesn't invalidate the results because that is sometimes true, it just means that they are not generally applicable.

While what you say is true in principle, it is of little practical importance at base ISO (and some way up the ISO scale) for sharpness tests of the kind conducted by DxO. Where the amount of total light and its implications for noise come in is when we get to sharpening and NR. With higher SNR you can sharpen more and need less NR.

 Anders W's gear list:Anders W's gear list
Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Olympus E-M1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS Panasonic Lumix G Vario 7-14mm F4 ASPH +28 more
Keyboard shortcuts:
FForum MMy threads