JohnNEX

Veteran Member
Messages
2,715
Solutions
5
Reaction score
4,048
If you read the various lens review websites you will have noticed that some include metrics for sharpness, which are usually scores of "Line widths per Picture Height" (LW/PH) calculated with imatest. I decided to try to aggregate all of this data. Yep, I am a numbers junky and this will be very nerdy stuff so don't say I did not warn you. Also, if you don't like MTF data etc then there is nothing here of interest for you.

The websites with several reviews with sharpness measurements are:

Photozone.de

ephotozine

erphotoreview

dpreview (using dxomark data)

lensrentals

I am not including dxomark's own website data here because they do not provide the source data for the scores.

The problem is that the websites are not consistent with the camera used. A lens tested on an NEX-3 will give much worse sharpness scores than the same lens tested on an a6000. Most of the websites are consistent with the camera used for their own tests, the exception being ephotozine.

You can see the ratings for maximum centre sharpness score from each review site in the table below. I have only included the ratings for a single camera from each site. The blank coloured areas for ephotozine ratings show where the lens was tested but with a different camera from usual. [continued below the table]

a3ce6ed156d84674ae91acb107884564.jpg

[For the zooms, the focal lengths were: 200mm for the 55-210; 10mm for the 10-18; 24mm (ish) for the 16-50; 30mm for the 18-55 and 70mm for the 18-200]

You will see that the ratings are not directly comparable, in the sense that the numbers are benchmarked differently. I have run the data through a regression to calculate the relative scalings for each reviewer and taking the average of the scaled results, which are shown in the table below (worst to best), with the scores put onto the photozone.de basis. The 'obs' column just shows the number of data points ('observations'). There are five separate measurements for the Sony 50mm prime, for example.

a9b6ceae1f404eb09875bc45f758e5aa.jpg

Mostly, this lines up with what seems to be a consensus view, at least from my reading, with a few very weird results.

The big surprise is the high ranking of the a6000 kit lens (16-50mm), where the maximum sharpness (typically at f/5.6) was measured as better than the 50mm prime in both cases (photozone and erphotoreview). Owning both lenses, I find that difficult to believe but maybe I need to test mine properly. Keep in mind that the measurements from erphotoreview were done with a NEX-3, where aliasing may be a problem, although I don't know why that would affect some lenses and not others.

The two Sony main primes (35 and 50) are very low on the list, but there are five different measurements for the 50mm here and all five are a bit middle-of-the-road so perhaps there is something in that. The ratings from Lens Rentals were (I think) done with multiple copies of each lens, and the 50mm has the worst sharpness of the primes, even lower than the 16mm. The 18-55 kit lens is measured as sharper than the 50mm in two of the three measurements (and its close in the third).

Looking now at maximum edge sharpness, the rankings are:

49c78b438c6e4de5a5cc06d6f1bea963.jpg

Here the core Sony primes move well up the list while the Zeiss lenses drop a few places. The Sigma 19mm is disappointingly low ranked in both. As Roger at Lens Rentals says, "The Sony 50mm f/1.8 appears to be tuned like the Nikon 50mm f/1.4 G: it’s not quite as sharp in the center, but keeps good resolution across the entire front of the lens." I guess it comes down to your preference.

Of course, none of this takes into account lens speed, colours, distortion, 'pop', CA and all kinds of other stuff which are also important for lenses.

Cheers,

John.
 
Last edited:
Pretty much all available lenses today have a good centre sharpness when stopped down (which it sounds like you're looking at here). I think it's much more interesting to look at corner/edge sharpness or sharpness wide open.
 
Thanks for that.

But, Mate, your going to get yourself in deep trouble by scoring [edges] the Sigma 30mm f/.28 DN A higher that either the Z Touit 32mm or the SCZ 24mm ZA :-O
 
Last edited:
Thanks for that.

But, Mate, your going to get yourself in deep trouble by scoring [edges] the Sigma 30mm f/.28 DN A higher that either the Z Touit 32mm or the SCZ 24mm ZA :-O
Ha - I'll be careful to not give out my home address :-)

I guess its ok because these were not my scores. Roger at Lens Rentals measured the Sigma 30mm as sharper than the Zeiss 24mm at both centre and edges. Photozone.de measured the Sigma 30mm as sharper than the Zeiss 32mm at the edges but not the centre.

Interestingly, the ratings on B+H photo have the Sigma 30mm with an average of 4.62, the Zeiss 24mm at 4.71 and the Zeiss 32mm at 4.38. Of course, those ratings will partly reflect perceived value for money as well as raw performance.

The Amazon ratings are quite different, with the Sigma rated very low (average of 4.08), the Zeiss 24 at 4.69 and Zeiss 32 at 4.54.
 
Wow. Very cool of you to do this.

A few reactions.

1. I am likewise surprised by the kit showing such good scores relative to some of the primes. I wonder how much of a sharpness hit the kit lens would take from the software corrections for its massive distortion. The photozone review shows a substantial sharpness decrease from software correction at 16mm but did not show results from other focal lengths (http://www.photozone.de/sony_nex/842-sony1650f3556oss?start=1). I doubt many people use uncorrected raw files from the kit lens. So the PZ review (and perhaps others - I didn't check) may be scoring the kit a little too highly due to not including the hit from software distortion correction.

That being said, I have liked the kit lens. I generally use other lenses because I am often looking for better DOF control, not because I am too displeased with the sharpness of the kit lens.

2. I have also perceived the Sony 50mm 1.8 as substantially sharper than the kit lens. Maybe the shallower DOF just makes the subject 'pop' more, which might be perceived as greater sharpness?

3. I have not done a formal analysis, but it seems like DXOMark is coming up with somewhat different scores for sharpness. It would sure be nice to know how well these various ratings hold up in real world shooting.
 
Where's the 18-105mm?
I have found only two websites with sharpness scores for the 18-105mm.

The first is ephotozine, which usually uses an NEX-5 for testing but used an NEX-7 for the 18-105, which makes the results not comparable. Annoying. They also used an NEX-7 for the 16-70mm. Of those two lenses, the 18-105mm was measured as quite a lot sharper, particularly wide open.

The other website with measurements for the 18-105mm is 'pcmag', which only has scores in the text, rather than charts or tables, and mostly they choose not to say much about them anyway. pcmag also measured the 16-70mm. Both were on an a5000. In this case, the 16-70 came out a lot sharper in the centre but the 18-105 was sharper at the edges.

If you know of any other websites with sharpness scores then please let me know!
 
Thanks for that.

But, Mate, your going to get yourself in deep trouble by scoring [edges] the Sigma 30mm f/.28 DN A higher that either the Z Touit 32mm or the SCZ 24mm ZA :-O
I guess its ok because these were not my scores. Roger at Lens Rentals measured the Sigma 30mm as sharper than the Zeiss 24mm at both centre and edges. Photozone.de measured the Sigma 30mm as sharper than the Zeiss 32mm at the edges but not the centre.
That is one of the main reasons I got the Sigma 30 f2.8, since it is just as good or better than lenses costing 3-5 times a much. Since I like both center and edge sharpness, the Sig 30 gives me both. It's also much smaller and lighter which is and additional bonus. Right now, the Sigma 30 f2.8 is the best bang for the buck lens for NEX/a5X00/a6X00 cameras! Besides that, both kit lenses are a close second, considering their size, weight, IQ and cost.
Interestingly, the ratings on B+H photo have the Sigma 30mm with an average of 4.62, the Zeiss 24mm at 4.71 and the Zeiss 32mm at 4.38. Of course, those ratings will partly reflect perceived value for money as well as raw performance.

The Amazon ratings are quite different, with the Sigma rated very low (average of 4.08), the Zeiss 24 at 4.69 and Zeiss 32 at 4.54.
I think, the more people pay for lenses, the higher they will score them on all websites. It would sure be nice if/when Sigma makes a great 24 mm f1.8/2.0 for ~$3-400. I'm sure Sigma can do it, I'm just not quite sure why they haven't yet?! I'm sure they would sell many thousands of them, especially if the reviews and images extremely close to the Zeiss 24mm.

BTW, if I want more 'Pop' in my images taken from my Sigma 30 f2.8, I can and will very simply give it to them during post processing. Lenses don't give 'Pop' to images, PP'ing does that ;-)
 
Where's the 18-105mm?
I have found only two websites with sharpness scores for the 18-105mm.

The first is ephotozine, which usually uses an NEX-5 for testing but used an NEX-7 for the 18-105, which makes the results not comparable. Annoying. They also used an NEX-7 for the 16-70mm. Of those two lenses, the 18-105mm was measured as quite a lot sharper, particularly wide open.

The other website with measurements for the 18-105mm is 'pcmag', which only has scores in the text, rather than charts or tables, and mostly they choose not to say much about them anyway. pcmag also measured the 16-70mm. Both were on an a5000. In this case, the 16-70 came out a lot sharper in the centre but the 18-105 was sharper at the edges.

If you know of any other websites with sharpness scores then please let me know!
If you could link to those, that would be awesome!
 
JohnNEX,

Thanks so much for doing this, I see it was a considerable effort. Bravo sir.

Maybe I missed something but where is the 16-70? Its probably the most talked about lens for the a6300 beyond the kit lens or the Sony nifty fifty.
 
JohnNEX,

Thanks so much for doing this, I see it was a considerable effort. Bravo sir.

Maybe I missed something but where is the 16-70? Its probably the most talked about lens for the a6300 beyond the kit lens or the Sony nifty fifty.
I have found only two websites with sharpness scores for the 16-70mm and the 18-105mm.

The first is ephotozine, which usually uses an NEX-5 for testing but used an NEX-7 for the 18-105, which makes the results not comparable. Annoying. They also used an NEX-7 for the 16-70mm. Of those two lenses, the 18-105mm was measured as quite a lot sharper, particularly wide open.

The other website with measurements for the 18-105mm is 'pcmag', which only has scores in the text, rather than charts or tables, and mostly they choose not to say much about them anyway. pcmag also measured the 16-70mm. Both were on an a5000. In this case, the 16-70 came out a lot sharper in the centre but the 18-105 was sharper at the edges.

If you know of any other websites with sharpness scores then please let me know!
 
JohnNEX,

Thanks so much for doing this, I see it was a considerable effort. Bravo sir.

Maybe I missed something but where is the 16-70? Its probably the most talked about lens for the a6300 beyond the kit lens or the Sony nifty fifty.
I have found only two websites with sharpness scores for the 16-70mm and the 18-105mm.

The first is ephotozine, which usually uses an NEX-5 for testing but used an NEX-7 for the 18-105, which makes the results not comparable. Annoying. They also used an NEX-7 for the 16-70mm. Of those two lenses, the 18-105mm was measured as quite a lot sharper, particularly wide open.

The other website with measurements for the 18-105mm is 'pcmag', which only has scores in the text, rather than charts or tables, and mostly they choose not to say much about them anyway. pcmag also measured the 16-70mm. Both were on an a5000. In this case, the 16-70 came out a lot sharper in the centre but the 18-105 was sharper at the edges.

If you know of any other websites with sharpness scores then please let me know!
There is the Photozone review of the 16-70


Rather woeful MTF scores and if you read the full review, it doesn't get any better
 
Lens rentals has also tested a bunch of Sony 35mm OSS lenses and averaged the scores. It's with the 32mm Touit review.

The 32mm Touit is best with respect to sharpness when compared to E35 and Sigma 30. E35 comes in a bit higher than the 30mm 2.8 sigma in the lens rentals test.


I found when I compared my kit lenses to my primes, they might have similar sharpness in the center but that area ends up being very small. Im not sure how useful comparing peak center sharpness is at different apertures /focal lengths. Personally, I don't freak out if a lens has softer corners but I do like my lenses to be very sharp at least half way to the edge at wide apertures.
 
JohnNEX,

Thanks so much for doing this, I see it was a considerable effort. Bravo sir.

Maybe I missed something but where is the 16-70? Its probably the most talked about lens for the a6300 beyond the kit lens or the Sony nifty fifty.
I have found only two websites with sharpness scores for the 16-70mm and the 18-105mm.

The first is ephotozine, which usually uses an NEX-5 for testing but used an NEX-7 for the 18-105, which makes the results not comparable. Annoying. They also used an NEX-7 for the 16-70mm. Of those two lenses, the 18-105mm was measured as quite a lot sharper, particularly wide open.

The other website with measurements for the 18-105mm is 'pcmag', which only has scores in the text, rather than charts or tables, and mostly they choose not to say much about them anyway. pcmag also measured the 16-70mm. Both were on an a5000. In this case, the 16-70 came out a lot sharper in the centre but the 18-105 was sharper at the edges.

If you know of any other websites with sharpness scores then please let me know!
There is the Photozone review of the 16-70

http://www.photozone.de/sony_nex/901-sony1670f4oss?start=1

Rather woeful MTF scores and if you read the full review, it doesn't get any better
Thanks, but this is a test done with a a6000 camera, whereas every other test with NEX lenses on the site is with a NEX-7, so its unfortunately not comparable.
 
There is the Photozone review of the 16-70

http://www.photozone.de/sony_nex/901-sony1670f4oss?start=1

Rather woeful MTF scores and if you read the full review, it doesn't get any better
If you can live with shooting at f5.6-f8 from 20 - ~60mm, and don't mind the cost, it would be a nice lens. That is of course, if your lucky enough to get one of the good copies.

If not, stick with the SEL1650 and get a Sigma 30 f1.4 (or f2.8) and 1-2 other good primes you may need for the money you will save. Photozone is not the only professional review site that has pretty much trashed the 16-70 lens, not only in accurate data, but in their conclusions.

Out of all the people who praise it, I've never yet seen a photo that proves why they love it so much. Especially around 16mm or 70mm at f4!
 
Thanks for that.

But, Mate, your going to get yourself in deep trouble by scoring [edges] the Sigma 30mm f/.28 DN A higher that either the Z Touit 32mm or the SCZ 24mm ZA :-O
I guess its ok because these were not my scores. Roger at Lens Rentals measured the Sigma 30mm as sharper than the Zeiss 24mm at both centre and edges. Photozone.de measured the Sigma 30mm as sharper than the Zeiss 32mm at the edges but not the centre.
That is one of the main reasons I got the Sigma 30 f2.8, since it is just as good or better than lenses costing 3-5 times a much.
For a while, the 30mm Sigma looked like the sharpest e-mount lens available. Amazing, considering that it was also one of the cheapest. It does have a couple of negatives -- AF isn't as good as most e-mount lenses (but usually fine), and bokeh is a bit odd, and some people would rather have something faster than f2.8, but overall, I think it's a great bang-for-the-buck choice. I rarely use it mostly because I'd rather have the convenience of a zoom, and partly because I prefer wider angles.
Since I like both center and edge sharpness, the Sig 30 gives me both. It's also much smaller and lighter which is and additional bonus. Right now, the Sigma 30 f2.8 is the best bang for the buck lens for NEX/a5X00/a6X00 cameras! Besides that, both kit lenses are a close second, considering their size, weight, IQ and cost.
Interestingly, the ratings on B+H photo have the Sigma 30mm with an average of 4.62, the Zeiss 24mm at 4.71 and the Zeiss 32mm at 4.38. Of course, those ratings will partly reflect perceived value for money as well as raw performance.

The Amazon ratings are quite different, with the Sigma rated very low (average of 4.08), the Zeiss 24 at 4.69 and Zeiss 32 at 4.54.
I think, the more people pay for lenses, the higher they will score them on all websites. It would sure be nice if/when Sigma makes a great 24 mm f1.8/2.0 for ~$3-400. I'm sure Sigma can do it, I'm just not quite sure why they haven't yet?! I'm sure they would sell many thousands of them, especially if the reviews and images extremely close to the Zeiss 24mm.

BTW, if I want more 'Pop' in my images taken from my Sigma 30 f2.8, I can and will very simply give it to them during post processing. Lenses don't give 'Pop' to images, PP'ing does that ;-)
Lenses can affect color, microcontrast, etc., and of course differences in bokeh, which could affect one's feeling of "pop". But, yeah, you can always add a bit of saturation, and give a color "pop" pretty easily in PP, if that's your thing. One problem is that "pop" can mean any number of things.
 
Hi John,

This is kind of neat in a geeky way :)

If you have interest in keeping up with this over time and trying to make it more useful, I have a couple recommendations:
  1. For each lens, I would try to get numbers for wide open and best f-stop (I think you're just doing "best", right ?)
  2. For zooms, I would try to get numbers for widest and longest setting.
This would get you up to 4 numbers per prime (center & edge wide open, center & edge best f-stop) and 8 per zoom (same as prime but for two focal lengths).

I've dabbled with this kind of thing for my own sense of relative performance of lenses, but never to the extent of trying to normalize ratings from various sources. Thanks for posting it.
 

Keyboard shortcuts

Back
Top