The problem with lens reviews: a rant

Started 3 months ago | Discussions thread
Flat view
Dibyendu Majumdar Regular Member • Posts: 429
The problem with lens reviews: a rant

There are a number of problems with lens reviews and comparisons published online by testers:

  • Comparing lenses that are not in the same price bracket
  • Comparing lenses that do not have the same focusing mechanism - AF vs manual focus
  • Comparing lenses that have VR versus that do not
  • Not taking into account the use case for which the lens is designed - applying a single computational metric to all lenses
  • Not testing a lens at different distances 
  • Comparing lenses that have different mount size / flange distance
  • Comparing lenses designed at different times (newer designs will always tend to be better than older designs - and a lens released later can obviously aim to be better than previously released lens by a competitor)
  • Not testing the lens in the real world by a competent photographer
  • Not taking into account sample variations
  • Not taking into account the 'status' of the lens being tested - is it new / fresh from manufacturing, could it have been damaged in shipping, is it a rented lens? 
  • Biases of the tester (everyone has a bias and this usually becomes apparent in little things such as the way something is worded) 
  • Rigour during testing 

Sadly there seems to be no way to overcome these issues. Here is a thought experiment:

  • Look at photographs taken by competent photographers
  • Anonymise the results and let 10 random people choose the photographs they like best

What brand do you think will win such a test?

So the question is how do you decide what lens to buy?

My suggestion is: don't go by tests published somewhere but try it out for yourself. Rent it maybe. See if it helps you take better pictures. But remember that in the end the skill and artistic sensibility of the photographer is more important in deciding the final outcome.

Flat view
Post (hide subjects) Posted by
MOD JimKasson
MOD JimKasson
MOD JimKasson
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark MMy threads
Color scheme? Blue / Yellow