Previous news story    Next news story

Coming soon: Lens Reviews to return to dpreview.com

By dpreview staff on Oct 2, 2012 at 16:00 GMT

We're incredibly pleased to announce the imminent return of lens reviews to dpreview.com thanks to a joint venture with DXO Labs, involving the establishment of a dedicated DxO lens and camera testing facility in Seattle, and the incorporation of DxO test data into dpreview.com’s class-leading lens reviews. We're currently ironing out the last few bugs in the system, and hope to publish the first review later this week.

We launched lens reviews back in 2008 and they were an immediate hit, gaining praise for the unique user-friendly presentation of complex data via a patented test data widget, supported by numerous real-world sample images and expert commentary. Logistical issues put the lens reviews on an extended hiatus in late 2010, and the new venture with DxO Labs not only ensures the return of lens reviews to dpreview.com, but allows more of them to be produced, more quickly.

We're currently putting the finishing touches to a completely re-written version of our lens widget, which will use DXO data and present it in a familiar format.

The new lab is now fully operational, and we expect to publish the first review within the next week or so. The popular lens review data widget has been entirely re-written to allow dpreview.com’s visitors to visualize the test results from the new lab and compare lenses just as they could before. Dpreview.com’s highly respected lens expert, Andy Westlake, will once again produce the lens reviews.

As part of the joint venture agreement the test results obtained will also be made available on DxO Mark website (www.dxomark.com). DxO Labs and dpreview.com will also be collaborating on the testing of digital cameras with a view to adding even more valuable image quality information to dpreview.com’s legendary in-depth reviews.

'We are very happy to provide dpreview with our measurement technology for testing cameras and lenses' said DxO Labs CEO Jerome Meniere. 'Dpreview’s articulate and creative writing style makes difficult photography concepts accessible to even the most novices of photographers – they are a perfect complement to DxOMark’s scientific measures.'

Simon: Producing the data for lens reviews in an incredibly long-winded process requiring a large, dedicated studio and hundreds, sometimes thousands of high precision exposures and measurements. The establishment of a dedicated DxO Labs lens and camera testing facility on our doorstep allows us to entrust the measurement and studio testing of lenses to an established world leader in image quality analysis, and to work with its team to offer our readers the perfect combination of accurate, consistent measurements with real-world shooting experience and expert analysis.

Comments

Total comments: 275
123
Lee Jay
By Lee Jay (Oct 3, 2012)

By the way, in my opinion, lens resolution tests are all entirely invalid if they are shot through an AA filter.

1 upvote
Simon Joinson
By Simon Joinson (Oct 3, 2012)

In which case all our lens reviews are, in your opinion, invalid, and will continue to be so when we relaunch them.

6 upvotes
Der Steppenwolf
By Der Steppenwolf (Oct 3, 2012)

Most cameras have a AA filter and lenses as such are meant to be used with one.
They should be tested in the same maner they will be used.

5 upvotes
photo nuts
By photo nuts (Oct 3, 2012)

@ljfinger: This only means we cannot compare lens performances across different platforms. But each lens test on its own can still reveal important and relevant clues such as edge vs center performance, vignetting, distortion...

Secondly, just like DXOMark sensor data, lens performance data per se without attachment to actual cameras is useless to real world users.

1 upvote
Lee Jay
By Lee Jay (Oct 3, 2012)

"This only means we cannot compare lens performances across different platforms."

And across different cameras on the same platform. Really it means you have to test every lens on every camera it could be used on. Given adapters and cross-platform use, this makes the number of permutations and combinations nearly endless.

0 upvotes
Lee Jay
By Lee Jay (Oct 3, 2012)

"In which case all our lens reviews are, in your opinion, invalid, and will continue to be so when we relaunch them."

That's correct. However, this situation is needless given the ease at which you could procure cameras with the AA filters removed.

0 upvotes
MichaelEchos
By MichaelEchos (Oct 4, 2012)

The only valid lens resolution tests are the one with infinite pixels, by your logic. A 15MP camera is still gonna get 15MP even if it loses its AA filter, only that the one with AA filters are slightly worse. A conventional 35MP camera is still going to beat a 15MP sensor without an AA filter in terms of resolution.

0 upvotes
Lee Jay
By Lee Jay (Oct 5, 2012)

"A 15MP camera is still gonna get 15MP even if it loses its AA filter,..."

Not necessarily.
The IMATEST approach uses an oversampling method, which can therefore examine spacial frequencies accurately far above the Nyquist frequency, if the AA filter is not present.

http://www.imatest.com/docs/sharpness/

"Briefly, the slanted edge method calculates MTF by finding the average edge (4X oversampled using a clever binning algorithm), differentiating it (this is the Line Spread Function (LSF)), then taking the absolute value of the fourier transform of the LSF. The edge is slanted so the average is derived from a distribution of sampling phases (relationships between the edge and pixel locations). The algorithm is described in detail here."

"The four bins are combined to calculate an averaged 4x oversampled edge. This allows analysis of spatial frequencies beyond the normal Nyquist frequency."

0 upvotes
Lee Jay
By Lee Jay (Oct 3, 2012)

Isn't this what SLRGear already provides?

1 upvote
Simon Joinson
By Simon Joinson (Oct 3, 2012)

no

2 upvotes
TN Args
By TN Args (Oct 3, 2012)

slrgear leave too many lenses untested. Of the last 50 lenses entered on their database, stretching back to January, only 13 have been tested.

Even many classic lenses are untested by slrgear; such large gaps weaken the site's usefulness. e.g. the Canon 300mm f4 IS has never been tested by them.

Comment edited 15 seconds after posting
1 upvote
mikeumo
By mikeumo (Oct 3, 2012)

Dxo provides the best public measurement data on cameras and lenses. Hopefully this move will help both companies to publish test results and reviews faster.

1 upvote
peevee1
By peevee1 (Oct 3, 2012)

Given how little was published on DxOMark recently, they probably lost their staff in France?

3 upvotes
Sosua
By Sosua (Oct 3, 2012)

Rah rah rah DXO rah rah rah, said my camera sensor was bad, rah rah rah I don't know how to interpret test data.

5 upvotes
MichaelEchos
By MichaelEchos (Oct 4, 2012)

Do note that DxOMark is testing raw RAW data, not those which had noise reduction applied. The reason you might not agree with (say that you are a Canon user) the test is because Canon applies a stronger noise reduction than Nikon for its RAW files. And I'm guessing, one big reason might be because Canon has very efficient noise reduction in comparison to Nikon. But I'd still prefer a RAW file without any noise reduction at all, so that I can reduce it on computer, with greatly higher processing power.

0 upvotes
Peiasdf
By Peiasdf (Oct 3, 2012)

Is it possible for DP to have a comparison page for lenses? Like allowing four MTF graphs for a particular camera?

3 upvotes
Octane
By Octane (Oct 3, 2012)

That is awesome

0 upvotes
Abhijith Kannankavil
By Abhijith Kannankavil (Oct 3, 2012)

That's great news. Hope it'll be more usefull than just numbers and statistics from DXO.

0 upvotes
creeker
By creeker (Oct 3, 2012)

I have found the DXO reviews to be pretty close to how my lens perform. Many times I hear people say " in real life photos" the lens is fine. Bull. It's fine if you're happy with snapshot quality images. The DXO tests along with SLRgear tests have shown me the weak points of even my best lens. I stay away from those areas for better images. Some people just don't want to hear that their lens is a dog. I glad to hear dpreview is teaming with DXO, and I'm looking forward to the first tests.
Ed

1 upvote
Gesture
By Gesture (Oct 3, 2012)

All these months absent would have been a good time to re-think how to test lenses and what to include by way of technical and photographic evaluation. I hope this is what has been accomplished.

2 upvotes
ahoeflak
By ahoeflak (Oct 3, 2012)

Great news!

0 upvotes
jenbenn
By jenbenn (Oct 3, 2012)

DXo lens test do not conform to real world experience. Just check their test of the Canon 16-35 II lens. They claim the corners at 16mm are sharper at f/2.8 than at f/5.6. Thats obviously rubbish. The lens (like most, if not all, non-diffraction limited lenses) sharpens up when stopped down. (No wonder photozone, slrgear and a dozen other test sites unanimously disagree with DOX on this point). Of course the lens doesnt get real sharp in the corners at 16mm but its sharper at f/5.6 than at f/2.8!!!

Repeated, polite request to DXO to explain their weird findings have been answered with ignorance and silence. Thus, I must conclude that DXo has not made a genuine mistake or used a bad sample but considers its measurement true and accurate. Something they obviuoly wouldnt, had they even taken the lens out of the lab and snapped a few shots or compared their results to other tests.

So please dpreview, assure that lenses are tested in the wild, on 3D subject by real photographers.

5 upvotes
Simon Joinson
By Simon Joinson (Oct 3, 2012)

we will not publish any data we cannot verify, and we will be working with more than one copy of each lens. Andy will be working closely with DXO to ensure his findings in the field match theirs in the lab. This will improve both reviews.

11 upvotes
MPA1
By MPA1 (Oct 2, 2012)

I hope you'll be asking Ken Rockwell in as a Guest Reviewer. ;-)

4 upvotes
SM7
By SM7 (Oct 3, 2012)

After the Amazon purchase, he has been superior to DPR. In particular, cause DPR has a very limited lens review section... Glad to hear that lens reviews will be back, and perhaps less book reviews?

0 upvotes
Absolutic
By Absolutic (Oct 3, 2012)

wow, thank you MPA1 for a dose of laughter this morning

0 upvotes
Cobber55
By Cobber55 (Oct 2, 2012)

This is indeed good news. I think this collaboration can only be positive. I've been quite impressed with my recently purchased DXO Optics Pro 7.0 Standard program. They seem to know their trade and if this helps dpreview bring back lens reviews then you will have no complaint from me.

2 upvotes
Scales USA
By Scales USA (Oct 2, 2012)

I am a bit concerned. Some of their conclusions regarding lenses just do not seem to be valid.

Like all others, they test lenses mounted to cameras, and then, only a small sample. The lens to lens variation and camera to camera variation leaves me wanting a real test of a lens. I am interested in seeing more information about their testing, but the single resolution figure for a lens on their site leavs me wondering.

I am hoping that they obtain the technology to first properly adjust a lens before testing it, as well as test a significant sampling. Otherwise, its just one data point, and not worth making a big deal out of it.

2 upvotes
paulwright
By paulwright (Oct 2, 2012)

Absolutely. I quit reading the DxO reviews some time ago as their highly technical, narrow field findings were frequently quite different from real-world reality. As I've said elsewhere, the feedback from hard working photographers who actually use the lenses is review "gold-standard".

5 upvotes
Simon Joinson
By Simon Joinson (Oct 3, 2012)

we are simply using their test data. Go look at any of our lens reviews from a couple of years ago. Now imagine them pretty much the same, but with a DXO logo on the lens test data widget. That's what to expect.

7 upvotes
SM7
By SM7 (Oct 3, 2012)

It all depends on how much DXO is paying you guys... I hope it's only for the logo, and not the content. I really do.

2 upvotes
Simon Joinson
By Simon Joinson (Oct 3, 2012)

why would DXO be paying us?

6 upvotes
Scales USA
By Scales USA (Oct 3, 2012)

I'm certainly reserving any opinion until I have a chance to see a few reviews and compare the results with other lens testers. I'm glad to see that your widget will be used, and that DXO will be testing more than one sample - 10 or more from different lots would be nice, but, as long as samples are not from tha same production lot, its better than just one. Also happy that someone will provide expert commentary and a real world test of reasonableness. That is lacking in the current setup.

0 upvotes
Gothmoth
By Gothmoth (Oct 2, 2012)

DXO the company who rates a EF 85mm f1.8 better then any other lens?

that´s a bad joke.. you may think about their sensor test what you like. normalizing bull... will most think i guess.

but their lens tests are completely out of this world and have NOTHING TO DO wit real life performance.

4 upvotes
Gary Dean Mercer Clark
By Gary Dean Mercer Clark (Oct 2, 2012)

I like this development. I use DXO software extensively for my photography and love it. Their testing is excellent as well and spot on. Its a win situation for Dpreview.com and DXO working together like this. Great news!

3 upvotes
HB1969
By HB1969 (Oct 2, 2012)

Funny, I was about to make a comment in the forums on how there haven't been any new lens reviews lately and then I read this :)
Nice. Looking forward to its return.

2 upvotes
Mikael Risedal
By Mikael Risedal (Oct 2, 2012)

why not a real MTF tests of lenses

Comment edited 2 minutes after posting
5 upvotes
roustabout66
By roustabout66 (Oct 2, 2012)

It would appear that the censors did not approve of my criticism of DXO results in my previous post so I will simply say I am not a fan of DXO testing. Hopefully this post sticks.

7 upvotes
Barney Britton
By Barney Britton (Oct 2, 2012)

Noted.

4 upvotes
SM7
By SM7 (Oct 3, 2012)

Censors?

0 upvotes
roustabout66
By roustabout66 (Oct 3, 2012)

Yes, I have had 2 posts deleted in the past 3 hours.

0 upvotes
Bjorn_L
By Bjorn_L (Oct 2, 2012)

Great news looking forward to the expanded lens data and in particular the inclusion of the reliable DXO data.

1 upvote
HBowman
By HBowman (Oct 2, 2012)

I do not like DxO but I trust DPR using it in the right way and making the right conclusions. Sure some conclusions will be different than DxO own sites :D

6 upvotes
SirSeth
By SirSeth (Oct 3, 2012)

Thank you HBowman. Yours is a pretty balanced perspective; not as reactionary as I might have felt like being before reading your comment. I do hope DPR takes note that many of their loyal readers consider DxO data to reveal less about real life photography than a washed up psychic with a cracked crystal ball. Ok, so a little hyperbole there. I do trust DPR to handle the data well. They are generally very good at that.

1 upvote
HBowman
By HBowman (Oct 3, 2012)

Yea I'm a little bit reactionary :) Defending SIGMA technology world wild is a bit frustrating along the years ;)

0 upvotes
SirSeth
By SirSeth (Oct 3, 2012)

Does that mean you don't even bother trying to defend Sigma's pricing strategy? ;) Really though, it's good to allow others their opinions without feeling attacked or incensed.

0 upvotes
HBowman
By HBowman (Oct 3, 2012)

Ho yea I defend SIGMA pricing strategy. For me, with the actual performances of lense like 50f1.4, 85f1.4, 120-300f2.8 ... I have nothing more to defend than the facts. They are excellent optics ;)

Yes It good to allow others opinion but only if the opinion is stated. Many ppl criticize things they do not use. I do not allow that ;)

0 upvotes
jj74e
By jj74e (Oct 2, 2012)

"legendary in-depth reviews"

lol. not that i'm saying dpreview's reviews aren't good. but legendary? eh...

10 upvotes
Dafffid
By Dafffid (Oct 2, 2012)

Legendary - like dragons and fairies, we've heard of them but nobody's seen one for years

19 upvotes
SM7
By SM7 (Oct 3, 2012)

:D

0 upvotes
photo nuts
By photo nuts (Oct 2, 2012)

DXO lens reviews have so far been highly disappointing in terms of accuracy. Actually, most of their lens-camera combo data has been nothing but rubbish. Hope this does not spill into the new collaborative effort.

11 upvotes
ethanolson
By ethanolson (Oct 2, 2012)

I was looking at their site recently and saw some of my favorite Nikkor lenses at the top of the list, and being a Nikon fan, I was quite pleased.

Then I looked at the ratings by specific criteria that I felt mattered most and I kept seeing Canon and Zeiss at the top of the list.

I concluded that their testing is as you suggest: slanted by the camera the lens is mounted to. They need to get that out of the equation or they can't be taken seriously.

0 upvotes
Gothmoth
By Gothmoth (Oct 2, 2012)

so what are you saing.. DXO tests are fine as long as your nikon stuff is on top?

or you saing the nikon lenses are rated better then they should because of the latest sony sensor improvements?

Comment edited 2 times, last edit 1 minute after posting
1 upvote
DavidBoulder
By DavidBoulder (Oct 3, 2012)

"I concluded that their testing is as you suggest: slanted by the camera the lens is mounted to. They need to get that out of the equation or they can't be taken seriously"

More and more I expect lens design to relate to specific sensor design. For example a typical DSLR setup suffers at the edge because light hits sensor at an angle. 4/3rds hits parallel. Some sensors may be designed to deal with specific characteristics of lenses signed alongside th camera.

0 upvotes
StevenE
By StevenE (Oct 2, 2012)

That's great! It's hard to find really good lens reviews, especially on new lenses.

0 upvotes
Artpt
By Artpt (Oct 2, 2012)

I have to say this is a great resource site for enthusiasts like me. It is free, that is without cost and without obligation. The information is rather fast. It is still difficult to read through some of the more visceral, knee jerk comments...but some are very useful indeed. I hope to submit to one of the photo challenges soon.

0 upvotes
VA-ArtG
By VA-ArtG (Oct 2, 2012)

Hats off to you! It's a lot of work , but it is appreciated. Thanks

1 upvote
Esa Tuunanen
By Esa Tuunanen (Oct 2, 2012)

Just keep those dumbed down DxOMarketing scores out of site. Those shouldn't be available for people whose knowledge is latest marketing campaign brochure deep.

Same for cameras, DxO tests noise in fully synthetic benchmark/theoretical way which shouldn't belong to PHOTOGRAPHY site.
Even shooting resolution test charts at various ISOs and looking effects of noise in it would be almost closer to actual photography.
Your JPEG/RAW image comparison tool is lot more usefull.
Actual printing test of image at various sizes like done by Imaging resource would be also more usefull addition to camera reviews.

10 upvotes
photo nuts
By photo nuts (Oct 2, 2012)

Totally agreed. If DPReview starts to function like DXOMark, it's time for me to stop visiting the site.

11 upvotes
roustabout66
By roustabout66 (Oct 3, 2012)

To quote Simon "we'll be working with DXO on enhancing camera reviewing in the near future - watch this space."

I think that explains a lot.

0 upvotes
Photato
By Photato (Oct 2, 2012)

I still remember when Phil Askey blasted DxO for lacking RAW support in their software. That was before hiring Simon.
Things have come a long way.

Comment edited 2 times, last edit 1 minute after posting
6 upvotes
ChrisKramer1
By ChrisKramer1 (Oct 2, 2012)

This is really great news (and not before time too).This almost makes me forgive DPR for reviewing the Phone 5...

8 upvotes
Barney Britton
By Barney Britton (Oct 2, 2012)

Open your heart. You'll find the strength to forgive.

5 upvotes
SM7
By SM7 (Oct 3, 2012)

Barney -- this is the INTERNET :)

1 upvote
facedodge
By facedodge (Oct 2, 2012)

I like to see an emphasis go on the new lenses, rather than playing catch-up.

Referring to Canon, .. I'd rather see the new 24-70 II or 200-400 f/4L take precedent over older lenses that have been out for a while.

0 upvotes
EssexAsh
By EssexAsh (Oct 2, 2012)

about time! there's a lot of holes in the nikon lens database.

1 upvote
Annex
By Annex (Oct 2, 2012)

Brilliant news. Lets hope the photo character analysis is as good as the technical is likely to be (give the DxO focus).

0 upvotes
Erik Magnuson
By Erik Magnuson (Oct 2, 2012)

You mention using DxO data - does this mean that you will have widget comparison data for all (or at least some) lenses DxO has previously tested in addition to any new ones that you test? And/or that some reviews will combine both, i.e. lab data from DxO in France combined with commentary on handling/build/usability from dpreview? (And vice-versa - data from your lab can be used by DxO to produce lens modules so that any reviewed lens should be supported by DxO quickly.)

1 upvote
Aaron MC
By Aaron MC (Oct 2, 2012)

Awesome.

0 upvotes
Chaitanya S
By Chaitanya S (Oct 2, 2012)

Great news, its a good step in right direction.

1 upvote
WilliamJ
By WilliamJ (Oct 2, 2012)

Agreed !

0 upvotes
Absolutic
By Absolutic (Oct 2, 2012)

Can you guys, or Andy in particular, list some of the lenses, that he intends to review first? You have done it before with a list of cameras in the pipeline for review, so I don't see why not give us a preview or at least a list of those you contemplate on reviewing, or those in progress. Yes, I know that it might cause people to yell, "review this one too please.." but still?

1 upvote
WilliamJ
By WilliamJ (Oct 2, 2012)

Anything is better than nothing anyway.

Comment edited 1 minute after posting
0 upvotes
Absolutic
By Absolutic (Oct 2, 2012)

I know I am waiting for EF 24-70 F/2.8 II because the reviews for that new marvel have differed, to say the least. ... from photozone to lensrentals

Comment edited 2 minutes after posting
2 upvotes
Frozenshutter
By Frozenshutter (Oct 2, 2012)

This is a great point. Amateur lenses seem to be highly variable, and one review can differ greatly from another. Despite the additional work involved, it would be good to review two randomly chosen samples of a give optic - even better if their serial numbers aren't close together.

2 upvotes
JohnMatrix
By JohnMatrix (Oct 2, 2012)

@ frozenshutter You mean "amateur lenses" like Canon's $2300 24-70mm II for example? :)

1 upvote
Steven Rounds
By Steven Rounds (Oct 2, 2012)

Interestingly, I made a comment about an hour ago about how dxomark.com sat on their results for the E-M5 camera sensor for months, without giving any reasonable reason for the delay. Why was my comment expunged? I'd love for dpreview.com to address the issue...

2 upvotes
tkbslc
By tkbslc (Oct 2, 2012)

Do they need a reason?

1 upvote
Steven Rounds
By Steven Rounds (Oct 2, 2012)

Sure they do...if we're questioning whether dxomark is impartial, they need to clear the issue up.

1 upvote
noirdesir
By noirdesir (Oct 2, 2012)

Their results are impartial. That is what matters. If you want to see a conspiracy in everything, fine, whatever makes you happy.

How do we know their results are impartial? Because they make sense. They are consistent with most other available data, they reflect a very reasonable technological progress, they show similarities between sensors from the same design team etc.

I really don't understand why so many people see foul play whenever something deviates from a presumed ideal behaviour. Is it because they constantly have first hand experience of constant foul play?

4 upvotes
Esa Tuunanen
By Esa Tuunanen (Oct 2, 2012)

Yes, DxO's E-M5 results are entirely consistent for the size of pixel and modern sensor tech from Sony.
So why did they bury those results for this long, noirdesir?

If it looks like crap and smells like crap there aren't many logical conclusions.

2 upvotes
robert1955
By robert1955 (Oct 2, 2012)

Maybe their original results were not consistent with what might be expected. There is no reasonabl explanation why their would be a plot

1 upvote
Absolutic
By Absolutic (Oct 2, 2012)

Steven, it is easy, everyone knows DXO results always favor Nikon or Sony cameras, they always win in everything everytime. Canon always loses. M43 typically loses big too. Nikon and Sony always win

0 upvotes
wootpile
By wootpile (Oct 2, 2012)

Excellent Dpreview! Finally a move in the right direction!

0 upvotes
Barney Britton
By Barney Britton (Oct 2, 2012)

'finally'?

4 upvotes
Absolutic
By Absolutic (Oct 2, 2012)

so you guys in Dpreview are buying into DXO's analysis I gather???

0 upvotes
Simon Joinson
By Simon Joinson (Oct 2, 2012)

actually we're using their test data, which is subtly, but crucially different

11 upvotes
pdcm
By pdcm (Oct 2, 2012)

And about time too. But I have to question whether DXO are the best possible choice; they don't really go into enough depth.

1 upvote
WilliamJ
By WilliamJ (Oct 2, 2012)

Maybe you could tell us what should be done, and then DxO could see how to improve their way of doing things according to your suggestions. Could you give us an insight of what should be done for instance (something that DxO doesn't do yet of course) ?

2 upvotes
hansolo82
By hansolo82 (Oct 2, 2012)

Great news IF you don't use their mystical "DxOMark Score" and put that in the front page of your site (i.e. meaningless because using only one data point on the overall test, and that's for lenses AND sensors). Anyone who digs a bit deeper in their tests (and other's) will soon see that it takes more than one number to caracterize a lens (especially zoom) or a sensor. Take the canon 70-200mm f/2.8 I & II for reference.

That's why my No1 reference for lenses is now Roger Cicala's lensrental blog because :
1 - he tests as many copies as he can because he knows sample variation is important, even for a 50 000$ cinema lenses.

2 - tries to test at different focal length

3 - Gives the (average) resolution for the center and for an average over the entire coverage (although he should also gives the maximum and minimum he got in a batch)

Just unfortunate that he does really have the time to test everything he got (but at least seems to test everything that's new).

12 upvotes
Simon Joinson
By Simon Joinson (Oct 2, 2012)

we will be publishing our own conclusions

6 upvotes
j900
By j900 (Oct 2, 2012)

The "one number magical score" is meant for people in a hurry. For all the others DxO provides data which is detailed enough to carry out a finer analysis.

1 upvote
Absolutic
By Absolutic (Oct 2, 2012)

Totally agree, and that is why I always review Roger's tests. Dpreview, photozone, slrgear, their reviews are usually based on one sample they get and one sample only. So if they get a bad sample, there are far going consequences. Roger always tests 5 or 10 samples which is much more representative

2 upvotes
Barney Britton
By Barney Britton (Oct 2, 2012)

We test multiple samples when necessary.

6 upvotes
neo_nights
By neo_nights (Oct 2, 2012)

There's one fantastic article that Roger wrote about lens variation and it seems some people have missed it. He states that, although THERE IS sample variation, the differences are almost negligeable for 'real world shooting'. One BAD sample is BAD and it's easily spot (and it's not that common). If DPReview was to find a BAD sample, they'd be clever enough to ask for another sample. Also there's a thing called 'camera variation' (that may affect the lens). Or review's sites should try also many copies of each camera with each lens?
So, stop complaining.

3 upvotes
hansolo82
By hansolo82 (Oct 2, 2012)

Well if you use their test but produce your own conclusion then I'm looking forward to this. Even if you will probably do only one copy of a lens, that's still one more tested by a professional site, so one more opinion.

And to reply to j900, to say that their score is just to give a quick look at the lens/sensor (if that is the real reason) is just lame considering that when you do dig deeper into the review (or compare to other's), you realize that their score means absolutly nothing. Then why would I use it if I'm in a hurry! It's completely misleading.

I just think their score system is inappropriate and they should revise it entirely...

1 upvote
hansolo82
By hansolo82 (Oct 2, 2012)

"We test multiple samples when necessary."

This is getting even better!

neo_nigths : Check out it's latest post : http://www.lensrentals.com/blog/2012/09/why-arent-the-damn-numbers-the-same

While I agree that a really bad copy will be easily recognized and relatively rare, and would probably not be included in a profesionnal review, the sample variation is still very important.
I mean the best 3 copies of 24-70 I is almost as good overally (and better in center) as the worst 3 copies of the 70-200 II. And the variation is pretty wide even for the 70-200II (+- 10% in MTF) Which is a lot when you put out a score value of only one copy

I know that in real world shooting you wouldn't probably not notice a +-10% difference in MTF but when a "professional" website which makes a review of a lens with a score entirely based on resolution it does. Otherwise you end up with a 70-200I better than a 70-200II...

1 upvote
AbrasiveReducer
By AbrasiveReducer (Oct 2, 2012)

It sounds simple--when you get a lens that's a dog, try another one. But this can be a slippery slope. How many do you try and how many does the consumer have to try? Or, do you keep trying until you find a good one and then conclude the others were "aberrations"?

There seem to be two fundamental schools of lens testing. The "impress me" approach of guys like Photozone de and at the other extreme, the "If you can't say anything nice, don't say anything" approach, popularized by Popular Photography and Photoshop User.

Comment edited 2 times, last edit 7 minutes after posting
1 upvote
Joseph S Wisniewski
By Joseph S Wisniewski (Oct 2, 2012)

I've missed Len. He was my favorite of all the reviewers at dpReview, although Simon and Andy are pretty cool, too.

2 upvotes
Richard Butler
By Richard Butler (Oct 2, 2012)

We've not left an apostrophe lying around, have we?

8 upvotes
larrytusaz
By larrytusaz (Oct 2, 2012)

Well I have to give you credit. Now THIS is the DPreview I've visited and loved for many years.Reviews about real gear, for real photographers--hobbyist, professional, whatever, but PHOTOGRAPHERS.

If I gripe about the iPhone articles, I have to be consistent & holler out some praise for times like this, times when you remind us what we've been coming here for years to see and read about.

Right on. +1. Thumbs up. Whatever you want to call it, I TOTALLY and 100% approve.

11 upvotes
WilliamJ
By WilliamJ (Oct 2, 2012)

I can't tell it better !

0 upvotes
BayAreaWZ
By BayAreaWZ (Oct 2, 2012)

YAY Andy! I thought his lens reviews were the best so very happy to see that is coming back.

2 upvotes
neo_nights
By neo_nights (Oct 2, 2012)

Some people are asking for more Canon and Nikon reviews and then a comparison between them. What's the point? Bragging rights? I know there are some people who shoot both, but I don't think it's a majority.
It'd make HEAPS more sense comparing "first party" lenses (Sony, Canon, Nikon, Pentax, Oly and so) with their third party counterparts.

7 upvotes
WilliamJ
By WilliamJ (Oct 2, 2012)

Don't be affraid my friend ! Nowadays, there are a lot more to review than just "classic" lens, and I guess the Olympus, Pentax, Panasonic/Leica, Fujifilm, Sony/Zeiss, Tamron, Sigma ones will have their share of the future reviews.

1 upvote
Frozenshutter
By Frozenshutter (Oct 2, 2012)

The timing of this reappearance of lens reviews is great, as we're entering a golden age of lens manufacture, with the traditional makers and newbies alike jumping in with some very interesting glass. Different topic - are you considering any ways to gauge public interest and demand for particular reviews?

1 upvote
EmmanuelStarchild
By EmmanuelStarchild (Oct 2, 2012)

I was one of those posters asking for these test comparisons, mainly because I have outgrown my entry level Canon 550D with kit lens combo(18-55, 55-250, 50 1.8) and am looking for a more advance system of Camera w/lenses. I have not decided just yet whether I will go with Canon or Nikon. The quality of lenses will determine largely which way I will lean. I don't care much about bragging rights, I just want a thorough and complete analysis before I start spending thousands of dollars.

0 upvotes
neo_nights
By neo_nights (Oct 3, 2012)

@Emmanuel: I see your point. But the tops lenses of all manufacturers are about the same. Ok, the Zeiss (Sony) 24-70mm F2.8 is regarded as the best in its class. But does it make the others (Nikon and Canon) bad lenses? No! Also, I *believe* that other popular lenses (i.e. 18-135/18-105 F3.5-F5.6) are about the same as well.

0 upvotes
jkrumm
By jkrumm (Oct 2, 2012)

One strange thing about Dxo lens tests, unlike lenstip and slrgear, they rate 4/3 and m43 lenses from poor to middling, at least in resolution. Not sure what the reason is. Maybe they are not taking sensor size into account? Will be interesting to see what they get out of the new Oly 75 1.8, which appears to be about as flawless as lenses get.

2 upvotes
Andy Westlake
By Andy Westlake (Oct 2, 2012)

DxOMark's lens scoring gives lower marks to Micro Four Thirds simply because it's a system score that does effectively take sensor size into account (and considers larger sensors to be better). The explanation is on their website.

3 upvotes
Mssimo
By Mssimo (Oct 2, 2012)

Its based on how large of a print you can get out of a image. The problem is how they format it. If you could only filter by the print size. Same issue with the sensor rating, you should be able to rank the camera based on whats important to you. DXO data is good but the use of it is not. Maybe DPreview can help them out with that.

3 upvotes
villebon
By villebon (Oct 2, 2012)

And so does photozone.de!
-
M4/3 uses in camera processing to overcome lens shortcoming. They were probably the frist to do so.

0 upvotes
Richard Butler
By Richard Butler (Oct 2, 2012)

That's not strictly true - you could argue that the Micro Four Thirds approach isn't about overcoming shortcomings, it's about using software as an aspect of the lens' design.

3 upvotes
jkrumm
By jkrumm (Oct 2, 2012)

So in essence they aren't rating lens performance, they are rating lens+sensor performance. I think the main problem is the way they present resolution, with a graph from "poor" to "excellent." It's a little misleading, since it implies that one lens is less sharp than another, regardless of sensor. And I suppose if they ever tested medium or large format cameras, they'd have to invent a new graph that goes beyond excellent.

2 upvotes
RichardAB
By RichardAB (Oct 2, 2012)

Will the new lens testing include fixed lenses too, such as on the Sony RX1, Sony RX100, Canon G1X, Fufi X100, Fufi X10?

2 upvotes
Andy Crowe
By Andy Crowe (Oct 2, 2012)

Will the new lens test data be directly comparable to the older lens test data, or will the new lab affect the sharpness scores?

1 upvote
Andy Westlake
By Andy Westlake (Oct 2, 2012)

It'll be roughly comparable, in principle, but it won't be presented in the same widget and we won't be allowing direct comparison. Instead the aim is to include properly-comparable data for similar lenses to whatever's being reviewed.

1 upvote
Anfy
By Anfy (Oct 2, 2012)

There's a simple solution: retest the more significant among the older lenses... ;-)

Comment edited 12 seconds after posting
2 upvotes
Absolutic
By Absolutic (Oct 2, 2012)

Oh Nooooo, I loved the Widget!!!! The Widget was unique and I could actually understand it; Don't kill the Widget!

5 upvotes
desmond2046
By desmond2046 (Oct 2, 2012)

dpreview should acquire DxO Lab to make the all-in-one professional website for photography.

1 upvote
Total comments: 275
123