Roger Cicala compares three 24-70mm F2.8 lenses

They clearly state that they wanted to compare lens+body combinations, not just the lens. A Canon+Nikon combination does not make much sense to test, since no one really uses something like that.

Well yes. With camera systems the major investment is with the lenses. Bodies come and go. But most pros spend 5 x the money on a lens setup than they do with the body. That's why it was with a heavy heart the many major picture agencies and freelancers gave up all their great Cannon lenses to jump to Nikon after Cannon failed to deliver a decent body for years. EOS 1D, mark 2, mark3 all had issues and needed constant firmware upgrade etc....

If you haven't yet bought in to a system this would go some way to helping but what's the point in having a great lens from Cannon if the bodies have issues?

Yes I think Canon has been upgrading their lenses in advance of higher res bodies.

You're missing, or ignoring, one of the key points of this article. Both Lenstip and Photozone test lenses on camera bodies like almost everyone else, including DxO (for both DxOMark and our reviews) and SLRGear. In system tests the camera's anti-aliasing filter becomes a limiting factor in what can be measured.

What Roger has done in the first part of his comparison is take the lenses off the cameras and put them on a test bench to measure the MTF directly. This is pretty crucial to the overall conclusion - that the Canon is demonstrably sharper than the Nikon in these tests, but this advantage is entirely negated when you put the lenses on the respective manufacturers highest-resolution bodies.

Your 'fanboyism' is perplexing, exasperating and entertaining, all at the same time. :)

Seriously? Check DXOMark test, the most creditable test that I believe.

http://www.dxomark.com/index.php/Publications/DxOMark-Reviews/Canon-EF24-70mm-f-2.8L-II-USM-A-Peerless-Performer/Comparisons

Canon copy is better in sharpness especially in edges/corners and significantly better in CA (more than double). Check Measurements | Sharpness | FieldMap. Canon copy is better in every major focus stop.

DXOMark concluded that "Canon EF24-70mm f/2.8L II USM: A Peerless Performer", period.

According to DXOMark, Canon sensors are the worst.

My point is testing a 24-70 zoom lens at only 50mm, one focal length, does not tell the whole story. When Lenstip tested the 24-70 Nikkor, it was the highest resolving zoom they ever tested at 24mm. But again, that's not the whole story.

Photozone also demonstrated that the Canon was softer than the previous version at 70mm. But the point is MTFs at one focal length of a zoom cannot inform us about overall optical superiority or inferiority of a that lens . To say the EF 24-70 2.8 L II is the best standard zoom based on such a "sort-of" test is simply not possible. Distortion, vignetting, MTF, coma, astigmatism, CA, are all necessary in evaluating optics.

If MTFs are all that interest LensRentals, he should have at least tested 24mm and 70mm, not just 50mm.

@marike6 - so the 24-70 II has less CA, less distortion, and is sharper than the 24-70 nikkor. and according to photozone.de .. the nikkor 24-70 fell off a bit at 70mm as well. it just turned out that canon prioritized 24mm as the focal to be the best at.

of course i'm sure you will tell me that this shows the nikon to have the same or better optical characteristics..

http://the-digital-picture.com/Reviews/ISO-12233-Sample-Crops.aspx?Lens=618&Camera=614&Sample=0&FLI=0&API=0&LensComp=787&CameraComp=0&FLIComp=0&APIComp=0

considering the vast majority of people will want a standard zoom to be the best corner to corner at 24mm then at 70mm I can appreciate canon's design decision far more than you hanging everything on 70mm which is about the on focal that the nikkor looks better at.

the new Canon is amazing but Nikon users having been using a top quality lens for more than five years and they should expect a better one in five years.

I doubt that. But I do know that Canon users have been buying the 14-24 2.8 Nikkor, by far the best UWA zoom, for years.

Besides, there's more to testing a standard zoom than just resolution at 50mm. But if you insist, the new Canon does worse at 24mm and 70mm than the Nikkor.

@ fanboy marike6:
On the contrary, Nikon 24-70 f/2.8 has WORSE performance than Canon 24-70 f/2.8 at 24 mm. When normalized to center resolution @ f/2.8,
Nikon edge:center resolution = 0.68
Canon edge:center resolution = 0.74

What can I say? Your fanboyism is exasperating, perplexing and downright ridiculous. :)

Please, Canon shooters are drying up the Pentax lens market, especially our lovely Super Takumars

How? By measuring the combined resolution of both camera and lens and then publishing the figures. Voilá: a comparison.

I this case the combination of great sensor with good lens was better than a good sensor with great lens. The conclusion I draw from this is that the best modern lenses tend to be quite good, as state of the art sensors can squeeze out more resolution form them than a good sensor. Quite clearly it is the camera which is the bottleneck with Canon systems if wee are looking for maximum resolution.

You have both missed the point of the test.

The object was not to compare just cameras or just lenses. We already know the Nikon is the highest resolving camera and Canon 24-70 this highest resolving lens.

The idea was to compare SYSTEMS because you can't mount the Canon lens on a Nikon body to have the ultimate resolution combination.

Therefore you need to look at the overall resolving power of a practical camera and lens combination to draw any meaningful conclusion.

What he was interested in was the resolution that a particular camera system (camera and lens combination) gives.

So as a SYSTEM the Nikon/Tamron is equivalent to a Canon/Canon system. The fact this is so because the Nikon has no AA filter and has more pixles (thus it makes up for the poorer resolving Tamron lens) is just NOT the point.

And it tells Canon users you can just about match a Nikon 800e (with either Nikon or Tamron 24-70 as the lenses are close optically) if you are can afford the Canon 24-70.

whether it gives knowledge or information may be up to the reader. It gave me knowledge - I may contemplate selling my 24-70 Nikkor for a Tamron just for the VC. Also, if you are already enamored with your lens purchase, the decision to read the article was yours alone. I am also thinking that if the article had shown that the Canon combo was Nikon's equal, the tenor or your post would be somewhat different.

You’re right, Gothmoth. Today’s zoom lenses are often optimised for their extreme focal lengths (which doesn’t necessarily mean they perform best at their extremes, but they sometimes do). Nonetheless, I’ve never met a fast trans-standard zoom that wasn’t pretty close to its best on-axis at 50 mm. At any rate, the differences there are minuscule compared to, say, the corners at 24 mm.

For example, zoom lenses of the positive-lead type like the new Canon behave fairly predictably for lateral CA: they have a bit at the wide end (historically a lot, but the new Canon does very well there), tapering down to very little at intermediate focal lengths, and rising again at the long end.

Negative-lead types like the Nikkor are more complicated, but the Nikkor has lots of lateral CA at the wide end, yet less than the new Canon at 70 mm.

Similarly, other aberrations compare differently in these two lenses at other focal lengths. These lenses are very different despite sharing a basic role.

Don’t get me wrong: I found the article interesting (thanks Lens Rentals and DPReview), and I think the new Canon f/2.8 zoom is one of the most impressive lenses in years, and a pretty compelling statement about Canon’s lens-making prowess. That doesn’t mean the Nikkor doesn’t beat it in several optical, mechanical, and fitness-for-purpose ways.

Lenses are complicated.

"How can a lens have on-axis astigmatism? Clearly something’s wrong there: either the lens is improperly manufactured or the test is faulty, or both."

I'd put it slightly differently - all manufactured products are built to tolerances, and all tests are subject to a degree of experimental error, no matter how hard you try.
The most likely interpretation is that the Canon lens appears to be unusually well-centered in this test.

Andy: you might well be right (and I’d be content with such an explanation), but it would have been nice to hear some reassurances from Roger Cicala on the matter.

It would also be interesting to know whether the Tamron and Nikkor had ‘on-axis astigmatism’ of the same orientation – if they did it would be worth double-checking the test setup – and for that matter, what ‘vertical’ and ‘horizontal’ refer to in this case.

This is a very pertinent point and one I can't say I understand completely. The graphs printed are for single lenses, but the results consistent with each of three copies.

The bench lens mounts were an obvious question, but the Tamron lenses were the same on both Canon and Nikon mounts, so I don't think the mount is an issue.

I don't know the answer, but here's one bit of data we've seen on numerous zooms: many show slight on-axis astigmatism at one area of the zoom range but not at others. I don't think element tilt would cause an on-axis asitmatism (it certainly makes off-axis go crazy) but I believe an element spacing issue could. My thoughts, since this seems more common in zooms, is element spacing can't be perfect throughout the zoom range - at least not given current manufacturing tolerance.

But that's just my current thought. Perhaps some of the people with more optical physics than I can confirm or correct that.

Roger Cicala

Good of you to check up on us, Roger.

If the graphs look the same for three examples of each lens I’d be cautious about attributing the result to the lens instead of the test, and doubly so if the on-axis astigmatism shows a consistent bias in the vertical or horizontal orientation (I’m guessing those orientations are relative to the lens position if mounted on an upright camera).

Since a spherical (or conventionally aspherical) lens is symmetrical about its axis, on-axis astigmatism can only happen if the symmetry is harmed at manufacture (assembly, etc.). Spacing the elements incorrectly wouldn’t produce on-axis astigmatism. Tilting an element could produce on-axis astigmatism, but I’d call that a badly made lens if it significantly affects performance.

If a lens definitely did have this problem, I’d refocus to the circle of least confusion, to try to superimpose the curves on the graph.

dont tiwst and make things up as you Nikon fanatics always do.
and do not demean or insult others or go back to your Nikon fanboy club (the Nikon FX forum).

anyway, Roger is the best, I love his writing and sense of humor!

Let's reword that:
What did we learn? That the Canon 24-70 II is the absolute best standard zoom lens that money can buy, the D800E the highest resolving DSLR, and the Tamron 24-70 VC is very close in performance to the Nikon 24-70.

Well, I guess others are also uncertain as to what we learned from all this lens testing in today's article.

lol .. we learnt that? .. did your browser do a google nikon translation somewhere?

what we learnt is that the 5D Mark III + 24-70 versus the D800E + nikkor 24-70 is alot closer than alot of people wanted to hear and that the Canon 24-70 II is certainly the highest resolving 24-70 2.8 out there.

Systematically .. a 24mp versus 36mp depends on alot more moving parts than simply a sensor.

Another point is that I believe roger was using default jpg - which is far more of a resolving delta between jpg and actual RAW for the 5DIII versus D800E. With RAW processing, the 5DIII should go up more % wise than the D800E.

this was also tested at 50mm .. which if I recall, isn't the canon's 24-70mm strongest focal range - it's actually better at 24mm.

What did we learn? That people will continue to believe what they want to believe regardless of what the information and data show.

I haven't read the article yet, but from the comments I think I learned that both Nikon and Canon put out some pretty amazing gear, and that the third party lens companies do as well.

Actually rrccad,
What we learned is Canon's new lens is currently better than Nikon's almost 6 year old lens.

What we also learned is that the Nikon combination will still out resolve the Canon combination despite having to pay a premium for the latter... :)

@Petka: The DxO lens ratings are influenced by the resolution of the test body, which is why Canon lenses end up scoring lower (note that the score is quoted specifically for a lens/body combination). If it were practical to test the lenses in an equalised fashion (high res sensor without AA filter), the mix would certainly be far more even.

What we learned was nothing more than two things:
#1) At 50mm on optical bench testing, the Canon 24-70 is superior
#2) The combination of body + lens greatly influences the overall sharpness

What is left unanswered are how the lenses in question compete at the various distance ranges (since lens performance often varies by focus distance), at focal lengths other than 50mm, and of course a variety of other parameters.

What should also be thought about more is at what point is sharp enough, sharp enough. If you peruse the Zeiss MTF white paper (google it), you'll see that even the slighest miss on your focus results in a pretty drastic MTF drop. Point being, how many photographers actually nail, 100%, focus, on a top tier tripod, in light with no atmospheric issues, where on a large print you could tell the difference between the 1st and 2nd best lens mounted on equal resolution cameras? IMO subjective quality differences will be seen more than a slight MTF 'win' in my experience.