DPR Oly Lens tests will NOT be on E3

I can't argue with any of your conclusions as I never used the E-3, never used the ZD12-60 and also never used the E-300.

It still sound a bit strange to me though. That is only from general experience from several other dSLR cameras.

With my 5D I used lenses that clearly outresolved the sensor in various degrees. The same with the DS and also with the E-510. The colors are always a little different. As I shoot raw only and PP my picture and use my own profiles in ACR, sometimes also the Incamera plugin in Photoshop, the colors are never a problem.

The contrast... do you mean macro contrast?

The lens is an analog apparatus, I have some problems seeing how it can interfere with the sensor's pixel pitch.

I'm sorry, but there is something I don't understand here. Maybe it's just me being thick, stuck in old ideas and understandings.

regards,

--
Jonas

pgabor said:

The colors are a little bit off. I think it is because of the
different coating between the cheap entry-level and the more
expensive lenses.

The contrast was quite weird too. I think it is just not tuned for
the brick.

And finally, the resolution (sharpness). The 12-60mm clearly
outperforms the resolution of the 8MP sensor. Unfortunately, it does
this in a very strange way and therefore interferes with it. This is
of course just what I concluded.

Click to expand...

... then how long will the test results remain valid? I think dpreview's choice to select the L10 shows a flaw in their test methodology... I just don't know how they will be able to measure lens performance without being prone to (constantly evolving) sensor / body performance.

Kind Regards

Brian
--



Join the Olympus UK Photo Safari Group in March, for free coaching and a chance to review the Olympus E-420 for yourself!
http://forums.dpreview.com/forums/read.asp?forum=1022&message=27079230

For details of this and upcoming free events, please email me at my address above / click below :
http://forums.dpreview.com/forums/email_poster.asp?poster=hjigidiuhv

I elaborated a little on just that in my first reply to you. As I said (or I think I did that without going back now) experience from evolution from 6 to 14MP hasn't been very dramatic. Some weak lenses (known already at early stages) got sort of "weaker" but there really hasn't been a massive need of new lens tests.

But basically you are right. The question is how long it will take for the 4/3 system to reach something around 20MP?

I guess the DPR management just has decided to the best they can.

kind regards,

--
Jonas

Brian Mosley said:

... then how long will the test results remain valid? I think
dpreview's choice to select the L10 shows a flaw in their test
methodology... I just don't know how they will be able to measure
lens performance without being prone to (constantly evolving) sensor
/ body performance.

Click to expand...

Andy Westlake said:

Just in response to some of the more extreme opinions in this thread:

We have actually tested several Olympus lenses on the E-3 using our
standard methodology. And after a lot of careful consideration, we've
decided to scrap this data and use the L-10 instead, as it delivers
measurably higher resolution. This isn't some slight, trivial
difference; the MTF50 results on the E-3 are surprisingly low
compared to cameras from other brands. And it's not a decision taken
lightly, as it involves several lens profiles and a week's worth of
work.

So, in order to level the playing field, we've chosen to use the L-10

Click to expand...

No, you'd use an optical bench if you wanted to level the field.

Andrew Westlake said:

as this provides files which are far more comparable to our other
test bodies. We have tested multiple lenses and RAW converters, and
the pattern is very clear; the L-10 delivers higher resolution. By
far the most plausible explanation is that Olympus have used a
stronger-than-usual AA filter on the E-3.

The decision to use the L-10 has been taken purely because I'm
reviewing lenses, not bodies, and therefore want the test cameras to
behave as similarly as possible. If this is any kind of bias, it is
in favour of Olympus; I will be providing data which make the
lenses look better, not worse.

Click to expand...

But you ARE testing bodies - your whole methodology is useless for generating lens performance data. If you were REALLY testing the lens, then the body used wouldn't be at issue, would it?

Andrew Westlake said:

We're not the only site to have taken this approach; as you will
know, Klaus at Photozone is also using the L-10 for much the same
reasons. BTW anyone who believes that as experienced a tester as
Klaus would make this decision based solely on Simon's review, and
not his own judgement, is seriously deluded.

Click to expand...

Not as deluded as Klaus, if he thinks he can come up with good lens data based on system performance.

Andrew Westlake said:

This is not some 'side-swipe' at the E-3 (believe me, I'd prefer to
be using it in the studio than the L-10). It makes no judgement
whatsoever on Olympus's decisions with respect to AA filters, or
whether a strong or weak one is 'better'. It just means that when
people compare Olympus lenses to other brands using the widget, the
data isn't strongly skewed against Olympus.

Click to expand...

The data is so skewed by the method anyway what difference would it make?

--
--
mumbo jumbo

Jonas B said:

I can't argue with any of your conclusions as I never used the E-3,
never used the ZD12-60 and also never used the E-300.

Click to expand...

Wow. What do You have then?

Jonas B said:

It still sound a bit strange to me though. That is only from general
experience from several other dSLR cameras.
With my 5D I used lenses that clearly outresolved the sensor in
various degrees. The same with the DS and also with the E-510. The
colors are always a little different. As I shoot raw only and PP my
picture and use my own profiles in ACR, sometimes also the Incamera
plugin in Photoshop, the colors are never a problem.

Click to expand...

Well, the problem is not the outperforming itself. It is the interference. This is a similar effect as the moire. The problem happens when the resolution of the lens and the sensor is only slightly different. Let's say the resolution of the 12-60 is 32.1MP. The resolution of the E-3 is 10. There is clearly no problem. However, in case of the E-300 it is slightly more than 4x (2x to each direction). Well, now You can be sure about that You will have an interference.

Jonas B said:

The contrast... do you mean macro contrast?

Click to expand...

No, I mean contrast. Try to use the same body with the exact same setup but with different lenses. You will see that the levels of Your images will be different (this is actually one reason why measuring DR is really tricky). Here, I do not mean only that the curve will be shifted to the left or the right but it will also be shrinked or expanded.

Jonas B said:

The lens is an analog apparatus, I have some problems seeing how it
can interfere with the sensor's pixel pitch.

Click to expand...

It is not. In reality its surface is made of straight lines.

Jonas B said:

I'm sorry, but there is something I don't understand here. Maybe it's
just me being thick, stuck in old ideas and understandings.

Click to expand...

No worries. I make mistakes too. It is understandable if You are critical but hope the above clarification helps.

Jonas B said:

regards,

--
Jonas

Click to expand...

G.

D Q E said:

D Q E said:

So, in order to level the playing field, we've chosen to use the L-10

Click to expand...

No, you'd use an optical bench if you wanted to level the field.

Click to expand...

Even with an optical bench I reckon you'd have to settle on some
specific lp/mm to make the measurement at. This should relate to the
pixel pitch to be as relevant as possible. But when optimising this for
a specific pixel pitch you are in effect choosing a camera as a model
for the test. And you'd use different lp/mm for different sensor sizes,
i.e. mounts.

And bedies, Andy has explained why using an optical bench would
be an unsuitable solution for DPR for practical reasons.

Just my two oere
Erik from Sweden

I'm perplexed and don't know what to say. Clarification - lol, I got headache!

There are so many questions I would like to ask. This is probably not the place though so I'll go read up on optics.

I think this is the first time ever I heard about a lens really having element surfaces being straight lines . I can imagine a mathematical model where we search for a small part (angle) of a lens surface (where limes [surface] can be said to be flat). In practise the elements are nicely curved and make an analog device, in my world.

Resolution isn't absolute, it is a value we determine after having settled for a certain contrast ratio. I think it can be expressed that way: "what is the resolution at 50% contrast for example. It is a gliding scale. This again is as far as I have thought about it for a long time.

The above may serve to demonstrate how thick I am. I need time to digest your facts.

My dSLR cameras have been an *ist DS, a 5D, a Pentax again: K10D, and now the E-510. I have mounted Pentax, Canon, Leica, Olympus (OM and ZD), Zenitar and... lenses to these cameras. Many of the lenses have outresolved the sensors. In the center more than at the edges.

Why haven't this interference showed up somewhere in the pictures? And why haven't I ever heard about it earlier? Retorical questions I guess, but you surely understand I have to read and think a lot about this.

kind regards,

--
Jonas

pgabor said:

Jonas B said:

I can't argue with any of your conclusions as I never used the E-3,
never used the ZD12-60 and also never used the E-300.

Click to expand...

Wow. What do You have then?

pgabor said:

It still sound a bit strange to me though. That is only from general
experience from several other dSLR cameras.
With my 5D I used lenses that clearly outresolved the sensor in
various degrees. The same with the DS and also with the E-510. The
colors are always a little different. As I shoot raw only and PP my
picture and use my own profiles in ACR, sometimes also the Incamera
plugin in Photoshop, the colors are never a problem.

Click to expand...

Well, the problem is not the outperforming itself. It is the
interference. This is a similar effect as the moire. The problem
happens when the resolution of the lens and the sensor is only
slightly different. Let's say the resolution of the 12-60 is 32.1MP.
The resolution of the E-3 is 10. There is clearly no problem.
However, in case of the E-300 it is slightly more than 4x (2x to each
direction). Well, now You can be sure about that You will have an
interference.

pgabor said:

The contrast... do you mean macro contrast?

Click to expand...

No, I mean contrast. Try to use the same body with the exact same
setup but with different lenses. You will see that the levels of Your
images will be different (this is actually one reason why measuring
DR is really tricky). Here, I do not mean only that the curve will be
shifted to the left or the right but it will also be shrinked or
expanded.

pgabor said:

The lens is an analog apparatus, I have some problems seeing how it
can interfere with the sensor's pixel pitch.

Click to expand...

It is not. In reality its surface is made of straight lines.

pgabor said:

I'm sorry, but there is something I don't understand here. Maybe it's
just me being thick, stuck in old ideas and understandings.

Click to expand...

No worries. I make mistakes too. It is understandable if You are
critical but hope the above clarification helps.

Click to expand...

Bootstrap said:

I do think that the lens tests would be best made on the entry level
cameras and appropriate conclusions can be drawn from those tests for
the upscale users.

Click to expand...

It's better to do it the other way around.

Bootstrap said:

How about, for example, using the much maligned E-300 as a body for
testing the lenses? I think that's a great idea!

Click to expand...

Photozone used to do that, with the effect that Oly lenses scored way below
those tested on the also 8Mp Canon 350D. Despite clear notifications about
results not being cross-comparable, many people probably got the false
impression that Oly lenses sucked. If you want people to be impressed
by the 4/3rds optics it's about the worst you could do.

Bootstrap said:

For those of us that don't have the benefit of upgrading camera
bodies at our whim, tests of the lenses on the less expensive bodies
might make more sense.
If they perform well on an E-300, it's quite likely that they will
perform well on an E-3.

Click to expand...

But a low-resolution camera will not reveal the peak performance well, you
can't say if the lens outresolves the sensor barely or a lot. Two lenses that
look similar on the low-res camera might score (and deliver photographically)
quite differently on the hi-res one.

(See the review of the Sony 18-70 that looked ok on 6Mp but is beginning
to look old on higher res cameras.)

Going the opposite way is easier: Simply put, any score achieved on the
hi-res camera above some number (which could be estimated to reasonable
accuracy) can safely be assumed to outresolve the low-res camera.

Just my two oere
Erik from Sweden

I understand what you are trying to accomplish by your methodology and it may even be quite well thought out and sound in it's reasoning...

EXCEPT....

Your intended test methods are going to create "Unrealistic Expectations" in your readership...

I mean - you are going to use a seldom purchased camera body to facilitate your tests while the lenses you test will end up being used 90% of the time on - what you describe as - an under performing body (s).....

It will take exactly 1.3 minutes to generate your first "My lens must be buggered because it is nowhere near as good as the DPR tests indicate" thread followed by umpteen different complaints saying the same thing...

I mean let's dyno test my car using the absolute best atmospheric conditions and temperature to REALLY see what it can produce when it will NEVER be driven under those conditions ???????

PLEASE add the following APPROPRIATE CAVEAT to your published test...

NOTE: The results of this test are in no way indicative of results one should expect unless using the exact same equipment under the exact same conditions present at the time of the test... Further, for the VAST majority of owners or potential owners of this lens - please understand that these results are BEST CASE scenario results and - due to limiting factors in the vast majority of camera bodies that will be used in conjunction with this lens - one should not REASONABLY anticipate achieving results close to what we have reported herein...

Bottom line for me is that... You should be looking to publish REAL WORLD results that owners can anticipate - if not expect - from their camera / lens combinations... That WOULD set you apart from other lens reviews and give us something constructive to read...

Regardless - I honestly believe your venture into Lens Reviews is a FOLEY & that you should stick to what you have been doing best all along to garner clicks... Unless posting contentious reviews generates clicks and that is the only thing that matters most to you ?????????????????????????

I just don't know WHY - when you are Sooooooooo backlogged already ( 3 - 6 months by some estimates ) with reviews - do you venture into uncharted waters that are likely to only generate negative criticism...

The old saying "Bit off more than you can Chew" comes immediately to mind....

Cheers...

Although I have expressed to Simon my doubts that the E-3 really does have a very strong AA filter (the JPEGs just seem to be too sharp for this - and they don't look as though they've had excessive sharpening applied - so IMHO the problem is more likely to be with Adobe's E-3 RAW converter), the fact that your methodology works best with the L10 means it is definitely the best body to use. A lens test is not a measure of overall image quality; it is a measure of distortion, sharpness and resolution. The first will be the same across all bodies, but for the last two, you should obviously use the highest resolution body available. And given that under your methodology the L10 is exactly that, it is clearly the body to use (its shortcomings are irrelevant in this context).

I still question your assumption that ACR is a level playing field, however. I also question how a little extra highlight range in the D300 (which doesn't show up in RAW conversions anyway) can be given so much more weight than the vastly better - and measurably more accurate - tonal and colour response of the E-3 when you judge IQ. I also question how the superior quality of Olympus lenses can be dismissed as irrelevant in your reviews of camera bodies (e.g. the way the superior sharpness of the Olympus 50mm macro is dismissed in the D300 review).

Unfortunately, lenses are not analog devices. That would be the ideal case, but in reality they use grinding and polishing and therefore this (among other things) determines the maximum possible resolution of a lens. Here You can find more:

http://www.madehow.com/Volume-2/Camera-Lens.html

There are quite a few reasons why You have not heard about interference. First, manufacturers like not to talk about it. Second, it is really rear. I myself was also surprised this happened to me. The resolution of the sensor and the lens have to be really really close. Third, even different copies of the same lens can behave differently and therefore some might others might not interfere with he exact same sensor.

G.

Jonas B said:

I'm perplexed and don't know what to say. Clarification - lol, I got
headache!
There are so many questions I would like to ask. This is probably not
the place though so I'll go read up on optics.

I think this is the first time ever I heard about a lens really
having element surfaces being straight lines . I can imagine a
mathematical model where we search for a small part (angle) of a lens
surface (where limes [surface] can be said to be flat). In practise
the elements are nicely curved and make an analog device, in my world.

Resolution isn't absolute, it is a value we determine after having
settled for a certain contrast ratio. I think it can be expressed
that way: "what is the resolution at 50% contrast for example. It is
a gliding scale. This again is as far as I have thought about it for
a long time.

The above may serve to demonstrate how thick I am. I need time to
digest your facts.

My dSLR cameras have been an *ist DS, a 5D, a Pentax again: K10D, and
now the E-510. I have mounted Pentax, Canon, Leica, Olympus (OM and
ZD), Zenitar and... lenses to these cameras. Many of the lenses have
outresolved the sensors. In the center more than at the edges.

Why haven't this interference showed up somewhere in the pictures?
And why haven't I ever heard about it earlier? Retorical questions I
guess, but you surely understand I have to read and think a lot about
this.

kind regards,

--
Jonas

Click to expand...

The lenses are going to far outlast the bodies. What happens when someone buys the next generation body and uses the lens reviews to help them figure out which lenses to buy? You can't let the body dictate the lens review.

A sharp lens is a sharp lens, and should still perform better on an E-3 than a weak lens.

Higuma said:

I understand what you are trying to accomplish by your methodology
and it may even be quite well thought out and sound in it's
reasoning...

EXCEPT....

Your intended test methods are going to create "Unrealistic
Expectations" in your readership...
I mean - you are going to use a seldom purchased camera body to
facilitate your tests while the lenses you test will end up being
used 90% of the time on - what you describe as - an under performing
body (s).....
It will take exactly 1.3 minutes to generate your first "My lens must
be buggered because it is nowhere near as good as the DPR tests
indicate" thread followed by umpteen different complaints saying the
same thing...

Click to expand...

--
dgrogers

http://www.pbase.com/drog

Jonas B said:

I elaborated a little on just that in my first reply to you. As I
said (or I think I did that without going back now) experience from
evolution from 6 to 14MP hasn't been very dramatic. Some weak lenses
(known already at early stages) got sort of "weaker" but there really
hasn't been a massive need of new lens tests.

But basically you are right. The question is how long it will take
for the 4/3 system to reach something around 20MP?

Click to expand...

20mp on a 4/3rds? About as long as it takes for people to want $100 P&S dynamic range on a pricey DSLR...Hopefully, never.

--



'I cried because I had no E-3. Then I met a man with no E-510'

Hi Jonas,

Jonas B said:

I elaborated a little on just that in my first reply to you. As I
said (or I think I did that without going back now) experience from
evolution from 6 to 14MP hasn't been very dramatic. Some weak lenses
(known already at early stages) got sort of "weaker" but there really
hasn't been a massive need of new lens tests.

Click to expand...

So why the big fuss about the difference between the E-3 and the L10? it doesn't add up... either the body/sensor resolution is insignificant and the results will remain valid over time and across multiple generations/models of body, or the tests are highly dependent on the test body/sensor resolution.

Or, are dpreview saying that the E-3 has terrible resolution and it's thrown out their tests completely... making them have to scrap all those previous results with the E-3 and restart with the L10 as a 'big favour' to Olympus? lol

Kind Regards

Brian

Jonas B said:

But basically you are right. The question is how long it will take
for the 4/3 system to reach something around 20MP?

Click to expand...

Nope, the question is, how long will it take for another increment in performance as that between the E-3 and the L10 (same sensor, different processing?)

Kind Regards

Brian

Jonas B said:

I guess the DPR management just has decided to the best they can.

Click to expand...

I don't know what the DPR management are trying to do... I'm not in charge of their business development.

I would like to see the RAW files for the L10 and the E-3 published, so that we can have an independent review of their RAW processing approach.

Kind Regards

Brian

Jonas B said:

kind regards,

--
Jonas

Click to expand...

--



Join the Olympus UK Photo Safari Group in March, for free coaching and a chance to review the Olympus E-420 for yourself!
http://forums.dpreview.com/forums/read.asp?forum=1022&message=27079230

For details of this and upcoming free events, please email me at my address above / click below :
http://forums.dpreview.com/forums/email_poster.asp?poster=hjigidiuhv

BigMike1 said:

But it's aimed more at wedding photographers rather than people who
need super-sharp images for large enlargements.

Click to expand...

That must be narrowed down to weddings in pouring rain then. I mean E-3 is weather sealed like no other DSLR. There must be many like me who bought one for landscape and nature photography who now ask themselves what kind of construction this is. Supersharp glass and a body with permanent soft filter - makes no sense.

Like Klaus at Photozone said: This is not what you wanna use for landscape. Now Klaus relied 99% on the DPR tests that stand alone as far as I know.

I'd be great to hear from Olympus but I don't think it will happen although their flagship body is being humiliated big time right now.
--
************************
I like shooting cats - I'm allergic

Hi Brian,

Brian Mosley said:

So why the big fuss about the difference between the E-3 and the L10?

Click to expand...

I don't know for sure but it is probably the same old thing as always; people are touchy when it comes to their choice of camera. Maybe some E-3 owners got their feelings, or even egos, a bit hurt?

Brian Mosley said:

Brian Mosley said:

[Big part with a mainly repeatinf discussion was cut]

Click to expand...

Click to expand...

Brian Mosley said:

I would like to see the RAW files for the L10 and the E-3 published,
so that we can have an independent review of their RAW processing
approach.

Click to expand...

And the next step would be that somebody here accuses DPR for not focusing correctly?

Earlier in this thread I saw a post where it was mentioned that raw files from another review site are available. A test with dcraw showed the same result; the Panasonic camera still had better resolution.

Thinking about it... If I was in charge for the lens testing I would make an effort and remove the AA filter from a cheap 10MP camera (read E-420) and check the result and compare it to an unmodified camera. I would also publish the raw files if I had the bandwidth. Then I would start to worry about the Canikopentony crowds, and then I would have to worry about my income. Lol.

Maybe DPR will check the kit lenses only anyway.

And a random recycled one, taken with an inferior camera (E-510) with a really inferior lens (Sigma 30/1.4):



Cheers,

--
Jonas

Thank you G.
I guess I have been lucky.

--
Jonas

pgabor said:

http://www.madehow.com/Volume-2/Camera-Lens.html

There are quite a few reasons why You have not heard about
interference. First, manufacturers like not to talk about it. Second,
it is really rear. I myself was also surprised this happened to me.
The resolution of the sensor and the lens have to be really really
close. Third, even different copies of the same lens can behave
differently and therefore some might others might not interfere with
he exact same sensor.

G.

Jonas B said:

I'm perplexed and don't know what to say. Clarification - lol, I got
headache!
There are so many questions I would like to ask. This is probably not
the place though so I'll go read up on optics.

I think this is the first time ever I heard about a lens really
having element surfaces being straight lines . I can imagine a
mathematical model where we search for a small part (angle) of a lens
surface (where limes [surface] can be said to be flat). In practise
the elements are nicely curved and make an analog device, in my world.

Resolution isn't absolute, it is a value we determine after having
settled for a certain contrast ratio. I think it can be expressed
that way: "what is the resolution at 50% contrast for example. It is
a gliding scale. This again is as far as I have thought about it for
a long time.

The above may serve to demonstrate how thick I am. I need time to
digest your facts.

My dSLR cameras have been an *ist DS, a 5D, a Pentax again: K10D, and
now the E-510. I have mounted Pentax, Canon, Leica, Olympus (OM and
ZD), Zenitar and... lenses to these cameras. Many of the lenses have
outresolved the sensors. In the center more than at the edges.

Why haven't this interference showed up somewhere in the pictures?
And why haven't I ever heard about it earlier? Retorical questions I
guess, but you surely understand I have to read and think a lot about
this.

kind regards,

--
Jonas

Click to expand...

Click to expand...

Hi Jonas,

Jonas B said:

Brian Mosley said:

So why the big fuss about the difference between the E-3 and the L10?

Click to expand...

I don't know for sure but it is probably the same old thing as
always; people are touchy when it comes to their choice of camera.
Maybe some E-3 owners got their feelings, or even egos, a bit hurt?

Click to expand...

I actually meant the big fuss by dpreview... stating that they'd had to 'abandon' testing with the E-3 because it was 'so much inferior' to the L10 in performance.

This is not about egos, it's about understanding how much of a difference will require all previous lens test results to be 'scrapped' because a marginally better performing sensor becomes available... will they have to scrap all the L10 results when the E-420 is available? will they have to scrap all the D300 tests in favour of the better performing D??? I'm just trying to get to the underlying facts here... that's all, and if the test approach is flawed - perhaps an improvement can be made which will save dpreview lots of time repeating tests later, and give us more useful information about lenses?

Kind Regards

Brian
--



Join the Olympus UK Photo Safari Group in March, for free coaching and a chance to review the Olympus E-420 for yourself!
http://forums.dpreview.com/forums/read.asp?forum=1022&message=27079230

For details of this and upcoming free events, please email me at my address above / click below :
http://forums.dpreview.com/forums/email_poster.asp?poster=hjigidiuhv

Why not just let them (DPReview) get on their business the way they decide to go about it?

They reached a compromise that they believe is workable. Instead of all this speculating, worrying, positioning, line-in-the-sand drawing...

...JUST CHILL!!

Let's see what they come up with. If you don't like it, remember that the results are worth every Dollar/Pound/Euro/Yen you paid for.

--
If you don't talk to your cat about catnip, who will?

And for those of you in the US...In the words of a former boss (who was the county elections official): 'Vote early and often'.

as MP's go up surely all lens tested on this planet will have to be retested from the arguments I've seen, what i don't understand is all the fuss on these forums because the L10 has a better AA filter to make lens tests with, imagine if the E3 had been used and not so good results posted and the reason for the results were due to a strong AA filter......blah blah Oly bias....blah blah!
--
http://illy.smugmug.com