Puzzled Lens Testing - m43 lenses on e-m5

A few days ago, with a friend I ran a series of tests comparing my friend's Olympus 14-150 on his e-m5 against various lenses on my e-m5.

We shot the same shots at distances of minimum 50 to 300 feet or so, targets with lettering. Each shot was only in raw, handheld, at identical f stops (the largest f stop that was common to both--the limiting factor usually being the Olympus 14-150 smaller maximum apertures). The ISO chosen for both was the slowest that was consistent with conservative shutter speeds of at least 1/500 or faster. Usually, this meant, on a cloudy day, ISO of 400 or more usually 800. Since most of the lenses were fixed, the shots were zoomed for the 14-150, of course, at the mm of the fixed lens on the other camera.

The lenses tested were, again, the 14-150 on his camera, vs. the following on mine:

1) 20mm f 1.7

2) Panasonic 25mm f 1.4

3) Panasonic 45-175

4) Olympus 75mm 1.8

5) Olympus 45mm 1.8

I did not compare the shots on a computer screen (or, God forbid, printing). Rather, after shooting, we first checked that the frame for each shot was roughly identical, and then magnified on the camera screen the shots taken to maximum magnification and compared for image detail, one lens vs. the 14-150, lens after lens.

I was surprised that, except for the shots with the 75mm, upon comparing the images there was no noticeable difference in detail captured in the images! Only the 75mm images had more detail, but really only somewhat noticeable at maximum magnification. All the other comparisons suggested that it mattered not a whit if the shot was taken with a, for example, 20mm 1.7 or a 25mm on one em-5, or the 14-150 on the other set to the same mm.

I find these empirical observations difficult to understand. The only thing I could think of was that my friend's 14-150 was fortuitously the best sample ever made of the 14-150, which is unlikely. Can anyone suggest what I might have done wrong, or a different way of objectively testing which would have yielded results closer to what I strongly believed would be the case--that the fixed lenses would result in images with far more detail?

Thanks.

The 14-150mm is really not a bad lens.  However, given that the max aperture is f/4 at the wide end, and f/5.6 at the long end, you were presumably shooting the primes at those same apertures, which would neutralize one of the big advantages of the primes.

It's hard to judge without seeing images, but depending on what you were shooting, the time of day, and the aperture, it's not unreasonable to think that the zoom might match the performance of the primes.  However, if you're (i) shooting in low light, (ii) wishing to have a narrow DOF, or (iii) wanting nice bokeh, you're going to see the limitations of the zoom when compared to the prime lenses.

While I'm not necessarily advocating pixel peeping, I've seen shots that looked good on my camera, even zoomed to 14x, but when I got into lightroom I was unsatisfied with. While the OMD's screen is pretty good, I don't think it always accurately displays everything. This isn't to knock the 14-150... I haven't used it, so I can't knock it. Just a warning that the screen can be deceiving.

I bet that if you included similar photos made with a a good camera smartphone you also would have not seen much difference. Shooting in plenty of light outdoors of static subjects will yield similar result when compared the way you compared. Try your test indoors in poorly lit environment...  Al

Corpy2 wrote:

A few days ago, with a friend I ran a series of tests comparing my friend's Olympus 14-150 on his e-m5 against various lenses on my e-m5.

We shot the same shots at distances of minimum 50 to 300 feet or so, targets with lettering. Each shot was only in raw,

Completely useless to shoot RAW and then try to judge MTF (resolution, microcontrast) on the camera. What you actually looked at was very low-quality jpegs embedded in the RAW files.

Precisely because the jpegs embedded in the RAWs are too poor for good in-camera review, I regularly shoot RAW + Fine jpeg where the jpegs are used for in-camera review only.

But if I want to test lenses, I would of course not use in-camera review at all. I would open the RAWs in the RAW converter. Of course, I would also use a tripod unless the shutter speeds are way above the safety limit with regard to camera shake.

In short: No wonder you didn't see much of a difference under the testing conditions you chose.

handheld, at identical f stops (the largest f stop that was common to both--the limiting factor usually being the Olympus 14-150 smaller maximum apertures). The ISO chosen for both was the slowest that was consistent with conservative shutter speeds of at least 1/500 or faster. Usually, this meant, on a cloudy day, ISO of 400 or more usually 800. Since most of the lenses were fixed, the shots were zoomed for the 14-150, of course, at the mm of the fixed lens on the other camera.

The lenses tested were, again, the 14-150 on his camera, vs. the following on mine:

1) 20mm f 1.7

2) Panasonic 25mm f 1.4

3) Panasonic 45-175

4) Olympus 75mm 1.8

5) Olympus 45mm 1.8

I did not compare the shots on a computer screen (or, God forbid, printing). Rather, after shooting, we first checked that the frame for each shot was roughly identical, and then magnified on the camera screen the shots taken to maximum magnification and compared for image detail, one lens vs. the 14-150, lens after lens.

I was surprised that, except for the shots with the 75mm, upon comparing the images there was no noticeable difference in detail captured in the images! Only the 75mm images had more detail, but really only somewhat noticeable at maximum magnification. All the other comparisons suggested that it mattered not a whit if the shot was taken with a, for example, 20mm 1.7 or a 25mm on one em-5, or the 14-150 on the other set to the same mm.

I find these empirical observations difficult to understand. The only thing I could think of was that my friend's 14-150 was fortuitously the best sample ever made of the 14-150, which is unlikely. Can anyone suggest what I might have done wrong, or a different way of objectively testing which would have yielded results closer to what I strongly believed would be the case--that the fixed lenses would result in images with far more detail?

Thanks.

Corpy2 wrote:

If the camera is not showing the RAW, or a good version of it, but only a lousy small jpg for the camera screen, how am I able to zoom in all the way and see tiny details, as if the whole image were being used?

Whatever you're seeing is not RAW, but the JPG thumbnail embedded with the RAW with all the in-camera JPG settings.

Set your camera to capture in Mono, and shoot RAW only.   What will result in review is a monochrome version of the RAW capture (until you off the image to your computer).   Use the review to asses for shake and composition - colors and detail, especially regarding RAW, is best done with the computer.

It was curiosity comparison. No wonder results are similar. I takes knowledge, time, and meticulous job to reveal the differences in lenses performance.

Bob Tullis wrote:

Corpy2 wrote:

If the camera is not showing the RAW, or a good version of it, but only a lousy small jpg for the camera screen, how am I able to zoom in all the way and see tiny details, as if the whole image were being used?

Whatever you're seeing is not RAW, but the JPG thumbnail embedded with the RAW with all the in-camera JPG settings.

I agree with Bob here. You think you are zooming all the way in, but in reality you aren't. You are still dealing with a low resolution JPG. Put it on the computer and you'll see things differently.

Corpy2 wrote:

Anders W wrote:

Corpy2 wrote:

A few days ago, with a friend I ran a series of tests comparing my friend's Olympus 14-150 on his e-m5 against various lenses on my e-m5.

We shot the same shots at distances of minimum 50 to 300 feet or so, targets with lettering. Each shot was only in raw,

Completely useless to shoot RAW and then try to judge MTF (resolution, microcontrast) on the camera. What you actually looked at was very low-quality jpegs embedded in the RAW files.

Precisely because the jpegs embedded in the RAWs are too poor for good in-camera review, I regularly shoot RAW + Fine jpeg where the jpegs are used for in-camera review only.

Wait. As I replied to the other person, I am able to zoom in and see tiny detail, the kind that I would only normally be able to see if pixel peeping on a computer screen.

If I were only looking at very low quality jpg's, the jpg's should fuzz out vcery quickly once I start zooming in a bit, should they not?

You can zoom in fully even on these low-quality jpegs. It's just that they look poorer when you do so than a high-quality jpeg or what you get if you open the RAW file in the RAW converter.

If you can't open RAW files on a monitor, the camera screen can be a guide, but not a reliable one.

I too have done various comparisons that looked identical zoomed in on the camera screen but differed greatly on a monitor.