11 MFT camera ISO image comparison - image heavy

[mfuhlendorf]

I got thinking after the huge "How Many m43 bodies do you own" thread, I found out that I'm kind of a camera hoarder. I've gotten newer, prettier or better cameras with time, never letting go of the older ones, and it became sort of a collection, not only for their usability but for objects of beautiful industrial design... Not expensive cameras, but all of them fun, most bought used already.

The poll showed I was in the minority 2% of users with a lot of old cameras, so I thought, why not compare them all side by side and see if there any outliers in noise and overall image quality. I decided to do a strict m43 comparison, not including the S5 or my Nikon 1 bodies, and these are the contenders:

1. 2mp Blackmagic Pocket Cinema Camera (1080p CinemaDNG RAW)
2. 12mp Olympus Pen E-PL2
3. 16mp Lumix GM1
4. 16mp Lumix GX-80
5. 16mp Olympus E-M1
6. 16mp Olympus Pen E-PL6
7. 16mp Olympus Pen E-PL9
8. 16mp Olympus Pen E-PL10
9. 16mp Olympus Pen E-PM2 (IR 590nm converted)
10. 16mp Z-Cam E1
11. 20mp Lumix G9

The test was done on a tripod, not a great one so there a minor frame variations. I mounted on it a borrowed Olympus 40-150mm f2.8 PRO, set it to f5.6, focused manually on the scene below and switched cameras without moving the lens. For each camera, I shot ISOs 200-400-800-1600-3200-6400-12800, compensating on exposure time to keep the same exposure. I only reframed once, for the Blackmagic pocket (BMPCC), as it's a video camera that I use for work and the sensor is closer to s16mm in size. Saved everything raw (the BMPCC as clips of cinemaDNG raws, from which I picked one image per clip), imported into lightroom, adjusted small exposure shifts as well as possible, matched white balances, and did not touch anything else. Then, I created photoshop actions to automate the generation of comparison PNGs, which I'll post below.


SOOC jpg from the G9 of the test scene. My wall has warm white LED spotlights only, and I did this at night to avoid any color shifts from windows.

I did all this as a test for my personal use, to check if there are any outliers in quality among my cameras, and decided to post it here since it's been a lot of work. Also, maybe someone mulling getting one of these cameras can get some use out of this.

I also didn't bother with UniWB, over/under exposure or any ISO invariance tests, as these are far from a scientific comparison: mostly, I want to know how my cameras perform overall, to make it easier to choose which I'll take on different kinds of trips

I'll post the comparisons in the next posts, and all renamed but otherwise SOOC RAWs are being uploaded here if anyone wants to play with them (the images are still uploading): https://drive.google.com/open?id=1Rr0wO5tHF77zZpt68CJ9uazX3G7riZkC&usp=drive_fs . The files are named with ISO, camera, exposure time and aperture, like this: iso(200)cam(DC-G9)time(167ms)fstop(5.6)__P1037063.RW2


This is a somewhat central crop, 432x432px, of all cameras vs all ISOs

The camera ISO comparisons were done at their original resolutions, and the comparisons between cameras were done upsampling all the cameras to the G9's 20mp, using bicubic sharper from photoshop.

My takeaways:

• The G9 is, as expected, the best of the bunch, amazing camera, and I only don't use it exclusively because of weight and size when travelling.
• The E-PL2 is great at low ISO, but terrible otherwise. It really fails here at anything higher than 800 or so, except for web use.
• All the 16mp Olympus Pens, and also the 16mp Panasonics look remarkably similar, with the exception of the known 10-bit issue with the GM1 using e-shutter. But even with e-shutter, the lower ISOs look great (this image does not maximize DR enough to test clipping and such), and I'd use it happily for daily snapshots. In sum, the 16mp sensors are all excellent, especially without pixel peeping, IMHO.
• The BMPCC is an outlier, it is a dedicated video camera, with raw 1080p CinemaDNG output on a s16mm sized sensor. It doesn't really belobg here in this comparison but, since it's m43 mount, I did it anyway. It only goes up to ISO 1600, but the noise at the maximum rating is very pleasing, almost filmic. For 1080p video, it is an AMAZING camera.
• The Z-Cam E1, which I bought on a whim during the fire sales they did after discontinuing it, has TERRIBLE jpegs. But, the DNG raws are surprisingly great. It has some inconsistencies (I couldn't perfeclty match exposure with the higher ISOs, for example), and the user interface is absolutely useless, but it can be used as a micro camera for pocket use, with a wide angle manual focus prime, and render excellent results. It's MUCH less useful than a GM1, but ends up being fun at times.

Anyways, hope this can be of use to anyone considering one of these models.

Cheers,

--
Max M. Fuhlendorf

 

[mfuhlendorf]

These compare all cameras, with 6 432x432px crops from each camera at each ISO.




ISO 200




ISO 400




ISO 800




ISO 1600




ISO 3200




ISO 6400 (not an option on BMPCC)




ISO 12800 (not an option on BMPCC and Pen E-PL2)



--
Max M. Fuhlendorf

 

[mfuhlendorf]

BMPCC




G9




GM1 E-Shutter




GM1 Hybrid shutter




GX80




E-M1 original




E-PL2




E-PL6




E-PL9




E-PL10




Z-Cam E1




E-PM2 590nm Infrared. Note that I had no IR light source available at the time, and LED warm white lights emit almost zero IR light, so this B&W rendition is made almost exclusively from the RED channel. Due to the bayer patteen, this is pretty much a ~4mp image.



--
Max M. Fuhlendorf

 

[Jack Tingle]

Interesting experiment.

Using similar methods I decided to keep my M10.2 at ISO 800 & my G95 at 1600 mostly.

 

[Will Rose]

Thanks for posting this, it’s interesting.

You're right the G9 is obviously the standout.

But of the others, it looks to me like the EPL6 and GM1 are conspicuously sharper than the rest…

 

[mfuhlendorf]

Thanks for posting this, it’s interesting.

You're right the G9 is obviously the standout.

But of the others, it looks to me like the EPL6 and GM1 are conspicuously sharper than the rest…

Yeah, as far as sharpness goes, I wouldn't take too much stock in my test: I used an Olympus pro lens with focus clutch, not a fully manual lens. I checked focus after each camera change but I wasn't using external monitors or anything, my focusing might be less than perfect...

 

[Regor250]

Interesting experiment.

Using similar methods I decided to keep my M10.2 at ISO 800 & my G95 at 1600 mostly.

Yikes... while the images can be cleaned up, there is a significant loss of details, DR and latitude in post processing nonetheless. I will do it out of necessity, but certainly not as my default. Last spring I took an entire day images @ ISO1600 with the OM-1 inadvertently, bright daylight and the loss of details was obvious and took significant post-processing effort to clean up nicely. Distant foliage, small branches and ground texture pretty much got all messed up. It was what I call salvage, far from optimal.

--
Roger

 

[Regor250]

And what did you conclude?

 

[mfuhlendorf]

And what did you conclude?

Nothing super special... It was a fun experiment, something of an excuse to spend a day with all my gear. I only take pictures, I'd never attempted any sort of testing of equipment before. I'll happily use any of these cameras for the right occasion, after testing them, though.

As expected the 12mp sensor is much worse at high iso, but I don't need the E-PL2 for that... For daylight use it is just fine.

The G9 is great, but still too big for some use cases.

To me the most surprising thing is how similar most of the 16mp camera's results were: I imagined the E-PL10 might look noticeably better than the GM1, but to my eyes the differences are subtle. Heck, even the G9 is not tremendously better than the rest, it's better when compared side by side but not by much really.

Main takeaway: all sensors of the past decade at least are more than excellent for most uses and I'm very happy to know I don't have to worry too much about loss of quality when using my older cameras (except maybe the E-PL2 in the dark hahaha)

--
Max M. Fuhlendorf

 

[ahaslett]

The performance of sensors when shooting RAW is largely a matter of age and specs. Not a lot of change to basic IQ for some time.

Now lenses differ quite a bit, and in subtle ways, requiring a basket of different tests.

Off you go!

A

--
Infinite are the arguments of mages. Truth is a jewel with many facets. Ursula K LeGuin
Please feel free to edit any images that I post

 

[mfuhlendorf]

The performance of sensors when shooting RAW is largely a matter of age and specs. Not a lot of change to basic IQ for some time.

Now lenses differ quite a bit, and in subtle ways, requiring a basket of different tests.

Off you go!

A

Yeah, I figured, but still the smallness of the difference between these cameras was a little surprising. But positive, I mean, they're mostly excellent and, knowing that, I can choose cameras for different outings based purely on ergonomic and aesthetic factors hehe.

As for the lens tests, I leave those to review sites. Comparing 11 cameras with a few shots each was fun, comparing lenses is not a fun pastime anymore...

Ps: Ursula Le Guin is the BEST

--
Max M. Fuhlendorf

 

[mmartel]

Very surprised to see this thread is 4 days old and no one has yet acknowledged the rather obvious: This took quite a considerable effort to conduct and document your results. So I would like to be the first to do so. Thank you for doing this test and sharing your results with the community.