OLYMPUS E-M10 Mark III - RAW issues

I'm not convinced that there has to be anything wrong with the raw files (that would be very rare, and they seem to work fine elsewhere), or that this Viewer problem is related to that of Silkypix.

When you shot your images (ORFs), did you use any non-standard settings that the DPReview people may not have used - for instance, shoot in AdobeRGB color space?

Raw files are linear and when half the histogram drops to zero, that suggests to me that something is going wrong with the handling of 16-bit data (which I assume is what the "Display RAW files at the highest quality" setting does -- it tries to show 16-bit/ch data rather than the normal 8b/ch). More specifically, the first 8 bits in each 16-bit pixel component seem to have been reduced to 0.

When you leave that 'highest quality'setting off, does this same error still occur anywhere?

If it does not, (and I know this may sound odd) my first suspicion here would be a buggy video driver. Have you visited the manufacturer's site to check if you are using the latest released driver for your videocard or chipset? You could also see if the video driver (or a separate configuration/management tool that may have come with it) has any relevant settings for handling of 48-bit or 16-bit/channel data.

Danielvr said:

I'm not convinced that there has to be anything wrong with the raw files (that would be very rare, and they seem to work fine elsewhere), or that this Viewer problem is related to that of Silkypix.

Click to expand...

Thank you for your response

The same problem was confirmed by OLYMPUS representative when he downloaded the images and tested on his computer.

Danielvr said:

When you shot your images (ORFs), did you use any non-standard settings that the DPReview people may not have used - for instance, shoot in AdobeRGB color space?

Click to expand...

No. Only the usual Aperture / Exposure Compensation controls, selecting Fine+RAW. The color space was not changed and is sRGB, no other advanced options like Tonal or Contrast settings were changed. Especially after Reset, all settings were standard factory.

Danielvr said:

Raw files are linear and when half the histogram drops to zero, that suggests to me that something is going wrong with the handling of 16-bit data (which I assume is what the "Display RAW files at the highest quality" setting does -- it tries to show 16-bit/ch data rather than the normal 8b/ch). More specifically, the first 8 bits in each 16-bit pixel component seem to have been reduced to 0.

When you leave that 'highest quality'setting off, does this same error still occur anywhere?

Click to expand...

Yes as in Examples 2 and 3, it occurs any time images changes in any of the editing modes (White Balance, Contrast, Exposure etc.) in Viewer 3. (It is the latest version of Viewer 3; same behavior before and after its internal update.)

Danielvr said:

If it does not, (and I know this may sound odd) my first suspicion here would be a buggy video driver. Have you visited the manufacturer's site to check if you are using the latest released driver for your videocard or chipset? You could also see if the video driver (or a separate configuration/management tool that may have come with it) has any relevant settings for handling of 48-bit or 16-bit/channel data.

Click to expand...

The system I am using is about a year old Lenovo T460 with Windows 10 64-bit and Intel HD Graphics 520. Intel updater shows that it's the up-to-date driver. Also there are no issues on my machine with the RAW files from the same camera from the DPReview review.

The RAW files are linked in the original post. If anyone wishes to try them on their machine with OLYMPUS Viewer or Silkypix, it would be greatly appreciated.

I tried the toy picture and sure enough here downunder the same as you get when I click the full resolution icon in OV3 - histogram shrinks and contrast lost.

Same effect with GPU on or off, again a recent Lenovo notebook. Same effect with OV3 on a homebuilt desktop i5 cpu on Asus motherboard.

DxO Photolab no help as not yet updated to include the Mk3.

RawDigger didn't see anything peculiar, but below is the EXIF seen by RawDigger if anyone else knows what to look for. FastRawViewer also handled it perfectly.

I have no suggestions apart from trying a different SD card and also a different lens and if still faulty camera needs to be returned for exchange or repair.

Regards..... Guy

---- ExifTool ----
ExifTool Version Number : 10.67
---- File ----
File Name : test image PB270051.ORF
Directory : C:/Users/Guy/Downloads
File Size : 14 MB
File Modification Date/Time : 2017:12:01 05:58:14+11:00
File Access Date/Time : 2017:12:01 05:57:50+11:00
File Creation Date/Time : 2017:12:01 05:58:14+11:00
File Permissions : rw-rw-rw-
File Type : ORF
File Type Extension : orf
MIME Type : image/x-olympus-orf
Exif Byte Order : Little-endian (Intel, II)
---- EXIF ----
Image Width : 4640
Image Height : 3472
Bits Per Sample : 16
Compression : Uncompressed
Photometric Interpretation : BlackIsZero
Image Description : OLYMPUS DIGITAL CAMERA
Make : OLYMPUS CORPORATION
Camera Model Name : E-M10 Mark III
Strip Offsets : 1507328
Orientation : Horizontal (normal)
Samples Per Pixel : 1
Rows Per Strip : 3472
Strip Byte Counts : 13494163
X Resolution : 350
Y Resolution : 350
Planar Configuration : Chunky
Resolution Unit : inches
Software : Version 1.0
Modify Date : 2017:11:27 09:28:41
Artist : Yuriy Kobchenko
Copyright :
Exposure Time : 1/60
F Number : 1.7
Exposure Program : Aperture-priority AE
ISO : 1600
Sensitivity Type : Standard Output Sensitivity
Exif Version : 0231
Date/Time Original : 2017:11:27 09:28:41
Create Date : 2017:11:27 09:28:41
Offset Time : :
Offset Time Original : :
Offset Time Digitized : :
Exposure Compensation : 0
Max Aperture Value : 1.7
Light Source : Unknown
Flash : On, Did not fire
Focal Length : 25.0 mm
User Comment :
Flashpix Version : 0100
Color Space : sRGB
File Source : Digital Camera
CFA Pattern : [Red,Green][Green,Blue]
Custom Rendered : Normal
Exposure Mode : Auto
White Balance : Auto
Digital Zoom Ratio : 1
Scene Capture Type : Standard
Gain Control : High gain up
Contrast : Normal
Saturation : Normal
Sharpness : Normal
Lens Info : 25mm f/1.7
Lens Model : LUMIX G 25/F1.7
GPS Version ID : 2.3.0.0
---- MakerNotes ----
Thumbnail Image : (Binary data 9441 bytes, use -b option to extract)
Special Mode : Normal, Sequence: 0, Panorama: (none)
Camera ID : OLYMPUS DIGITAL CAMERA
Equipment Version : 0100
Camera Type 2 : E-M10MarkIII
Serial Number : BHXA04194
Internal Serial Number : 4189706006220001
Focal Plane Diagonal : 21.6 mm
Body Firmware Version : 1.001
Lens Type : Lumix G 25mm F1.7 Asph.
Lens Serial Number : 07CX4268755B
Lens Firmware Version : 1.000
Max Aperture At Min Focal : 1.7
Max Aperture At Max Focal : 1.7
Min Focal Length : 25
Max Focal Length : 25
Max Aperture : 1.8
Lens Properties : 0x4100
Extender : None
Extender Serial Number :
Extender Model :
Extender Firmware Version : 0
Conversion Lens :
Flash Type : None
Flash Model : None
Flash Firmware Version : 0
Flash Serial Number :
Camera Settings Version : 0100
Preview Image Valid : Yes
Preview Image Start : 52224
Preview Image Length : 984021
AE Lock : Off
Metering Mode : ESP
Exposure Shift : 0
Macro Mode : Off
Focus Mode : Single AF; S-AF, Imager AF
Focus Process : AF Used; 64
AF Search : Ready
AF Areas : (118,119)-(137,136)
AF Point Selected : (58%,46%) (58%,46%)
AF Fine Tune : Off
AF Fine Tune Adj : 0 0 0
Flash Mode : Fill-in
Flash Exposure Comp : 0
Flash Remote Control : Off
Flash Control Mode : Off; 0; 0; 0
Flash Intensity : n/a (x4)
Manual Flash Strength : n/a (x4)
White Balance 2 : Auto
White Balance Temperature : Auto
White Balance Bracket : 0 0
Custom Saturation : 0 (min -5, max 5)
Modified Saturation : Off
Contrast Setting : 0 (min -5, max 5)
Sharpness Setting : 0 (min -5, max 5)
Scene Mode : Standard
Noise Reduction : Auto
Distortion Correction : Off
Shading Compensation : Off
Gradation : Normal; User-Selected
Picture Mode : Natural; 2
Picture Mode Saturation : 0 (min -2, max 2)
Picture Mode Contrast : 0 (min -2, max 2)
Picture Mode Sharpness : 0 (min -2, max 2)
Picture Mode BW Filter : n/a
Picture Mode Tone : n/a
Noise Filter : Standard
Art Filter : Off; 0; 0; 0
Picture Mode Effect : Standard
Tone Level : Highlights; 0; -7; 7; Shadows; 0; -7; 7; Midtones; 0; -7; 7; 0; 0; 0; 0; 0; 0; 0; 0; 0; 0; 0; 0; 0; 0; 0; 0
Art Filter Effect : Off; 0; 0; Partial Color 0; No Effect; 0; No Color Filter; 0; 0; 0; 0; 0; 0; 0; 0; 0; 0; 0; 0; 0
Color Creator Effect : Color 0; 0; 29; Strength 0; -4; 3
Monochrome Profile Settings : No Filter; 0; 8; Strength 2; 0; 3
Film Grain Effect : Off
Color Profile Settings : Min -5; Max 5; Yellow 0; Orange 0; Orange-red 0; Red 0; Magenta 0; Violet 0; Blue 0; Blue-cyan 0; Cyan 0; Green-cyan 0; Green 0; Yellow-green 0
Monochrome Vignetting : 0
Monochrome Color : (none)
Drive Mode : Single Shot; Anti-shock 0
Panorama Mode : Off
Image Stabilization : On, Mode 4
Stacked Image : No
Manometer Pressure : 0 kPa
Manometer Reading : -999.9 m, -999.9 ft
Extended WB Detect : Off
Roll Angle : -1.7
Pitch Angle : -6.2
Date Time UTC : 2017:11:27 14:28:41
Raw Dev Version : 0100
Raw Dev Exposure Bias Value : 0
Raw Dev White Balance Value : 0
Raw Dev WB Fine Adjustment : 0
Raw Dev Gray Point : 0 0 0
Raw Dev Saturation Emphasis : 0 0 0
Raw Dev Memory Color Emphasis : 0
Raw Dev Contrast Value : 0 0 0
Raw Dev Sharpness Value : 0 0 0
Raw Dev Color Space : sRGB
Raw Dev Engine : High Speed
Raw Dev Noise Reduction : (none)
Raw Dev Edit Status : Original
Raw Dev Settings : (none)
Image Processing Version : 0112
WB RB Levels : 434 546 256 256
WB RB Levels 3000K : 290 910 256 256
WB RB Levels 3300K : 346 716 256 256
WB RB Levels 3600K : 368 656 256 256
WB RB Levels 3900K : 390 608 256 256
WB RB Levels 4000K : 474 708 256 256
WB RB Levels 4300K : 418 570 256 256
WB RB Levels 4500K : 468 570 256 256
WB RB Levels 4800K : 448 522 256 256
WB RB Levels 5300K : 472 484 256 256
WB RB Levels 6000K : 508 438 256 256
WB RB Levels 6600K : 540 462 256 256
WB RB Levels 7500K : 550 380 256 256
WB RB Levels CWB1 : 382 384 256 256
WB RB Levels CWB2 : 382 384 256 256
WB RB Levels CWB3 : 382 384 256 256
WB RB Levels CWB4 : 382 384 256 256
Color Matrix : 432 -144 -32 -72 384 -56 14 -132 374
Black Level 2 : 249 250 250 249
Gain Base : 256
Valid Bits : 12 0
Crop Left : 8 0
Crop Top : 8 0
Crop Width : 4608
Crop Height : 3456
Sensor Calibration : 0 256
Noise Reduction 2 : (none)
Distortion Correction 2 : Off
Shading Compensation 2 : Off
Multiple Exposure Mode : Off; 1
Aspect Ratio : 4:3
Aspect Frame : 0 0 4607 3455
Faces Detected : 0 0 0
Face Detect Area : (Binary data 383 bytes, use -b option to extract)
Max Faces : 8 8 0
Face Detect Frame Size : 640 480 640 480 0 0
Face Detect Frame Crop : 0 0 640 480 0 0 640 480 0 0 0 0
Camera Temperature : 27
Keystone Compensation : Off
Focus Info Version : 0100
Scene Detect : 0
Zoom Step Count : 0
Focus Step Count : 2506
Focus Step Infinity : 353
Focus Step Near : 5203
Focus Distance : 0.47 m
AF Point : Left (or n/a)
External Flash : Off
External Flash Bounce : Bounce or Off
External Flash Zoom : 0
Internal Flash : Off
Manual Flash : Off
Macro LED : Off
Sensor Temperature : 22 22 0 C
Preview Image : (Binary data 984021 bytes, use -b option to extract)
---- Composite ----
Aperture : 1.7
Blue Balance : 2.132813
Extender Status : Not attached
Image Size : 4640x3472
Lens ID : Lumix G 25mm F1.7 Asph.
Megapixels : 16.1
Red Balance : 1.695313
Scale Factor To 35 mm Equivalent : 2.0
Shutter Speed : 1/60
Circle Of Confusion : 0.015 mm
Depth Of Field : 0.017 m (0.462 - 0.479 m)
Field Of View : 37.6 deg (0.32 m)
Focal Length : 25.0 mm (35 mm equivalent: 50.1 mm)
Hyperfocal Distance : 24.51 m
Light Value : 3.4

................................................................................................

Guy Parsons said:

I tried the toy picture and sure enough here downunder the same as you get when I click the full resolution icon in OV3 - histogram shrinks and contrast lost.

Click to expand...

Hello Guy! Your help with confirming the issue and the additional details is really helpful. Thank you!

Guy Parsons said:

Same effect with GPU on or off, again a recent Lenovo notebook. Same effect with OV3 on a homebuilt desktop i5 cpu on Asus motherboard.

DxO Photolab no help as not yet updated to include the Mk3.

RawDigger didn't see anything peculiar, but below is the EXIF seen by RawDigger if anyone else knows what to look for. FastRawViewer also handled it perfectly.

I have no suggestions apart from trying a different SD card and also a different lens and if still faulty camera needs to be returned for exchange or repair.

Click to expand...

I tried a different SD card, which came with the camera bundle from OLYMPUS -- same issues.

I also tried different output modes (RAW by itself, N+RAW, F+RAW), again with the same issues.

So if I don't get a definitive answer from Olympus, I'll have to send back this camera and order another one. I like how it fits in the hand and the controls.

But these RAW issues are a serious hassle and a deal breaker, if they cannot be resolved. (If people at OLYMPUS are reading.)

Guy Parsons said:

Regards..... Guy

---- ExifTool ----
ExifTool Version Number : 10.67
---- File ----
File Name : test image PB270051.ORF

Click to expand...

The EXIF information was good to see. This is helpful for others to see the shooting parameters, which are mostly basic standard in A mode with F+RAW output.

I also compared field by field with the DPReview good images, and did not find any deviations in any parameters. The same variations were seen in my files in DPreview files (e.g. Gradation : Normal; User-Selected, but also Auto for the same field in another file etc.). Also if there is some sort of corruption or deviation in RAW data, it wouldn't be necessarily reflected in EXIF, since they are separate sections in the file.

b8r said:

Hello!

I am experiencing issues with RAW files from OLYMPUS E-M10 Mark III, when they are edited in OLYMPUS Viewer 3: contrast drops, shrinking histogram. They also crash SilkyPix 8 Pro (cannot open them and hangs).

You can download the sample images that cause the issues

https://drive.google.com/open?id=1C22UO3i2FTOuGLM3zrSLfpoGvBiMbLqy

PB270023.ORF, PB270028.ORF, PB270051.ORF

Here are the issues in OLYMPUS Viewer 3, which effectively prevent any editing or even high quality viewing of any RAW images. The same issues persist on Export.

1. In View mode, Selecting “Display RAW files at the highest quality” (after double-click in browser).

(See camera and image parameters in Properties on the right).



After that image is distorted and contrast drops. Even though here the histogram does not change, it effectively shrinks dynamic range (just as in the Edit and RAW cases below).



2. In Edit mode, after applying the tool, the image is distorted, and the Histogram shrinks showing how dynamic range is reduced.



After the tool is applied, and it goes to background processing (Updating …%), the contrast drops, and the Histogram shrinks more than half way.



3. In RAW Development mode just clicking “Display RAW files at the highest quality” icon or using any of the tools in RAW1, RAW2 tool trays will cause the image distortion by reducing the contrast dramatically, which is reflected in the shrunk Histogram.

If anyone has some ideas what could be wrong with the RAW files produced by this camera, it will be greatly appreciated. I reached out to OLYMPUS customer support, but after several days of talking and email, they did not offer any solution.

Note 1: these images do open in Lightroom CC, but I want to make sure they work with all supported software, esp. "native" OLYMPUS software.

Note 2: I checked RAW images from the same camera used in DPReview web review:

https://www.dpreview.com/reviews/olympus-om-d-e-m10-mark-iii/6

And they had NO issues in Viewer 3. So I am suspecting the issue may be with the camera, or Viewer 3 not being able to handle the latest camera format. But since they also crash SilkyPix, I am suspecting issues with the camera itself.

I did perform camera Reset, and Pixel Mapping - no change.

Click to expand...

I checked the raw files. The difference in your camera vs. the one DPR stuff used is in tag 0x0805. Your camera has 0 in place of highlights limit (first of the two values recorded in the tag; second is midtone calibration for raw), while the other camera has the real value (4093). If the software expects a real value for highlights limit, and instead receives 0, the resulting rendition will be very odd.

On a side note, Olympus calibrates the exposure meter to place midtone at 6.25% , leaving 4 stops headroom in highlights. If you will check the raw histogram you will see that your raws are strongly underexoposed.

--

Iliah Borg said:

I checked the raw files. The difference in your camera vs. the one DPR stuff used is in tag 0x0805. Your camera has 0 in place of highlights limit (first of the two values recorded in the tag; second is midtone calibration for raw), while the other camera has the real value (4093). If the software expects a real value for highlights limit, and instead receives 0, the resulting rendition will be very odd.

On a side note, Olympus calibrates the exposure meter to place midtone at 6.25% , leaving 4 stops headroom in highlights. If you will check the raw histogram you will see that your raws are strongly underexoposed.

--
http://www.libraw.org/

Click to expand...

Hello Iliah, You are dead on. That was the issue.

I used exiftool to change the offending Maximum value in the tag:

>> 0x0805 SensorCalibration int16s[2] (2 numbers: 1. Recommended maximum, 2. Calibration midpoint)

After that, the RAW file is fixed. It works fine in OLYMPUS Viewer 3 and in SilkyPix.

(Guy's EXIF listing of course also included this line: Sensor Calibration : 0 256)

So the next question would be: how to fix it in the camera, so it populates correct calibration values in the EXIF?

I already performed the Reset and Pixel Mapping from the menu, but that didn't help.

I'll talk to OLYMPUS support again, to see if they know.

b8r said:

That was the issue.

Click to expand...

Happy to help. Based on your screenshots, I knew what to look for

b8r said:

I used exiftool to change the offending Maximum value in the tag:

>> 0x0805 SensorCalibration int16s[2] (2 numbers: 1. Recommended maximum, 2. Calibration midpoint)

Click to expand...

If you look at Olympus.pm in /Image-ExifTool/lib/Image/ExifTool, there is a comment:

0x0805 => { #IB
Name => 'SensorCalibration',
Notes => '2 numbers: 1. Recommended maximum, 2. Calibration midpoint',
Writable => 'int16s',
Count => 2,
},

Incidentally, 'IB' are my initials

b8r said:

(Guy's EXIF listing of course also included this line: Sensor Calibration : 0 256)

Click to expand...

Right.

b8r said:

So the next question would be: how to fix it in the camera, so it populates correct calibration values in the EXIF?

Click to expand...

Try re-programming firmware

b8r said:

I'll talk to OLYMPUS support again, to see if they know.

Click to expand...

Maybe tell Olympus to include a fallback to 4095 if the value is zero

Iliah Borg said:

Iliah Borg said:

That was the issue.

Click to expand...

Happy to help. Based on your screenshots, I knew what to look for

Iliah Borg said:

I used exiftool to change the offending Maximum value in the tag:

>> 0x0805 SensorCalibration int16s[2] (2 numbers: 1. Recommended maximum, 2. Calibration midpoint)

Click to expand...

If you look at Olympus.pm in /Image-ExifTool/lib/Image/ExifTool, there is a comment:

0x0805 => { #IB
Name => 'SensorCalibration',
Notes => '2 numbers: 1. Recommended maximum, 2. Calibration midpoint',
Writable => 'int16s',
Count => 2,
},

Incidentally, 'IB' are my initials

Click to expand...

Tip of the hat !

I'd say that's a nice use case for your enhancement, and your efforts came very handy.

Iliah Borg said:

Iliah Borg said:

(Guy's EXIF listing of course also included this line: Sensor Calibration : 0 256)

Click to expand...

Right.

Iliah Borg said:

So the next question would be: how to fix it in the camera, so it populates correct calibration values in the EXIF?

Click to expand...

Try re-programming firmware

Iliah Borg said:

I'll talk to OLYMPUS support again, to see if they know.

Click to expand...

Maybe tell Olympus to include a fallback to 4095 if the value is zero

--
http://www.libraw.org/

Click to expand...

Since it's a new release, there's no official firmware to be applied. But we'll see what support can offer.

I'll pass on to them the idea for the OV3 software. But it still wouldn't help other RAW software.

Interesting observation: this number 4095 is 0x0FFF or 12 bits in binary, which corresponds to another tag:

• MakerNotes:Olympus:Camera - 0x0611 Valid Bits : 12 0

Even though in another tag it says:

• EXIF:IFD0:Image - 0x0102 Bits Per Sample : 16

which probably more a storage parameter than image data, which is again -- 12 bits, and that should be sufficient for this kind of sensor:

• Raw bit depth is about dynamic range