X10: No RAW converter with full EXR support?

[Timur Born]

Hi guys!

Today I tested on Windows: Lightroom 3, Lightroom 4 (LR) Beta, RAW Converter EX (EX), Silkypix 4, Silkypix (SP) 5 Beta, Capture One 6 (CO).

I have yet to test: RPP (OS X) and whatever other RAW converter claiming to support the X10. Also there are several things I did not find time to test yet (12 MP DR 200/400, 6 MP DR 100 OOC vs. RAW detail).

Compared to the out-of-camera JPGs none of these RAW converters fully supports proper decoding of the EXR RAW data, neither in 12 MP nor in 6 MP, especially when DR 200/400 is used!

Some general observations about X10 RAF files that I learned from this comparison, like always these are my personal observations and interpretations:

OOC JPGs are lens (distortion) corrected! Both Lightroom and Silkypix also do correction but stretch the image slightly horizontally (only), both seem to apply the same correction (needs to be tested with a grid image). Capture One does not correct images and thus not only shows considerable distortion, but also more image content at all corresponding edges. Yes that means you can see more on CO images than on OOC JPGs!

Both 12 MP and 6 MP images (DR 100-400) store all information from both sensor halves in the RAW file which then needs to be properly decoded and combined by the RAW software. This is where all tested RAW converters fail compared to OOC JPG!

Only 6 MP RAWs of 9 mb filesize already contain the combined sensor information and thus don't rely on the RAW software for decoding. With DR 200/400 the 9 mb files are already noise-reduced and offer full color saturation (see below) in all RAW converters that can open them. CO does not seem to be able to read these, it gets stuck.

It also seems like ISO based DR indeed uses only half the sensor for ISO/amplification tricks, which then has to be properly combined with the overexposed half just like EXPosure time based DR. This is still tricky to interpret, though, as loss of saturation might also come from misinterpreting only half the sensor even when both offer the same information. On the other hand DR 100 18 mb files saturate fine, could be changing binning algorithms with different DR source files, but I somewhat doubt that.

 

[Timur Born]

Lightroom:

• Overall best result, but problematic with DR 200/400 where it seems to match the levels of both halves wrong (and offers to control over the process). Tends to push reds towards orange and slightly over-saturate blue, even with proper white-balance setting.

• 12 MP images are listed as full 4000x3000 resolution but offer somewhat less detail than OOC JPG 12 MP, but more detail than OOC JPG 6 MP. Sharpening doesn't help to match OOC JPG detail, but as a benefit some specific EXR artifacts on distinct corners are not present. Because of the image being horizontally stretched slightly more, there might be some areas with better detail (more room).

• 19 mb 6 MP DR 200/400 files suffer from a very considerable loss of saturation with orange/red colors plus lesser loss with other colors, regardless of whether EXPosure time or ISO based DR is used.

• EXP DR highlight motion artifacts from the longer exposed sensor half are shifted to blue/purple instead of being white/grey. ISO DR highlights also get blue/purple pixels in their outline instead of them being greyish, this leads to stepping pixels at the borders (especially with orbs) and an even more salient dark outline.

• 6 MP DR 100 images are properly saturated, likely because both sensor halves contain more or less the same information and can thus easily be combined (no proper balancing needed).

• Least patterned noise, but you need to watch your sharpening setting (default of 25 introduces lots of noise and stronger orb outlines already).

Silkypix / RAW Converter EX: (only differences compared to LR)

• Not so much worse than Lightroom to begin with, but seems to merge the sensor halves even less balanced.

• Introduces more "patterned" noise than LR, which seems to match the OOC JPG diagonal labyrinth noise closest.

• Overall suffers from the same problems as lightroom on all points, but may lose even more saturation (up to a point where parts of the image look like Sepia filtered). Interestingly RAW Converter EX is worse than Silkypix in this regard. SP 5 can only be used with "V4 compatible" color settings, else the image get a distinct greenish tint even at proper white-balance settings and that can hardly be dialed out.

• 12 MP files are listed as 6 MP files, but nevertheless still properly opened and converted at full 4000x3000 resolution. Same things apply for detail as with LR.

• 6 MP EXP DR sensor halves are either very differently balanced or information from the longer exposed half is nearly completely thrown away. The final image still looks like a full 6 MP image (in contrast to CO, see below), but the exposure/color information of the longer exposed half is not merged with the lower exposed half like with OOC JPGs and LR. This is very apparent with moving objects (especially highlights).

Capture One:

• Is useless at its current state! It literally throws away half the sensor's information both from 12 MP and 6 MP files as if they don't exist. This leads to visible artifacts that shows as strangely jaggy outlines (especially in diagonals) and seemingly missing pixels (like yellow text on a black book showing black pixels where yellow ones should be). Consequently displays a 12 MP file as only having 6 MP.

• EXP DR images look opposite to SP ones in that only the longer exposed half of the sensor seems to be used, with the exception of highlights that would be clipped to full white in the shorter exposed half. Interestingly CO shows that clipped highlights of the longer half still contain rescue-able information by decreasing exposure by some stops, both OOC JPGs and LR only seem to decrease brightness before merging, though (OOC JPG turning white to grey, LR turning white to blue/purple).

• Images are not lens corrected and thus distorted, but do contain more information than even OOC JPGs! Unfortunately you cannot even activate manual controls to tame things like barrel distortion at the wide end of the lens etc.

• Can introduce very patterned noise of connected horizontal and vertical labyrinth lines, whereas OOC labyrinths are more vertical and not connected. May be some kind of moire artifact, but SP also seems to keep some broken rests of this characteristic.

Here is the result of what happens when CO tries to build a 6 MP ISO DR 400 image with half the information missing. The crop shows horizontal window blinds, sharpened and resized to 500% via nearest neighbor (CO vs. SP vs. LR):

 

[VictorZ]

Timur,

Great stuff. Very informative and enlightening. Did you by chance try Adobe's DNG Converter? I don't know if it's the same as LR/ACR but since I don't have LR and still use CS4, this is the only way for me to get RAW files into Photoshop. The results are reasonably good, but I haven't played extensively with the RAW files ... I'm still getting acquainted with all the ins and outs of using the X10. I use RAW all the time with my DSLR photos, but conditions haven't been the greatest for getting out and shooting much other than tests, snapshots, etc. with the X10. As the JPG performance is pretty decent, I have been leaving RAW testing aside for the moment.

Many thanks again for all you're doing!
--
Victor Z

 

[Trevor G]

Compared to the out-of-camera JPGs none of these RAW converters fully supports proper decoding of the EXR RAW data, neither in 12 MP nor in 6 MP, especially when DR 200/400 is used!

That's an extravagantly big claim to make without presenting any evidence.

I see no evidence when I compare JPEG with RAW from my shots.

Under what circumstances is the "missing" information visible in JPEG and not in RAW?

--
Cheers ;-)

Trevor G

Silkypix tutorials at: http://photo.computerwyse.com

 

[Luego]

Hi Timor,

Again, I have to comment you for your effort, especially posting all your test results
during the "graveyard-shift," your time.... Get some sleep, my friend!

Since I'm not a PP-person, I will file all this juicy information of yours for the future.
It might come in handy, when I'm ready to tweak my RAW images.

In the mean time, I'll be anxiously awaiting your detailed test results comparing L versus M size.

Happy images

Luego

 

[Stephen Werst]

OOC JPGs are lens (distortion) corrected! Both Lightroom and Silkypix also do correction but stretch the image slightly horizontally (only), both seem to apply the same correction (needs to be tested with a grid image). Capture One does not correct images and thus not only shows considerable distortion, but also more image content at all corresponding edges. Yes that means you can see more on CO images than on OOC JPGs!

Yes, according to the post at http://forums.dpreview.com/forums/read.asp?forum=1012&message=40099085 all seem to be correcting lens distortions, but the noted "stretching" along the long axis of the frame is actually the SW converters showing a flaw in the JPG engine, namely that at the wide end of the zoom the camera compresses the JPG image along the long axis. This is evident by comparing images of a grid as noted by other posts in the thread above. There's also a note on how to make Silkypix show more of the image than it does by default.

Both 12 MP and 6 MP images (DR 100-400) store all information from both sensor halves in the RAW file which then needs to be properly decoded and combined by the RAW software. This is where all tested RAW converters fail compared to OOC JPG!

You should probably qualify this as 6MP images under some camera settings, otherwise you contradict yourself in the next paragraph shown below. When shooting RAW+JPG the RAF files for the M size images will be the larger 19 MB file as long as the IS0

Only 6 MP RAWs of 9 mb filesize already contain the combined sensor information and thus don't rely on the RAW software for decoding. With DR 200/400 the 9 mb files are already noise-reduced and offer full color saturation (see below) in all RAW converters that can open them. CO does not seem to be able to read these, it gets stuck.

 

[AdamJRed]

Timur,

Try These Sharpening and NR settings (in LR or ACR/Bridge) on a 12Mp RAF and see if you still think that, as I can now get more detail from my 12Mp RAF than any other OOC Jpg or 6Mp RAF...

Sharpening:
Amount 110
Radius 0.6
Detail 10 (Try 15-20 if using ISO 100-200)
Masking 45 (Increase to 65 as you reach ISO 3200)

NR:
Luminance 10 ( Increase to 15 when ISO > 1000 )
Detail 80
Contrast 35

Color 10 (Increase to 15 when ISO > 1000)
Detail 30

Compared to the out-of-camera JPGs none of these RAW converters fully supports proper decoding of the EXR RAW data, neither in 12 MP nor in 6 MP, especially when DR 200/400 is used!

That's an extravagantly big claim to make without presenting any evidence.

I see no evidence when I compare JPEG with RAW from my shots.

Under what circumstances is the "missing" information visible in JPEG and not in RAW?

--
Cheers ;-)

Trevor G

Silkypix tutorials at: http://photo.computerwyse.com

 

[Mr Sincere]

Thanks for the helpful post.

For what it's worth, I loaded up about a dozen RAF files in Raw Photo Processor tonight, of 12mp and 6mp files, in DR100 and DR400, and they all loaded with dimensions of 2944x2144. This was reflected in the EXIF values for RawImageFullSize and ImageWidth/ImageHeight. But, to my highly untrained eyes, the files at least looked sharper than anything I'd seen in Lightroom. If it didn't sound completely crazy, I'd start to wonder if Lightroom was actually upsampling a 6mp image up to 12mp for L size RAW files. Would at least explain the softness and artifacts.

Definitely some RAW strangeness going on. It's not like it's just one RAW converter that's having some issues...

 

[Timur Born]

For what it's worth, I loaded up about a dozen RAF files in Raw Photo Processor tonight, of 12mp and 6mp files, in DR100 and DR400, and they all loaded with dimensions of 2944x2144. This was reflected in the EXIF values for RawImageFullSize and ImageWidth/ImageHeight.

2944x2144 is the dimension of the RAW embedded JPG preview! EXIF seems to show information of the preview only.

But, to my highly untrained eyes, the files at least looked sharper than anything I'd seen in Lightroom. If it didn't sound completely crazy, I'd start to wonder if Lightroom was actually upsampling a 6mp image up to 12mp for L size RAW files. Would at least explain the softness and artifacts.

I did a comparison and Lightroom definitively has more detail than an upsampled 6 MP image. It's at the very heart of this thread that the full 12 MP information is used by both the camera and Lightroom to create both 12 MP and 6 MP. Silkypix does, too, while CO does not. With DR 200/400 they all more or less fail to produce good output.

 

[Timur Born]

Timur,

Great stuff. Very informative and enlightening. Did you by chance try Adobe's DNG Converter?

Glad you can use it. I don't own the DNG converter or Photoshop, but just got the trials/betas of all versions I tested here. Does it even support Fuji RAF files?

 

[GaryJP]

Capture One does not correct images and thus not only shows considerable distortion, but also more image content at all corresponding edges. Yes that means you can see more on CO images than on OOC JPGs!

I can't get Capture One to do anything to correct barrel distortion on the X10. The slider does nothing. Have you seen a way round this?

--

“There is only you and your camera. The limitations in your photography are in yourself, for what we see is what we are.” Ernst Haas

http://garyp.zenfolio.com/p518883873/

 

[Timur Born]

That's an extravagantly big claim to make without presenting any evidence.
I see no evidence when I compare JPEG with RAW from my shots.

Depends on what software you convert your RAWs with. Like I wrote both Lightroom and Silkypix also do lens correction.

Here is a comparison. The red boxed are perfect squares that I drew on top of the already converted RAWs.





















Interestingly the OOC JPG shows the horizontal/vertical labyrinth pattern in this shot while the Lightroom RAW shows the "Fuji typical" vertical labyrinth pattern in reds, purple and blue (I have another shot where it's in cyan, too). I suspect this to be moire/anti-aliasing artifacts.

 

[Timur Born]

Yes, according to the post at http://forums.dpreview.com/forums/read.asp?forum=1012&message=40099085 all seem to be correcting lens distortions, but the noted "stretching" along the long axis of the frame is actually the SW converters showing a flaw in the JPG engine, namely that at the wide end of the zoom the camera compresses the JPG image along the long axis.

Thanks for the work on that. Indeed the JPG is compressed, but it also does best at getting distorted lines straight. Take a look at the comparison images I just posted in this thread. Lightroom and EX (SP4) do quite well, too, SP5 not so much.

You should probably qualify this as 6MP images under some camera settings, otherwise you contradict yourself in the next paragraph shown below. When shooting RAW+JPG the RAF files for the M size images will be the larger 19 MB file as long as the IS0

No contradiction here. When the M size RAW is 19 mb then the RAW converter has to do the job of binning/blending both sensor halves together, when the M size RAW is 10 mb (as per RAW+JPG+DR setting) then the halves are already combined inside the RAW and the converter has one error less to make.

It is specifically the 19 mb RAWs that all converters fail to do properly. The loss of color saturation is considerable and cannot be dialed back in (you cannot turn greyish/sepia red back to red just by pulling a saturation lever).

 

[Timur Born]

I can't get Capture One to do anything to correct barrel distortion on the X10. The slider does nothing. Have you seen a way round this?

No, CO identifies the "file" as unsupported and thus neither does automatic lens correction nor allows manual correction (as I wrote before ). As a "positive" side-effect it allows us to judge the strength of the in-camera JPG and other RAW converter's automatic corrections.

 

[GaryJP]

--

“There is only you and your camera. The limitations in your photography are in yourself, for what we see is what we are.” Ernst Haas

http://garyp.zenfolio.com/p518883873/

 

[Mark H]

For what it's worth, I loaded up about a dozen RAF files in Raw Photo Processor tonight, of 12mp and 6mp files, in DR100 and DR400, and they all loaded with dimensions of 2944x2144. This was reflected in the EXIF values for RawImageFullSize and ImageWidth/ImageHeight.

2944x2144 is the dimension of the RAW embedded JPG preview! EXIF seems to show information of the preview only.

My investigations reveal...

• The RAF files do appear to record '2 x 2944 x 2144' (12.6MP) of pixel data from the sensor.

• However, the number of 'active' image pixels appears to be ' 2 x 2896 x 2144 ' (12.4 MP).

[As a side note: The sensor's diagonally arranged '2 x 2896 x 2144' active pixels have a 'pixels per unit area density' equivalent to that of a more conventional sensor/image layout of about '4095 x 3032' - hence the camera's chosen 4000 x 3000 image output I suppose.]

 

[Timur Born]

Try These Sharpening and NR settings (in LR or ACR/Bridge) on a 12Mp RAF and see if you still think that, as I can now get more detail from my 12Mp RAF than any other OOC Jpg or 6Mp RAF...

Thanks, I did and indeed these settings almost get out the same detail as the OOC JPG, so it's in there (albeit still slightly less). Unfortunately the "price" is high, but see for yourself:





Some intermediate sharpening setting should get you there, but at least for my test-shot there still remain some details that get lost in the Lightroom RAW (especially some spaces in letter, like between the arcs of a "m" outside this shot, or the inside of the "a" of the Uhu Pl"a"st bottle on this shot).

 

[Timur Born]

For what it's worth, I loaded up about a dozen RAF files in Raw Photo Processor tonight, of 12mp and 6mp files, in DR100 and DR400, and they all loaded with dimensions of 2944x2144. This was reflected in the EXIF values for RawImageFullSize and ImageWidth/ImageHeight.

2944x2144 is the dimension of the RAW embedded JPG preview! EXIF seems to show information of the preview only.

My investigations reveal...

• The RAF files do appear to record '2 x 2944 x 2144' (12.6MP) of pixel data from the sensor.

• However, the number of 'active' image pixels appears to be ' 2 x 2896 x 2144 ' (12.4 MP).

[As a side note: The sensor's diagonally arranged '2 x 2896 x 2144' active pixels have a 'pixels per unit area density' equivalent to that of a more conventional sensor/image layout of about '4095 x 3032' - hence the camera's chosen 4000 x 3000 image output I suppose.]

I am sorry, I mixed things up and was wrong in this. The preview JPG is 2048 x 1536 pixels. Again sorry for the unnecessary confusion.

Your investigations make sense and RPP likely only makes use of one half of the information. Once I find time I will check RPP myself and compare its output at various DR settings (which allow to at least partially "see" on the final image what happens under the hood).

 

[AdamJRed]

Was that comparison OOC JPG vs RAW, or unsharpened RAW vs sharpened?

Thanks,

Try These Sharpening and NR settings (in LR or ACR/Bridge) on a 12Mp RAF and see if you still think that, as I can now get more detail from my 12Mp RAF than any other OOC Jpg or 6Mp RAF...

Thanks, I did and indeed these settings almost get out the same detail as the OOC JPG, so it's in there (albeit still slightly less). Unfortunately the "price" is high, but see for yourself:





Some intermediate sharpening setting should get you there, but at least for my test-shot there still remain some details that get lost in the Lightroom RAW (especially some spaces in letter, like between the arcs of a "m" outside this shot, or the inside of the "a" of the Uhu Pl"a"st bottle on this shot).

 

[Timur Born]

I think one reason why the OOC JPGs are compressed is to get the nominal 28 mm angle that Fuji is advertising. The Lightroom and Silkypix images retain dimensions better, but at the cost of lower angle. Overall I prefer the Lightroom result.

Hopefully the LR v4. beta runs long enough for DXO to finally come out with their X10 RAW solution for comparison.