EXR Confusion not fully resolved

Started 11 months ago | Discussions
Kim Letkeman
Forum ProPosts: 33,128Gear list
Like?
Re: the data is there when it should be, and not when it should not ...
In reply to Timur Born, 11 months ago

Timur Born wrote:

Kim Letkeman wrote:

Not really hidden ... if there are two halves stored, then the converter follows the dynamic range setting to choose its algorithm ... 100 is binning if M size was shot and create a large image if not

L and M both use two stored halves and can use DR 100, that's why the EXIF of even L sized images lists the M size (half sensor size x 2 frames) resolution. There has to be an indication in the raw file that tells the raw converter whether the shot was meant to be L or M and develop accordingly. The source data is the same anyway, while the development is different (EXR HR demosaicing vs. SN blending + Bayer demosaicing).

Well, when DR100 is used, there is never any difference between the two sets of pixels. Meaning that they are all exposed the same. So you can choose to bin them or use them as a larger image. That can be an after the fact decision, regardless of how the camera was set.

In the camera, this is controlled of course by firmware's response to your settings. But in a RAW converter, the user would issue the command again, since the pixels are perfectly suitable for either use. Of course, Adobe products don't offer a user interface specifically for the EXR sensor. As it is, I don't know if any converter other than perhaps Silkypix has any knowledge whatsoever of the possibilities with that data ...

 Kim Letkeman's gear list:Kim Letkeman's gear list
Nikon Coolpix 990 Fujifilm FinePix F200EXR Fujifilm FinePix F770EXR Panasonic Lumix DMC-GH2 Panasonic Lumix DMC-GF3 +16 more
Reply   Reply with quote   Complain
Timur Born
Senior MemberPosts: 4,222
Like?
Re: the data is there when it should be, and not when it should not ...
In reply to Kim Letkeman, 11 months ago

Kim Letkeman wrote:

Timur Born wrote:

L and M both use two stored halves and can use DR 100... The source data is the same anyway, while the development is different (EXR HR demosaicing vs. SN blending + Bayer demosaicing).

Well, when DR100 is used, there is never any difference between the two sets of pixels. Meaning that they are all exposed the same. So you can choose to bin them or use them as a larger image. That can be an after the fact decision, regardless of how the camera was set.

Two wordings, same content.

In the camera, this is controlled of course by firmware's response to your settings. But in a RAW converter, the user would issue the command again, since the pixels are perfectly suitable for either use.

There is no raw converter that leaves this decision to the user. Those that properly support HR/L size images read the information from the raw file and act accordingly and automatically. So the information must be in there.

Of course, Adobe products don't offer a user interface specifically for the EXR sensor. As it is, I don't know if any converter other than perhaps Silkypix has any knowledge whatsoever of the possibilities with that data ...

Lightroom, Silkypix support both M and L size images, with and without DR. Last time I checked (and remember) CaptureOne only supported M size and did crash if the image was already binned by the camera. On the other hand CO had the best implementation of EXR DR of the three. It's been quite a long time since I did that comparison, so things may have changed for CO and Silkypix, last time I checked LR still had the issue of turning highlights from the clipped half of the X10 sensor towards purple tint.

-- hide signature --

Red flash eyes save lives and eye-sight!
http://en.wikipedia.org/wiki/Retinoblastoma

Reply   Reply with quote   Complain
NIK11
Senior MemberPosts: 2,781
Like?
Re: Here's The "Party Trick" Explained
In reply to Trevor G, 11 months ago

Trevor G wrote:

NIK11 wrote:

Trevor G wrote:

In that way it's a "party trick" in that you can get the same result by taking two separate exposures, 2EV apart, and combining them later on in HDR software.

bAs others have said your statement is a generalisation and undersells what EXR uniquely achieves in jpeg.

I don't think so.

I used to think like you but using it side by side with the X20 made me realise that it didn't have anything special. Not really.

Just today I examined some X10 pics I took early last year. They show me definitely that the minor usability advantage of EXR is blown away by its disadvantages. That's what I found when I shot both cams together over the course of a few weeks. My non-EXR X20 was not left wanting in the X10's wake - in fact, the opposite became obvious.

Here's some of my evidence (watch the histogram too):

1) Find the point at which DR100 doesn't over-expose

2) Check/compare RAW and ooc JPG

3) Find the point at which DR200 doesn't over-expose

4) Check/compare RAW and ooc JPG

5) Find the point at which DR400 doesn't over-expose

6) Check/compare RAW and ooc JPG

Let's see what happens:

Here is some evidence of highlight compression which causes a colour shift and loss of contrast. This isn't random, it's in every series that I have ever shot showing EXR DR.

The only way that I know of that this could occur is through curves-type compression being used to fit the extra highlights into the existing, available image space.

To my mind, that's not good for preserving image integrity; it's the reason I have never used the otherwise-excellent Fuji ooc JPEGs, especially when shooting DR400.

RAW preserves colour fidelity in highlights and lowlights; it's only when you have to fit that same highlight retention inside the available image space that problems occur. In other words, shooting RAW can have its own set of colour fidelity issues as you deal with wide dynamic range material.

These samples were shot with EXR Hardware in operation - in other words: M size, ISO less than DR (actually, base ISO) and DR as marked.

When you correctly expose at DR100 (no sensor "tricks" involved) both RAW and ooc JPEG histograms will match quite well. The minor differences are due to a slight colour profile difference between RAW converter output (Silkypix) and in-camera processing

When you correctly expose at DR100 (no sensor "tricks" involved) both RAW and ooc JPEG histograms will match quite well. The minor differences are due to a slight colour profile difference between RAW converter output (Silkypix) and in-camera processing.

Mind you, the slight build-up in green and blue (and the pointed tops) towards the right hand edge of the JPEG histogram does suggest that the JPEG was still over-exposed ever so slightly - it would have been perfect at -2EV

The screenshots are from Silkypix which allows you to compare 2 images side-by-side, like this.

Note how far the red, green and blue peaks have moved over in the OOC JPEG - this causes an obvious colour shift in the sky and will potentially cause problems in other areas outside of view of this crop

Note how far the red, green and blue peaks have moved over in the OOC JPEG - this causes an obvious colour shift in the sky and will potentially cause problems in other areas outside of view of this crop.

Lowlights are still essentially where they were before.

Maybe DR400 will save the day?

They've managed to stop the blue channel from wandering off the screen (compared to DR200), but red and green are moved up again, which causes more of a colour shift = cyan skies

Unfortunately, while blue's advance to the right has stopped, red and green are still shifting up. This increases the colour shift which leads to cyan skies. I don't like cyan skies but you are welcome to like them.

DR400 still has more highlight retention. In theory it provides 2EV of highlight protection, but because they use curves-type manipulation to keep the highlights under 0EV in the ooc JPEGs, in the end we seem to actually get up around 2.7EV of not-so-accurate, lowered contrast highlight protection.

Oops - where's the ooc JPEG's blue channel gone? If I reduce the output level slightly we will see that it has "climbed the wall" and clipped somewhat.

Oops - where has the ooc JPEG's blue channel highlight gone? If I reduce the exposure by just 0.4EV we can see - once we get that "church steeple" effect we know that we have a badly clipped channel.

Hmmm...absolutely impossible to get back the clipped, blue channel infiormation - fortunately it still exists in the matching RAW file

It's absolutely impossible to get back that clipped, blue channel information unless you have the matching RAW file. I always shoot RAW+.

Trevor,

Now that have had more time to digest the above, let me comment.

Firstly a great presentation showing the relationship between the electronic graph and what you actually see.

Secondly, you've moved the discussion onto your favourite hobby horse - RAW.

Thirdly, I don't think this clearly demonstrates significantly better DR from RAW. But it does show that RAW can sometimes offer more detail and PP choices. In this case you have shown how some of the blue channel gets clipped in jpeg but not RAW. I wonder what would happen if you showed a different scene of say woods/trees with sun streaming through and vey little 'blue' information? Would the blue channel still clip or is it scene specific?

Finally, as has been said before, it could be that Silkypix grabs a bit more data from RAW than does PSE11 that I use.

Whatever the answers, I shall happily stick with jpeg DR400, and you with RAW.

Nick

-- hide signature --

Cheers
Trevor G
Silkypix tutorials at: http://photo.computerwyse.com

Reply   Reply with quote   Complain
Kim Letkeman
Forum ProPosts: 33,128Gear list
Like?
Re: the data is there when it should be, and not when it should not ...
In reply to Timur Born, 11 months ago

Timur Born wrote:

Lightroom, Silkypix support both M and L size images, with and without DR. Last time I checked (and remember) CaptureOne only supported M size and did crash if the image was already binned by the camera. On the other hand CO had the best implementation of EXR DR of the three. It's been quite a long time since I did that comparison, so things may have changed for CO and Silkypix, last time I checked LR still had the issue of turning highlights from the clipped half of the X10 sensor towards purple tint.

The 2/3" sensor is poorly supported for RAW. I have not noticed that specific issue, but I have noticed that getting images to look sharp is much more difficult than it should be. To me, it is never worth shooting the 2/3" sensor in RAW, as the JPEG engines often do a better job than can be accomplished with Lightroom.

On the other hand, the XTrans and 1/2" EXR sensors do not suffer that problem in my experience.

 Kim Letkeman's gear list:Kim Letkeman's gear list
Nikon Coolpix 990 Fujifilm FinePix F200EXR Fujifilm FinePix F770EXR Panasonic Lumix DMC-GH2 Panasonic Lumix DMC-GF3 +16 more
Reply   Reply with quote   Complain
Timur Born
Senior MemberPosts: 4,222
Like?
Re: the data is there when it should be, and not when it should not ...
In reply to Kim Letkeman, 11 months ago

Kim Letkeman wrote:

The 2/3" sensor is poorly supported for RAW. I have not noticed that specific issue, but I have noticed that getting images to look sharp is much more difficult than it should be. To me, it is never worth shooting the 2/3" sensor in RAW, as the JPEG engines often do a better job than can be accomplished with Lightroom.

Was harder to get sharp L size images in older LR versions, but easier now. M size is standard Bayer pattern and usually is no problem for sharpness anyway.

On the other hand, the XTrans and 1/2" EXR sensors do not suffer that problem in my experience.

XTrans suffers its own problems in form of ugly artifacts and swapped corner pixels with Lightroom's demosaicing. More sharpness usually means more artifacts, albeit again the behavior has changed to the better with newer versions of LR.

I don't see why 1/2" EXR sensors should work any different from 2/3" ones in Lightroom other than Adobe having implemented something for newer EXR cameras that they forgot to put in for the older ones as well. In any case the newer versions of LR allows L size EXR sharpness very close to JPG output. And LR's demosaicing isn't the best out there anyway, for my Olympus MFT cameras (Bayer) I get more fine resolution detail out of RawTherapee (best) and DxO (second best) than LR (last), didn't check CO and Aperture. LR has a tendency for maze artifacts that then are smeared away by internally applying BayerGreenSplit.

-- hide signature --

Red flash eyes save lives and eye-sight!
http://en.wikipedia.org/wiki/Retinoblastoma

Reply   Reply with quote   Complain
Trevor G
Veteran MemberPosts: 6,473Gear list
Like?
Re: Here's The "Party Trick" Explained
In reply to NIK11, 11 months ago

NIK11 wrote:

Trevor,

Now that have had more time to digest the above, let me comment.

Firstly a great presentation showing the relationship between the electronic graph and what you actually see.

Secondly, you've moved the discussion onto your favourite hobby horse - RAW.

I was actually focusing entirely on the JPEG results, which to my eyes are just not good when using DR above 100 on a high contrast scene (which is what DR was designed for).

Thirdly, I don't think this clearly demonstrates significantly better DR from RAW.

It shows that RAW retains full, uncompressed highlights.

The only way for ooc JPEGs to do that is to under-expose by 1.5 to 2EV

But it does show that RAW can sometimes offer more detail and PP choices. In this case you have shown how some of the blue channel gets clipped in jpeg but not RAW. I wonder what would happen if you showed a different scene of say woods/trees with sun streaming through and vey little 'blue' information? Would the blue channel still clip or is it scene specific?

It's entirely scene and exposure-level specific as to which channel clips first.  Eventually the RAW clips as well.

Finally, as has been said before, it could be that Silkypix grabs a bit more data from RAW than does PSE11 that I use.

Actually PS or ACR handle clipped and crushed highlights better than Silkypix.  I recently had to buy PSE so that I could get a smooth roll-off of clipping highlights on some images I  worked on.

Whatever the answers, I shall happily stick with jpeg DR400, and you with RAW.

Glad you enjoyed the series.  Thanks for dropping in. 

-- hide signature --

Cheers
Trevor G
Silkypix tutorials at: http://photo.computerwyse.com

Reply   Reply with quote   Complain
Timur Born
Senior MemberPosts: 4,222
Like?
Re: Here's The "Party Trick" Explained
In reply to Trevor G, 11 months ago

Did you use default settings with the raw files of different exposures, or did you change highlight settings with each exposure? If one is allowed to work on raw files then one is allowed to use in-camera jpg options.

Did you try to better match the colors/tone-mapping of raw and jpg in order to better match the balance of single color-channels?

-- hide signature --

Red flash eyes save lives and eye-sight!
http://en.wikipedia.org/wiki/Retinoblastoma

Reply   Reply with quote   Complain
Trevor G
Veteran MemberPosts: 6,473Gear list
Like?
Re: Here's The "Party Trick" Explained
In reply to Timur Born, 11 months ago

Timur Born wrote:

Did you use default settings with the raw files of different exposures, or did you change highlight settings with each exposure? If one is allowed to work on raw files then one is allowed to use in-camera jpg options.

I think you misunderstand what I was showing.

Proponents of ooc JPEGs generally do so because they feel they don't need to do any "developing" or adjustments.

In any case, because the highlights are compressed in an EXR-based JPEG it doesn't matter what you do, it won't be easy to regain original colour or contrast.

Did you try to better match the colors/tone-mapping of raw and jpg in order to better match the balance of single color-channels?

I was showing that EXR DR ooc JPEGs are not really usable as they come (in my eyes, anyway) because of the way they are processed. The higher the DR setting, the more compression takes place and the less realistic or "actual" the colours are.

Only the RAW files retain the full colour gamut and exposure lattitude, irrespective of DR setting, at the same exposures.

Of course you can work on the ooc JPEGs, but that defeats the purpose of shooting JPEG, less fuss. And even if you do work on them you cannot restore the natural colours, especially, which are compressed and "curved" away during in-camera processing.

And, of course, there is no way to get back that highlight range which is fully retained in the RAW.

-- hide signature --

Cheers
Trevor G
Silkypix tutorials at: http://photo.computerwyse.com

Reply   Reply with quote   Complain
Timur Born
Senior MemberPosts: 4,222
Like?
Re: Here's The "Party Trick" Explained
In reply to Trevor G, 11 months ago

I didn't write "work on the JPGs", I wrote "use in-camera JPG settings". That's a huge difference. There are settings for highlights and shadows that allow some control over how the raw data is interpreted to JPGs in-camera. Not as sophisticated as with current Olympus MFT cameras, but available to some extend.

People who use their EXR cameras as point & shoot won't care about raw development regardless of the supposed benefits. People who have control over their exposures may benefit from a raw workflow, but the more experienced and skilled they are the less they won't need it. All within the boundaries of the specific camera of course.

-- hide signature --

Red flash eyes save lives and eye-sight!
http://en.wikipedia.org/wiki/Retinoblastoma

Reply   Reply with quote   Complain
Lightpath48
Senior MemberPosts: 2,392Gear list
Like?
Re: Here's The "Party Trick" Explained
In reply to Timur Born, 11 months ago

Timur Born wrote:

I didn't write "work on the JPGs", I wrote "use in-camera JPG settings". That's a huge difference. There are settings for highlights and shadows that allow some control over how the raw data is interpreted to JPGs in-camera. Not as sophisticated as with current Olympus MFT cameras, but available to some extend.

People who use their EXR cameras as point & shoot won't care about raw development regardless of the supposed benefits. People who have control over their exposures may benefit from a raw workflow, but the more experienced and skilled they are the less they won't need it. All within the boundaries of the specific camera of course.

-- hide signature --

Red flash eyes save lives and eye-sight!
http://en.wikipedia.org/wiki/Retinoblastoma

I've concentrated on bringing my X10 and X-S1 jpeg work up to a high success rate, by jiggling highlight and shadow settings. It has been a highly complex undertaking because of the way tonal curves tend to change along with changes in ISO and DR, notwithstanding L and M issues. Generally, I find that softening highlights and shadows in varying degrees can bring me close to comparable results using in-camera raw-to-jpeg conversions after the shoots. It's two ways to achieve the same outcome. Sometimes the softening happens for me in a scene, i.e., as clouds roll in or shooting moves into a building with soft indoor lighting. The variables can explode in a situation like travel photography. Then, if it's not critical shooting I switch to EXR Auto. Outside of the overexposure bug in Auto EXR DR, it rarely fails.

 Lightpath48's gear list:Lightpath48's gear list
Nikon Coolpix P7800 Nikon D3300 Nikon AF-S DX Nikkor 55-200mm f/4-5.6G VR Nikon AF-S DX Nikkor 35mm f/1.8G Nikon AF-S DX Nikkor 18-55mm f/3.5-5.6G VR II
Reply   Reply with quote   Complain
Timur Born
Senior MemberPosts: 4,222
Like?
Re: Here's The "Party Trick" Explained
In reply to Lightpath48, 11 months ago

Juggling the menu system for quick JPG changes isn't too much fun either. Does the newer quickmenu screen include highlight and shadow settings? Never checked for that. The H&S control on the E-M5/E-M1 is superb.

-- hide signature --

Red flash eyes save lives and eye-sight!
http://en.wikipedia.org/wiki/Retinoblastoma

Reply   Reply with quote   Complain
Leo33
Senior MemberPosts: 2,846Gear list
Like?
Now It's Time . . . .
In reply to chimpanzee, 11 months ago
No text.
 Leo33's gear list:Leo33's gear list
Canon PowerShot G11 Fujifilm XF1
Reply   Reply with quote   Complain
Leo33
Senior MemberPosts: 2,846Gear list
Like?
- - - To Take Some Pictures!
In reply to Axel Vercauteren, 11 months ago

Cheers,

Emil

 Leo33's gear list:Leo33's gear list
Canon PowerShot G11 Fujifilm XF1
Reply   Reply with quote   Complain
Trevor G
Veteran MemberPosts: 6,473Gear list
Like?
Re: Here's The "Party Trick" Explained
In reply to Timur Born, 11 months ago

Timur Born wrote:

Juggling the menu system for quick JPG changes isn't too much fun either. Does the newer quickmenu screen include highlight and shadow settings? Never checked for that. The H&S control on the E-M5/E-M1 is superb.

Yes, it works well, although since I don't use JPG I never get to see the results.

But I have shot the examples.

-- hide signature --

Cheers
Trevor G
Silkypix tutorials at: http://photo.computerwyse.com

Reply   Reply with quote   Complain
Lightpath48
Senior MemberPosts: 2,392Gear list
Like?
Re: Here's The "Party Trick" Explained
In reply to Timur Born, 11 months ago

Timur Born wrote:

Juggling the menu system for quick JPG changes isn't too much fun either. Does the newer quickmenu screen include highlight and shadow settings? Never checked for that. The H&S control on the E-M5/E-M1 is superb.

-- hide signature --

Red flash eyes save lives and eye-sight!
http://en.wikipedia.org/wiki/Retinoblastoma

Thanks, I'll look for highlight and shadow adjustments in the Q Menu.  That could be easier.  I am using C1 and C2 currently for raw + jpeg with differing jpeg profiles as well.

 Lightpath48's gear list:Lightpath48's gear list
Nikon Coolpix P7800 Nikon D3300 Nikon AF-S DX Nikkor 55-200mm f/4-5.6G VR Nikon AF-S DX Nikkor 35mm f/1.8G Nikon AF-S DX Nikkor 18-55mm f/3.5-5.6G VR II
Reply   Reply with quote   Complain
Keyboard shortcuts:
FForum MMy threads