How to get RAW histograms (G1/GH1/GF1)

Started Oct 1, 2009 | Discussions
kenw
Veteran MemberPosts: 4,489Gear list
Like?
How to get RAW histograms (G1/GH1/GF1)
Oct 1, 2009

Apologies if someone has covered this already in the distant past.

If you are a RAW shooter and like to ETTR (expose to the right) then you probably already know that the histograms the camera displays are actually based on the JPG engine and not the RAW pixel data. As a result even if you ETTR based on the camera histograms you probably still underexposed, often by as much as 1EV. Also, there is the problem of WB effecting the histograms and thus in many cases in the playback menu one of the colors may show it is about to clip even though it may actually still be more than one EV underexposed. Without better feedback it is tricky to decide how much more exposure you can apply without clipping.

Fortunately someone came up with a clever method around this, I modified the procedure slightly for the Panasonic cameras. Here is the basic idea:

  • Setup a My Film mode based on the Nostalgic film mode and set saturation and contrast to -2 (we do this to get the gentlest tone curve so that the histograms don't clip before the RAW data).

  • Set the color mode to aRGB (this may not be strictly necessary, but it will help histogram accuracy in scenes with saturated colors).

  • The difference for Panasonic is at the end of the procedure, the Panasonic cameras don't set white balance from a file but instead from a live display. Thus don't follow his instructions for Canon cameras where you copy back a RAW file, instead just point your camera at your "magenta chart" on your monitor and set a custom white balance off of it.

  • Now when you set your camera to your My Film mode and your custom WB derived from your "magenta chart" the histograms you get will be very representative of the actual RAW histograms. I've taken test shots and compared the camera displayed histograms to RAW file histograms and the saturation point is just about perfect for all three color channels.

Unfortunately since Panasonic does a live custom WB setting I can't distribute a file to help everyone, you'll have to do the WB procedure yourself (lucky Canon users can just share a file amongst themselves). I can perhaps help those of you with a calibrated monitor by telling you that for my G1 the "magenta chart" to use for the WB setting is R=110, G=64, B=90 in the sRGB color space. That might let you skip some of the above procedure and get pretty close.
--
Ken W

Rebel XT, XTi, Pany G1, LX3, FZ28, Fuji F30, and a lot of 35mm and 4x5 sitting in the closet...

 kenw's gear list:kenw's gear list
Sony RX100 Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Panasonic Lumix DMC-GM1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS +29 more
jwinberg1
Contributing MemberPosts: 619Gear list
Like?
Re: How to get RAW histograms (G1/GH1/GF1)
In reply to kenw, Oct 1, 2009

WOW, this is a GREAT approach and so kind of you to share like this. I'm going to try it right NOW.

Thanks so much...................... Jack Winberg

 jwinberg1's gear list:jwinberg1's gear list
Canon PowerShot S95 Olympus OM-D E-M5 Canon EOS 5D Mark III Canon EOS 650D Olympus E-M1
Reply   Reply with quote   Complain
plevyadophy
Veteran MemberPosts: 4,250Gear list
Like?
Thanks a lot Re: How to get RAW histograms (G1/GH1/GF1)
In reply to kenw, Oct 1, 2009

Thanks very much.

I know MANY filks will be grateful for this info.

Regards

Reply   Reply with quote   Complain
plevyadophy
Veteran MemberPosts: 4,250Gear list
Like?
A Query Re: How to get RAW histograms (G1/GH1/GF1)
In reply to kenw, Oct 1, 2009

kenw wrote:
Apologies if someone has covered this already in the distant past.

If you are a RAW shooter and like to ETTR (expose to the right) then you probably already know that the histograms the camera displays are actually based on the JPG engine and not the RAW pixel data. As a result even if you ETTR based on the camera histograms you probably still underexposed, often by as much as 1EV. Also, there is the problem of WB effecting the histograms and thus in many cases in the playback menu one of the colors may show it is about to clip even though it may actually still be more than one EV underexposed. Without better feedback it is tricky to decide how much more exposure you can apply without clipping.

Fortunately someone came up with a clever method around this, I modified the procedure slightly for the Panasonic cameras. Here is the basic idea:

  • Setup a My Film mode based on the Nostalgic film mode and set saturation and contrast to -2 (we do this to get the gentlest tone curve so that the histograms don't clip before the RAW data).

  • Set the color mode to aRGB (this may not be strictly necessary, but it will help histogram accuracy in scenes with saturated colors).

  • The difference for Panasonic is at the end of the procedure, the Panasonic cameras don't set white balance from a file but instead from a live display. Thus don't follow his instructions for Canon cameras where you copy back a RAW file, instead just point your camera at your "magenta chart" on your monitor and set a custom white balance off of it.

  • Now when you set your camera to your My Film mode and your custom WB derived from your "magenta chart" the histograms you get will be very representative of the actual RAW histograms. I've taken test shots and compared the camera displayed histograms to RAW file histograms and the saturation point is just about perfect for all three color channels.

Unfortunately since Panasonic does a live custom WB setting I can't distribute a file to help everyone, you'll have to do the WB procedure yourself (lucky Canon users can just share a file amongst themselves). I can perhaps help those of you with a calibrated monitor by telling you that for my G1 the "magenta chart" to use for the WB setting is R=110, G=64, B=90 in the sRGB color space. That might let you skip some of the above procedure and get pretty close.
--
Ken W

Rebel XT, XTi, Pany G1, LX3, FZ28, Fuji F30, and a lot of 35mm and 4x5 sitting in the closet...

It just occured to me, how is this setup gonna be of any use to someone out in the "field"?

I mean they aren't gonna have a monitor to hand to take a WB setting off of are they???

Unless of course, one prints a colour accurate magenta patch onto some quality paper? Would that do the trick?

Regards,

Reply   Reply with quote   Complain
kenw
Veteran MemberPosts: 4,489Gear list
Like?
Re: A Query Re: How to get RAW histograms (G1/GH1/GF1)
In reply to plevyadophy, Oct 1, 2009

plevyadophy wrote:

It just occured to me, how is this setup gonna be of any use to someone out in the "field"?

I mean they aren't gonna have a monitor to hand to take a WB setting off of are they???

You only need to do this once and store it in one of the two custom WB slots. As long as you don't overwrite that custom WB setting you shouldn't need to do it again. So you really have no need to do it in the field. (Remember, to get the RAW histograms you use this same WB setting regardless of the actual light you are shooting in).

Unless of course, one prints a colour accurate magenta patch onto some quality paper? Would that do the trick?

That would work, or go to your local paint store and see if they have a swatch that is close enough. Of course then in the field you'd have to illuminate this patch/swatch under the same light you chose it under (say daylight, or your cameras built in flash). Again, this really shouldn't be necessary unless you accidentally overwrite your custom WB setting or factory reset the camera.
--
Ken W

Rebel XT, XTi, Pany G1, LX3, FZ28, Fuji F30, and a lot of 35mm and 4x5 sitting in the closet...

 kenw's gear list:kenw's gear list
Sony RX100 Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Panasonic Lumix DMC-GM1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS +29 more
Reply   Reply with quote   Complain
plevyadophy
Veteran MemberPosts: 4,250Gear list
Like?
Re: A Query Re: How to get RAW histograms (G1/GH1/GF1)
In reply to kenw, Oct 2, 2009

kenw wrote:

plevyadophy wrote:

It just occured to me, how is this setup gonna be of any use to someone out in the "field"?

I mean they aren't gonna have a monitor to hand to take a WB setting off of are they???

You only need to do this once and store it in one of the two custom WB slots. As long as you don't overwrite that custom WB setting you shouldn't need to do it again. So you really have no need to do it in the field. (Remember, to get the RAW histograms you use this same WB setting regardless of the actual light you are shooting in).

I think therein lies a little problem as there are only two Custom WB slots so its likely that one may write over it.

Now something else occurred to me, if I am doing that "magenta thing" so as to get a realistic raw histo, doesn't that then add to my workflow because now every shot I take will have the incorrerct WB forcing me to use a Grey Card at every shoot or to find some white/grey reference point upon which I would use the ubiquitous "eyedropper tool" rather than getting WB right in-camera for every shot in the shoot?

Unless of course, one prints a colour accurate magenta patch onto some quality paper? Would that do the trick?

That would work, or go to your local paint store and see if they have a swatch that is close enough. Of course then in the field you'd have to illuminate this patch/swatch under the same light you chose it under (say daylight, or your cameras built in flash). Again, this really shouldn't be necessary unless you accidentally overwrite your custom WB setting or factory reset the camera.

I am also wondering whether, from experience and/or the data availabe from various websites, it would not be easier to simply dial in say +0.7 (or whaterver) Exposure Compensation on every shot? The reason I ask/suggest that is that one reviewer who reviewed the Sony R1 advised the use of that simple technique for shooting raw (his advice was that if the histo was just touching the right hand edge, you could simply dial +1 EV and have a well exposed raw file).

Thanks for the advice.

-- hide signature --

Ken W

Rebel XT, XTi, Pany G1, LX3, FZ28, Fuji F30, and a lot of 35mm and 4x5 sitting in the closet...

Regards,

Reply   Reply with quote   Complain
Trevor G
Veteran MemberPosts: 6,473Gear list
Like?
Does it really matter?
In reply to plevyadophy, Oct 2, 2009

Does it really matter?

Does it actually improve the result of PPing?

I would be surprised, since all you are doing is saving the need to make as much adjustment to the histogram, as if you do not make the "calibration."

You still will have to adjust levels for best results - and since you are doing PP anyway, what's the point?

I find that even with jpegs, I can easily pull 2 stops of underexposure out, without any other apparent problems.
--
Cheers

Trevor G

http://www.computerwyse.com/photo.html

Reply   Reply with quote   Complain
kenw
Veteran MemberPosts: 4,489Gear list
Like?
Re: A Query Re: How to get RAW histograms (G1/GH1/GF1)
In reply to plevyadophy, Oct 2, 2009

plevyadophy wrote:

I think therein lies a little problem as there are only two Custom WB slots so its likely that one may write over it.

Yes, it depends on how much you use the custom WB settings. For lots of RAW shooters the custom WB settings are unused as they leave WB to the RAW converter.

Now something else occurred to me, if I am doing that "magenta thing" so as to get a realistic raw histo, doesn't that then add to my workflow because now every shot I take will have the incorrerct WB forcing me to use a Grey Card at every shoot or to find some white/grey reference point upon which I would use the ubiquitous "eyedropper tool" rather than getting WB right in-camera for every shot in the shoot?

Well, if you are going to ETTR then you are going to have to adjust the exposure on every shot in the RAW converter already. That is always the cost of doing ETTR. Now you won't need to use a grey card unless you were going to use a grey card with the camera in the first place - WB is done in the RAW converter the same way as in the camera. Either you shoot a grey card, use a preset or use AWB. Again, all three things can be done at the time of the RAW conversion just as easily as if they had been done in camera.

Now, if you are hoping to have most of your RAW conversions done in a batch process or you are shooting RAW+JPEG and hope to use the JPEGs then this magenta trick is definitely not as useful. Of course ETTR itself isn't useful either, you can't batch convert it and it'll mess up your JPEGs already!

I am also wondering whether, from experience and/or the data availabe from various websites, it would not be easier to simply dial in say +0.7 (or whaterver) Exposure Compensation on every shot? The reason I ask/suggest that is that one reviewer who reviewed the Sony R1 advised the use of that simple technique for shooting raw (his advice was that if the histo was just touching the right hand edge, you could simply dial +1 EV and have a well exposed raw file).

Of course you could do this, and it would probably be correct a lot of the time. In some cases it might clip of course, but again you could use your experience to recognize such occasions and just do some bracketing. And with this approach you can set up a batch RAW conversion that always undid your exposure compensation.

Also you could spend a little time understanding how the histograms work with a normal daylight white balance and just compensate based on that instead of doing the magenta trick. Really, most daylight scenes without wild colors will probably have the green channel and luminance channel clip first - so you could just check those and be in pretty good shape. And just by setting Nostalgic mode with -2 contrast you'd immediately be much closer to evaluating RAW exposure even without the magenta trick.

So it is entirely a question of how you shoot and how much you want to fuss over the exposure. ETTR already implies a fair bit of fussing, so in some cases people might like this technique for more accuracy. In some cases getting the "perfect" ETTR exposure might save the shadows in an image, in other cases it might just be extra work for no purpose!

And finally you could always just turn on AEB and sort out the best exposure when you get home.
--
Ken W

Rebel XT, XTi, Pany G1, LX3, FZ28, Fuji F30, and a lot of 35mm and 4x5 sitting in the closet...

 kenw's gear list:kenw's gear list
Sony RX100 Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Panasonic Lumix DMC-GM1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS +29 more
Reply   Reply with quote   Complain
dbm305
Contributing MemberPosts: 788Gear list
Like?
Re: Does it really matter?
In reply to Trevor G, Oct 2, 2009

The point is not to make the image brighter, but to capture more of the range of tones in the higher part of the image range where there are finer discriminations. In fact aesthetically you'll usually have darken a ETTR file; but you will get more fine discriminations of tome.

 dbm305's gear list:dbm305's gear list
Sony RX100 Olympus OM-D E-M5 Sony Alpha 7R Canon EF 135mm f/2.0L USM Canon TS-E 17mm f/4L +15 more
Reply   Reply with quote   Complain
kenw
Veteran MemberPosts: 4,489Gear list
Like?
Re: Does it really matter?
In reply to Trevor G, Oct 2, 2009

Trevor G wrote:

Does it really matter?

Does it actually improve the result of PPing?

It depends on the shot. But yes, in many cases ETTR can greatly improve the shadows.

I would be surprised, since all you are doing is saving the need to make as much adjustment to the histogram, as if you do not make the "calibration."

You still will have to adjust levels for best results - and since you are doing PP anyway, what's the point?

This is a decent write up of the technique and its benefits:

http://www.luminous-landscape.com/tutorials/expose-right.shtml

I find that even with jpegs, I can easily pull 2 stops of underexposure out, without any other apparent problems.

Again, it depends on the shot and what you are trying to do with the final image. You can certainly do quite a bit of PP on a well exposed JPG file, and you can do even more with a normally exposured RAW file, and in some scenes you can do still even more with a ETTR RAW file. One step further would be to shoot 7 bracketed shots and create an HDR file from which you could do even more extreme things. Just a question of degrees.

Oh, and of course all this fussing with getting RAW histograms in the camera is just to make ETTR a bit easier. One can certainly do a pretty good job of ETTR without going to all this trouble. It is just that you see a lot of RAW shooters complain about the lack of RAW histograms and then I found this clever guy had come up with a way to do it and thought I'd share.
--
Ken W

Rebel XT, XTi, Pany G1, LX3, FZ28, Fuji F30, and a lot of 35mm and 4x5 sitting in the closet...

 kenw's gear list:kenw's gear list
Sony RX100 Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Panasonic Lumix DMC-GM1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS +29 more
Reply   Reply with quote   Complain
plevyadophy
Veteran MemberPosts: 4,250Gear list
Like?
Re: A Query Re: How to get RAW histograms (G1/GH1/GF1)
In reply to kenw, Oct 2, 2009

kenw wrote:

plevyadophy wrote:

I think therein lies a little problem as there are only two Custom WB slots so its likely that one may write over it.

Yes, it depends on how much you use the custom WB settings. For lots of RAW shooters the custom WB settings are unused as they leave WB to the RAW converter.

Now something else occurred to me, if I am doing that "magenta thing" so as to get a realistic raw histo, doesn't that then add to my workflow because now every shot I take will have the incorrerct WB forcing me to use a Grey Card at every shoot or to find some white/grey reference point upon which I would use the ubiquitous "eyedropper tool" rather than getting WB right in-camera for every shot in the shoot?

Well, if you are going to ETTR then you are going to have to adjust the exposure on every shot in the RAW converter already. That is always the cost of doing ETTR. Now you won't need to use a grey card unless you were going to use a grey card with the camera in the first place - WB is done in the RAW converter the same way as in the camera. Either you shoot a grey card, use a preset or use AWB. Again, all three things can be done at the time of the RAW conversion just as easily as if they had been done in camera.

Now, if you are hoping to have most of your RAW conversions done in a batch process or you are shooting RAW+JPEG and hope to use the JPEGs then this magenta trick is definitely not as useful. Of course ETTR itself isn't useful either, you can't batch convert it and it'll mess up your JPEGs already!

I am also wondering whether, from experience and/or the data availabe from various websites, it would not be easier to simply dial in say +0.7 (or whaterver) Exposure Compensation on every shot? The reason I ask/suggest that is that one reviewer who reviewed the Sony R1 advised the use of that simple technique for shooting raw (his advice was that if the histo was just touching the right hand edge, you could simply dial +1 EV and have a well exposed raw file).

Of course you could do this, and it would probably be correct a lot of the time. In some cases it might clip of course, but again you could use your experience to recognize such occasions and just do some bracketing. And with this approach you can set up a batch RAW conversion that always undid your exposure compensation.

Also you could spend a little time understanding how the histograms work with a normal daylight white balance and just compensate based on that instead of doing the magenta trick. Really, most daylight scenes without wild colors will probably have the green channel and luminance channel clip first - so you could just check those and be in pretty good shape. And just by setting Nostalgic mode with -2 contrast you'd immediately be much closer to evaluating RAW exposure even without the magenta trick.

So it is entirely a question of how you shoot and how much you want to fuss over the exposure. ETTR already implies a fair bit of fussing, so in some cases people might like this technique for more accuracy. In some cases getting the "perfect" ETTR exposure might save the shadows in an image, in other cases it might just be extra work for no purpose!

And finally you could always just turn on AEB and sort out the best exposure when you get home.
--
Ken W

Rebel XT, XTi, Pany G1, LX3, FZ28, Fuji F30, and a lot of 35mm and 4x5 sitting in the closet...

Hi,

Thanks very much indeed for your time and effort in this thread.

Regards,

P.S.

BTW, the reviewer I was referring to you will no doubt know of ...... Mr M Reichmann of Luminous Landscape

Reply   Reply with quote   Complain
plevyadophy
Veteran MemberPosts: 4,250Gear list
Like?
Re: A Query Re: How to get RAW histograms (G1/GH1/GF1)
In reply to kenw, Oct 4, 2009

kenw wrote:

plevyadophy wrote:

I think therein lies a little problem as there are only two Custom WB slots so its likely that one may write over it.

Yes, it depends on how much you use the custom WB settings. For lots of RAW shooters the custom WB settings are unused as they leave WB to the RAW converter.

Now something else occurred to me, if I am doing that "magenta thing" so as to get a realistic raw histo, doesn't that then add to my workflow because now every shot I take will have the incorrerct WB forcing me to use a Grey Card at every shoot or to find some white/grey reference point upon which I would use the ubiquitous "eyedropper tool" rather than getting WB right in-camera for every shot in the shoot?

Well, if you are going to ETTR then you are going to have to adjust the exposure on every shot in the RAW converter already. That is always the cost of doing ETTR. Now you won't need to use a grey card unless you were going to use a grey card with the camera in the first place - WB is done in the RAW converter the same way as in the camera. Either you shoot a grey card, use a preset or use AWB. Again, all three things can be done at the time of the RAW conversion just as easily as if they had been done in camera.

Now, if you are hoping to have most of your RAW conversions done in a batch process or you are shooting RAW+JPEG and hope to use the JPEGs then this magenta trick is definitely not as useful. Of course ETTR itself isn't useful either, you can't batch convert it and it'll mess up your JPEGs already!

I am also wondering whether, from experience and/or the data availabe from various websites, it would not be easier to simply dial in say +0.7 (or whaterver) Exposure Compensation on every shot? The reason I ask/suggest that is that one reviewer who reviewed the Sony R1 advised the use of that simple technique for shooting raw (his advice was that if the histo was just touching the right hand edge, you could simply dial +1 EV and have a well exposed raw file).

Of course you could do this, and it would probably be correct a lot of the time. In some cases it might clip of course, but again you could use your experience to recognize such occasions and just do some bracketing. And with this approach you can set up a batch RAW conversion that always undid your exposure compensation.

Also you could spend a little time understanding how the histograms work with a normal daylight white balance and just compensate based on that instead of doing the magenta trick. Really, most daylight scenes without wild colors will probably have the green channel and luminance channel clip first - so you could just check those and be in pretty good shape. And just by setting Nostalgic mode with -2 contrast you'd immediately be much closer to evaluating RAW exposure even without the magenta trick.

So it is entirely a question of how you shoot and how much you want to fuss over the exposure. ETTR already implies a fair bit of fussing, so in some cases people might like this technique for more accuracy. In some cases getting the "perfect" ETTR exposure might save the shadows in an image, in other cases it might just be extra work for no purpose!

And finally you could always just turn on AEB and sort out the best exposure when you get home.
--
Ken W

Rebel XT, XTi, Pany G1, LX3, FZ28, Fuji F30, and a lot of 35mm and 4x5 sitting in the closet...

Firstly, why was the Nostalgic Film Mode chosen? (EDIT: Silly question! I just had a look at the G1/GH1 reviews and it shows that the Nostalgic Film Mode gives the broadest dynamic range, so obviously that's the one to use to represent raw data. Easy isn't it, when one engages one's brain before asking silly questions? )

I actually like the idea of using the Nostalgic Film Mode as I think it has a certain "harmony" to it in that Raw is often seen as the digital equivalent of film and film (and the development of film) is often looked upon with nostalgia; thinking of it like that if I use this system you suggest then I can always know that I am in raw mode if the Film Mode is set to Nostalgic

By the way here is an article that kinda supports the ideas you have put forward: http://www.luminous-landscape.com/tutorials/right-hista.shtml

Regards,

Reply   Reply with quote   Complain
CWilt
Senior MemberPosts: 2,436
Like?
Re: How to get RAW histograms (G1/GH1/GF1)
In reply to kenw, Oct 4, 2009

Some articles I suggest for those who are interested. We can discuss this further after you have digested these and understand the intent of UNI white balance. I am not Iliah Borg, who I believe to be the first to suggest and use these methods, but I was there at the start and have been a student of his methods for several years and may be able to help. Following his methods I may not give you the answers you seek but instead may suggest ways for you to test and prove it to yourself. Doing is the best way to learn.

First one explains white balance and the problems it ca cause when using the histogram to judge proper exposure.
http://www.libraw.org/articles/white-balance-in-digital-cameras.html

Use of color correction filters with UNI-WB for daylight white balance.
http://www.libraw.org/articles/magenta-filters-on-digicam.html

Typical daytime exposures have noise in blue skies because the red channel is under exposed at capture time. White balance can be thought of as post capture exposure compensation on the red and blue channels.
http://www.libraw.org/articles/channel-noise-and-raw-converters.html

-- hide signature --

Charles
My family images are at http://www.stakeman.smugmug.com
Be sure of your subject.
Never, force the shot.

Reply   Reply with quote   Complain
mbk
mbk
Contributing MemberPosts: 565
Like?
Re: How to get RAW histograms (G1/GH1/GF1)
In reply to kenw, Oct 5, 2009

This looks like an interesting approach, but when I tried it I did not seem to get any different looking histograms when changing film type, or when changing white balance. I triedt to calibrate white balance to the magenta for isntance, and as an unpleasant side effect the WYSIWIG viewfinders on my G1 show all wrong colors, but the histogram still does not look any different with the custom WB, nor does btw the calculated exposure. Maybe when in RAW mode the G1 doesn't care about the jpeg settings. and already exposes the RAW to the right?
--
Markus

Reply   Reply with quote   Complain
CWilt
Senior MemberPosts: 2,436
Like?
Re: How to get RAW histograms (G1/GH1/GF1)
In reply to mbk, Oct 5, 2009

mbk wrote:

This looks like an interesting approach, but when I tried it I did not seem to get any different looking histograms when changing film type, or when changing white balance. I triedt to calibrate white balance to the magenta for isntance, and as an unpleasant side effect the WYSIWIG viewfinders on my G1 show all wrong colors, but the histogram still does not look any different with the custom WB, nor does btw the calculated exposure. Maybe when in RAW mode the G1 doesn't care about the jpeg settings. and already exposes the RAW to the right?

This is for RAW shooters. The camera settings need to be as flat as possible. Changing the "film" type will change contrast and the curves that apply that contrast. Try shooting something with a strong color like a tight shot of a rose. It is in those situations that these methods really shine.

You will see an ugly preview by using just the UNI-WB. Thats the step that gives accurate histograms. I am talking about the 3 color histogram display and not the composite single one. By using color correcting filter and UNI-WB you can gain a considerable out of exposure in the red or blue channels. In daylight the red channel can be under exposed by a stop or more. This leads to increased visible noise and a loss of detail.

Articles about ETTR and the problems it can cause.

http://www.libraw.org/node/46
http://www.libraw.org/articles/peace-in-lights.html
--
Charles
My family images are at http://www.stakeman.smugmug.com
Be sure of your subject.
Never, force the shot.

Reply   Reply with quote   Complain
kenw
Veteran MemberPosts: 4,489Gear list
Like?
Re: How to get RAW histograms (G1/GH1/GF1)
In reply to mbk, Oct 5, 2009

mbk wrote:

This looks like an interesting approach, but when I tried it I did not seem to get any different looking histograms when changing film type, or when changing white balance.

The luminance histogram (the one you see in live view) will not change much when you adjust WB. Luminance is almost always dominated by the green channel and the WB adjustments actually only effect the R and B channels. Obviously in a very red or very blue scene the R or B would play a larger role in the luminance histogram, but in most scenes the G and Y (luminance) are very close. In play mode instead look at the individual RGB histograms. You'll notice the R channel in particular shifts about a full stop between daylight WB and magenta WB. You'll also notice the similarity between the G and Y histograms in most scenes. It is getting the R and B histograms correct that the magenta WB helps with. Now whether someone wants to go to the trouble of checking is another story!

As far as film modes go you make and interesting observation. I went back and looked at both the GH1 and G1 reviews here and it does appear in fact that the clipping point really doesn't change between film modes - the black point and mid-grey are what change! So you are correct, as far as ETTR goes it would appear changing the film mode doesn't actually improve or hurt the part of the histograms we care about!

I tried to calibrate white balance to the magenta for isntance, and as an unpleasant side effect the WYSIWIG viewfinders on my G1 show all wrong colors, but the histogram still does not look any different with the custom WB, nor does btw the calculated exposure.

So I've actually reversed the WYSIWYG problem by using the LCD/Viewfinder color correction options on the camera. You can dial in a green/blue color shift in the LCD calibration screen to get your live view looking normal again. Yeah, it is getting a bit Rube Goldberg at this point...

Maybe when in RAW mode the G1 doesn't care about the jpeg settings. and already exposes the RAW to the right?

No, I think it is primarily that the camera never exposes to the right but instead always meters to middle gray. This type of metering shouldn't be very sensitive to WB, even an extreme WB shift like the magenta trick provides.

Thanks for your input, and discovering (or re-discovering I guess) that the film mode setting doesn't actually appear to affect the histogram clipping point.

I'm still playing with this and not sure which way I'll primarily shoot. I suspect that for most scenes, especially daylight which I shoot most often, the luminance histogram is almost always dominated by green and green clipping occurs first. This means the magenta trick doesn't really add that much information and just working off the live view luminance histogram should produce good results most of the time. In really weird lighting, or a colorful subject, a quick check of the daylight WB histogram would show if something odd was going on. If it was, well I could switch over to my custom Magenta WB and really try to get things spot on if I cared.
--
Ken W

Rebel XT, XTi, Pany G1, LX3, FZ28, Fuji F30, and a lot of 35mm and 4x5 sitting in the closet...

 kenw's gear list:kenw's gear list
Sony RX100 Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Panasonic Lumix DMC-GM1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS +29 more
Reply   Reply with quote   Complain
plevyadophy
Veteran MemberPosts: 4,250Gear list
Like?
Re: How to get RAW histograms (G1/GH1/GF1)
In reply to kenw, Oct 5, 2009

kenw wrote:

mbk wrote:

This looks like an interesting approach, but when I tried it I did not seem to get any different looking histograms when changing film type, or when changing white balance.

The luminance histogram (the one you see in live view) will not change much when you adjust WB. Luminance is almost always dominated by the green channel and the WB adjustments actually only effect the R and B channels. Obviously in a very red or very blue scene the R or B would play a larger role in the luminance histogram, but in most scenes the G and Y (luminance) are very close. In play mode instead look at the individual RGB histograms. You'll notice the R channel in particular shifts about a full stop between daylight WB and magenta WB. You'll also notice the similarity between the G and Y histograms in most scenes. It is getting the R and B histograms correct that the magenta WB helps with. Now whether someone wants to go to the trouble of checking is another story!

As far as film modes go you make and interesting observation. I went back and looked at both the GH1 and G1 reviews here and it does appear in fact that the clipping point really doesn't change between film modes - the black point and mid-grey are what change! So you are correct, as far as ETTR goes it would appear changing the film mode doesn't actually improve or hurt the part of the histograms we care about!

I tried to calibrate white balance to the magenta for isntance, and as an unpleasant side effect the WYSIWIG viewfinders on my G1 show all wrong colors, but the histogram still does not look any different with the custom WB, nor does btw the calculated exposure.

So I've actually reversed the WYSIWYG problem by using the LCD/Viewfinder color correction options on the camera. You can dial in a green/blue color shift in the LCD calibration screen to get your live view looking normal again. Yeah, it is getting a bit Rube Goldberg at this point...

Maybe when in RAW mode the G1 doesn't care about the jpeg settings. and already exposes the RAW to the right?

No, I think it is primarily that the camera never exposes to the right but instead always meters to middle gray. This type of metering shouldn't be very sensitive to WB, even an extreme WB shift like the magenta trick provides.

Thanks for your input, and discovering (or re-discovering I guess) that the film mode setting doesn't actually appear to affect the histogram clipping point.

I'm still playing with this and not sure which way I'll primarily shoot. I suspect that for most scenes, especially daylight which I shoot most often, the luminance histogram is almost always dominated by green and green clipping occurs first. This means the magenta trick doesn't really add that much information and just working off the live view luminance histogram should produce good results most of the time. In really weird lighting, or a colorful subject, a quick check of the daylight WB histogram would show if something odd was going on. If it was, well I could switch over to my custom Magenta WB and really try to get things spot on if I cared.
--
Ken W

Rebel XT, XTi, Pany G1, LX3, FZ28, Fuji F30, and a lot of 35mm and 4x5 sitting in the closet...

The test I did was to shoot a scene in three Film Modes, Standard, Dynamic, and Nostalgic.

I reduced everything to -2 in the Nostalgic Film Mode settings (Contrast, Saturation, Sharpness, Noise Reduction).

I shot in RAW mode.

Looking at the images on the back of the camera I noticed a great difference. The Nostalgic image was much brighter in appearance in shadows, midtone, and highlight and it also took on less of a colour cast.

Also, the historgram in all channels for the Nostalgic shots were about 0.7EV further to the right.

So is this the kinda result one would expect, with the tones further to the right in the actual raw file?

I am also finding the whole magenta thing hard to undestand, could you explain it in simple terms.

And what does one do with the resultant image taken using the magenta WB trick? I mean how does one correct the raw file to get the correct colours? Do I take it that the magenta trick is some kinda sophisticated Auto WB and one simply dials in certain set parameters in batch processing which will result in corrected images giving the correct colours no matter where the image has been taken?

I would also be grateful for your comments in respect of my two posts above.

Thanks in advance.

Regards,

Reply   Reply with quote   Complain
kenw
Veteran MemberPosts: 4,489Gear list
Like?
Re: How to get RAW histograms (G1/GH1/GF1)
In reply to plevyadophy, Oct 5, 2009

Sorry, I don't have much time at this moment, so I might not be able to reply in detail or to the other posts until tomorrow. But I'll try to answer the easy ones quickly.

plevyadophy wrote:

The test I did was to shoot a scene in three Film Modes, Standard, Dynamic, and Nostalgic.

I reduced everything to -2 in the Nostalgic Film Mode settings (Contrast, Saturation, Sharpness, Noise Reduction).

I shot in RAW mode.

Looking at the images on the back of the camera I noticed a great difference. The Nostalgic image was much brighter in appearance in shadows, midtone, and highlight and it also took on less of a colour cast.

Quick points:

  • Whether you shoot in RAW or not what is displayed on the camera - the images and the histrograms - will be as if you shot JPEG. So yes, as you change film settings or WB or contrast saturation settings you'll see big changes.

  • I think, based on info in the DPR reviews, that even though the histograms change with film modes that the upper edge of the histogram won't actually move much. I suspect though the Contrast setting would. I need to try this all out myself, I initially just set it as low contrast/saturation as possible.

  • Again, based on the curves in the DR testing section of the DPR reviews it appears that as you move to Nostalgic things will get brighter in the JPEGS.

  • Because of the reduced saturation in Nostalgic mode the apparent color shift will be less.

  • Again all of this applies only to what is displayed on your camera, the RAW data does not change regardless of film mode, contrast/saturation settings, or WB.

Also, the historgram in all channels for the Nostalgic shots were about 0.7EV further to the right.

Double check you had the same exposure in the two shots. If that is the same, again I expect that the lower and middle portions of the histogram will move up in Nostalgic vs. Vivid. I think that the very upper edge of the histogram won't move much - again haven't had time to try myself, just based on DPRs measurements of DR in their reviews.

So is this the kinda result one would expect, with the tones further to the right in the actual raw file?

No, the RAW file itself would be the same (assuming same shutter speed/aperture/ISO was used). But the camera won't tell you what is really in the RAW file, you'd need to use RAWAnalyze or something to examine the RAW histograms. Or you could take the two photos into a RAW converter and set the same parameters on each file and get identical results.

I am also finding the whole magenta thing hard to undestand, could you explain it in simple terms.

I'll try tomorrow. If you haven't already check out the link in my first post and the links posted by others in this thread. If you already have done this I'll try to come up with a simpler explanation tomorrow.

And what does one do with the resultant image taken using the magenta WB trick? I mean how does one correct the raw file to get the correct colours? Do I take it that the magenta trick is some kinda sophisticated Auto WB and one simply dials in certain set parameters in batch processing which will result in corrected images giving the correct colours no matter where the image has been taken?

Remember in RAW the WB is just metadata. It is only used by the RAW converter if you set the RAW converter WB to "As Shot". If you use any RAW converter preset (like Daylight, Shade) or the RAW converter's AWB function then the WB metadata from the camera is ignored. So you don't need any special processing, just need to set the RAW converter to ignore the camera and use either its own AWB or a color temperature preset.

I would also be grateful for your comments in respect of my two posts above.

Tomorrow hopefully.

-- hide signature --

Ken W

Rebel XT, XTi, Pany G1, LX3, FZ28, Fuji F30, and a lot of 35mm and 4x5 sitting in the closet...

 kenw's gear list:kenw's gear list
Sony RX100 Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Panasonic Lumix DMC-GM1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS +29 more
Reply   Reply with quote   Complain
mbk
mbk
Contributing MemberPosts: 565
Like?
Re: How to get RAW histograms (G1/GH1/GF1)
In reply to kenw, Oct 6, 2009

We're finally on to something I think. First off I tried a lot of things when I got my G1, one of which was to do my own film modes in the hope to maybe avoid RAW... but I still often go back to process RAW anyway. I found that in essence, each and every tweak ended having some other drawback. For instance I set my custom mode to nostalgic and -2 everything precisely in the hope that a shallow tone curve would get me tha maximum latitude for tightening later on etc. Actually for pleasing impression I found Smooth a better choice. But any improvement say in midtone was defeated by worse noise in shades etc. So now I went back to Standard for run of the mill JPEGS and RAW for

What ticked me off in this approach is that no matter what I do, even in RAW, skies are very hard to get right. You choose bwteen less NR and getting chroma noise, and more NR and getting lue shaeded blotches of larger and uglier kinds. So fixing the exposure of the different channels would be nice - and depending on how the WB influences RAW data this could work.

Now I don't know why my initial test did not show an influence. I have in the past seen that the G1 gave me a different exposure in nostalgic and smooth, because as you said it does a compromise, apparently, between middle gray and clipping prevention. It's very hard to find out what the camera really does, manufacturers won't tell you, so it's kinda depressing because all I want is to implement a digital zone system but the cameras of any brand re-arrange,m recalibrate and kindly fiddle so much with the data even in raw that you can only guess what's happening and why. This being said, and contrary to your recent post in reply to another portser, some cameras if memory serves, do actually modify the channel amplification depending on the WB setting and thus a different WB would indeed change the channel RAW data captured. So one could max out the red channel in blue skies and decrease noise. The histograms would not just change from a putative a calculated jpeg output to a putative real captured RAW data set, but it's entirely possible that what you see in even the color histgrams is real. And changing the WB then would really change the RAW capture.

I have to try this some time when the blue skies are back in my area, to see if I can meaningfully change IQ with it. And t see if my theory is really true, that RAW changes with WB (I dimly remember longish discussions on that when I got my previous D70 years ago).

And yes I had thought of fiddling with the color balance of the EVF but I started being exasperated

Thanks for the stimulating input!

-- hide signature --

Markus

Reply   Reply with quote   Complain
kenw
Veteran MemberPosts: 4,489Gear list
Like?
Re: How to get RAW histograms (G1/GH1/GF1)
In reply to mbk, Oct 6, 2009

mbk wrote:

So fixing the exposure of the different channels would be nice - and depending on how the WB influences RAW data this could work.

Yes, getting a good red channel exposure would definitely help out skies. And you are correct, it is in theory possible to adjust the readout amplifier for each channel to affect WB - I even thought of that when I wrote the previous reply but decided it might just confuse issues :). I suspect, but do not know for sure, that the G1 does not in fact use adjustable readout amps for each channel. The reason I suspect this is that it tells the RAW converter to use a digital gain of about 1.9 on the red channel data when set to daylight WB (i.e. it underexposes the red channel by a full stop in daylight). If it had adjustable gains I would expect it to use less digital gain. This is easy to check with RAWAnalyze and a few exposures - I'll do it sometime soon.

I would love to see a camera use adjustable analog gains to implement WB, this would help at base ISO quite a bit I think. Most of these cameras still have great noise performance two or three stops above base ISO, which means channel based gains for WB at the lower ISO settings would be an excellent choice. This does, however, violate what appears to be the current design ethic used by almost all cameras - namely that RAW data is unaffected by WB settings or any other settings (though a few cameras intentionally, or in some cases apparently unintentionally, apply NR to some RAW data).

Finally, be sure to read the links Charles posted. One approach linked there is to use a magenta color correcting filter on the lens to try to rebalance the RAW channels to get a better red channel exposure. I've looked at that before and my main concern was that I could find no CC filters that were multi-coated. Just the other day though, I noticed that Hoya makes a fluorescent correction filter that is available multi-coated and I believe would help - it is their FL-W filter. I tried out a crappy Tiffen FL-D I had laying around and it did improve the R channel by about 2/3rds of a stop which does make up a lot of the ground. If some test data I found on an underwater photography site is to be believed then the Hoya should be very close to the Tiffen I used. I may try out the Hoya - whenever B&H gets back from their holiday...

Thanks for your observations!
--
Ken W

Rebel XT, XTi, Pany G1, LX3, FZ28, Fuji F30, and a lot of 35mm and 4x5 sitting in the closet...

 kenw's gear list:kenw's gear list
Sony RX100 Panasonic Lumix DMC-G1 Olympus OM-D E-M5 Panasonic Lumix DMC-GM1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS +29 more
Reply   Reply with quote   Complain
Keyboard shortcuts:
FForum MMy threads