RAW vs White balance

Started Nov 20, 2006 | Discussions
w Regular Member • Posts: 138
Re: RAW vs White balance

Thanks vortout. I'll give it a try once I get a camera.

blende8 wrote:

Can you explain how to operate this software?

I don't have any files to try it on, but based on the source code I would try the following. Copying some PEF files in the same directory as malykh-pef.cmd, then double-click on malykh-pef.cmd. I think it will output some gif of the histograms.

blende8 Senior Member • Posts: 1,494
Re: RAW vs White balance

w wrote:

Copying some PEF files in the same
directory as malykh-pef.cmd, then double-click on malykh-pef.cmd.
I think it will output some gif of the histograms.

Yes, that works.
Thanks!

GordonBGood Veteran Member • Posts: 6,308
Re: RAW vs White balance

As interspersed with images removed:

vortout wrote:

There is very interesting results for RAW files done with different
presets

One fellow on a Russian Pentax forum made a very simple program
reading Pentax RAW file and showing histogram for all 3 channels. (
http://www.penta-club.ru/forum/index.php?showtopic=19120&st=0 )

He used different white balance presets

So enjoy with samples:

AWB:
http://img89.imageshack.us/img89/7518/imgp9705pefsrl2.jpg

Daylight
http://img291.imageshack.us/img291/9208/imgp9706pefszp6.jpg

Tungsten:
http://img149.imageshack.us/img149/3917/imgp9707pefssi3.jpg

Manual WB
http://img148.imageshack.us/img148/3273/imgp9708pefsff4.jpg

As you see files are different not only on the header level but
also it is different physically for red channels.

I don't see that at all, as if the gains for the red and blue were changing as for a WB setting all the way from Daylight to Tungsten, then the Blue channel would be prevalent for one and the red for the other. All I see is a possible reduction of exposure for the Tungsten WB setting to allow for more headroom on the Red channel to avoid clipping if the aperture, shutterspeed, and ISO were exactly the same for these changes in WB. Since we don't have EXIF data, are you sure that these settings didn't change between the shots?

Conclusions from forum:
1. Your WB settings can lead to overexposure for RED channel but it
is not proved yet since may be it just a way to turn down 3rd party
converters.

I don't see that as above.

2. It looks like that 3rd Party converters except SilkyPix do not
know about WB behavior and that’s why there is some strange color
development results.

That's one of the hardest, although most important, parameters to dig out of manufacturers custom raw image files without spending a lot of quality time doing comparisons of the files shot in controlled conditions with controlled changes in WB settings, unless the manufacturer is so kind as to provide details on the file format. So likely SilkyPix and Adobe ACR are the only ones to do so. Dave Coffins dcraw is often wrong on this critical parameter.

Also there are some examples for the histogram for different ISO
settings.

ISO200 1/5
http://img244.imageshack.us/img244/2150/imgp9716pefsev0.jpg

ISO400 1/10
http://img151.imageshack.us/img151/884/imgp9717pefsjp0.jpg

ISO800 1/20
http://img168.imageshack.us/img168/4219/imgp9718pefsii8.jpg

ISO1600 1/40
http://img169.imageshack.us/img169/919/imgp9719pefscm7.jpg

ISO3200 1/80
http://img135.imageshack.us/img135/2221/imgp9720pefswz8.jpg

Conclusions:
1. There is not 1 stop between ISO200 and ISO400

Are you sure that the aperture was controlled between the ISO200 and ISO400 shot, as I would be very surprised that exposure would be wrong by this amount between these settings?

2. ISO 1600 and ISO3200 are rather software made and shall not be
used in RAW mode since you can fix exposure in during RAW
conversion.

Yes, for the D cameras that has been observed several times, and I think I saw some similar results posted for the K100D. However, whether you do the conversion yourself in raw or use the camera settings won't make any difference to the final result. Since the lost bits are pretty much below the noise floor, there won't be anything significant lost in the way of Dynamic Range and as for banding, even if one could notice these gaps in tonality, they will pretty will be filled in by the dithering effect of further processing.

As an aside, the K10D may do all of its ISO sensitivity gain using "software" digital gain, but with a 22 bit ADC there are never any gaps between tonality steps, so it's not a problem exept that readout noise would be directly proportional to ISO sensitivity, if true.

Regards, GordonBGood

ptodd
ptodd Senior Member • Posts: 1,327
interesting criticisms

Perhaps if you have time you could download the software and do your own tests, with all appropriate controls in place?
http://malykh.com/temp/2006-11-19/malykh-pef-2006-11-19.zip

I might even do this myself sometime, but not right now...
--

 ptodd's gear list:ptodd's gear list
Pentax K-30 Samsung D-Xenon 50-200 F4-5.6 ED Tamron SP AF 17-50mm F/2.8 XR Di II LD Aspherical (IF) Tamron SP AF 90mm F/2.8 Di Macro Google Nexus 4 +1 more
GordonBGood Veteran Member • Posts: 6,308
Re: interesting criticisms

ptodd wrote:

Perhaps if you have time you could download the software and do
your own tests, with all appropriate controls in place?
http://malykh.com/temp/2006-11-19/malykh-pef-2006-11-19.zip

I might even do this myself sometime, but not right now...

Are you the original author?

I don't need the software, as I have my own on which I base my raw conversions that can be set to do the same thing.

There isn't a lot of point of repeating the experiments, as I don't have and won't be getting a DS and other cameras may behave differently.

I may have access to a K100D and/or eventually a K10D, and will be checking that ISO settings are accurate but won't be much concerned with the second range of tests other than that for my stated reasons. If I saw such a difference in ISO sensitivity that is not a factor of two as between the ISO 200 and ISO 400 shots, I would be looking for the reasons for this and aperture would be a very important parameter (I would have used full manual mode for these tests and stated aperture values).

As to the first range of tests showing a turned down sensitivity for very "warm" high red Tungsten WB shots, I wouldn't be surprised that this could be true for the reason of giving more Red channel headroom and that there is a private tag in the PEF file telling the raw convertor to adjust exposure for this after WB compensation. Again, full exposure values including aperture are an important part of confirming this.

Neither of these tests, other than the illogical results of the ISO sensitivity justify casting doubt on the useabiliy of PEF raw image files, especially when they are not backed up with full test details.

Regards, GordonBGood

blende8 Senior Member • Posts: 1,494
Re: interesting criticisms

As a practitioner I only want to know:
Is it correct that the RAW file is influenced by the WB preset?

The thesis seems to contradict an axiom of digital photography, namely, RAW is independent of WB.

ptodd
ptodd Senior Member • Posts: 1,327
Re: interesting criticisms

GordonBGood wrote:

ptodd wrote:

Perhaps if you have time you could download the software and do
your own tests, with all appropriate controls in place?
http://malykh.com/temp/2006-11-19/malykh-pef-2006-11-19.zip

I might even do this myself sometime, but not right now...

Are you the original author?

No, I'm not, I just posted the link their as I thought that might make it easier to find.

I don't need the software, as I have my own on which I base my raw
conversions that can be set to do the same thing.

There isn't a lot of point of repeating the experiments, as I don't
have and won't be getting a DS and other cameras may behave
differently.

Ah, I didn't realise that. What camera do you have? I think interesting results can be obtained from any *ist, or any camera that shoots raw. It would be good to know if they do behave differently.

I may have access to a K100D and/or eventually a K10D, and will be
checking that ISO settings are accurate but won't be much concerned
with the second range of tests other than that for my stated
reasons.

It seems to me there is still something to be gained (and little to be lost) by using ISO 800; wether or not the gain is significant is another matter, and you may be right in practice that all of the lower gradations are in the noise anyway. Still, I would say it's better to have a more accurate reading of even the noisiest part of the signal and it seems there is little to gain from high ISOs.

If I saw such a difference in ISO sensitivity that is not
a factor of two as between the ISO 200 and ISO 400 shots, I would
be looking for the reasons for this and aperture would be a very
important parameter (I would have used full manual mode for these
tests and stated aperture values).

Indeed, I pretty much assumed that they wouldn't bother to do a test in any other mode, but you're right there is no explicit indication of this.

As to the first range of tests showing a turned down sensitivity
for very "warm" high red Tungsten WB shots, I wouldn't be surprised
that this could be true for the reason of giving more Red channel
headroom and that there is a private tag in the PEF file telling
the raw convertor to adjust exposure for this after WB
compensation. Again, full exposure values including aperture are
an important part of confirming this.

I've never written any raw software, but I'd be surprised if the program used here would take any notice of such a tag. I haven't looked at the source, even, but I'd expect it to just skip straight past the header and run through the image data in a very crude, direct, way with no interpretation of any tags telling it to do differently. Can you confirm or deny this, or offer any other light on the subject from your own experience of raw software?

Neither of these tests, other than the illogical results of the ISO
sensitivity justify casting doubt on the useabiliy of PEF raw image
files, especially when they are not backed up with full test
details.

The bottom line as far as I'm concerned is that we have graphs showing a proportionally different distribution of intensities among the colour channels in different images of the same scene, where there should be none. Details of exposure should be controlled, but I don't see how that would change the basic fact.

I'm going to try this out myself, later. Probably. If I get 'round to it.

Regards,
Peter
--

 ptodd's gear list:ptodd's gear list
Pentax K-30 Samsung D-Xenon 50-200 F4-5.6 ED Tamron SP AF 17-50mm F/2.8 XR Di II LD Aspherical (IF) Tamron SP AF 90mm F/2.8 Di Macro Google Nexus 4 +1 more
ptodd
ptodd Senior Member • Posts: 1,327
Re: interesting criticisms

blende8 wrote:

As a practitioner I only want to know:
Is it correct that the RAW file is influenced by the WB preset?

It certainly looks that way to me.

The thesis seems to contradict an axiom of digital photography,
namely, RAW is independent of WB.

Quite.

-- hide signature --

 ptodd's gear list:ptodd's gear list
Pentax K-30 Samsung D-Xenon 50-200 F4-5.6 ED Tamron SP AF 17-50mm F/2.8 XR Di II LD Aspherical (IF) Tamron SP AF 90mm F/2.8 Di Macro Google Nexus 4 +1 more
distudio Veteran Member • Posts: 3,913
Re: interesting criticisms

blende8 wrote:

As a practitioner I only want to know:
Is it correct that the RAW file is influenced by the WB preset?

The thesis seems to contradict an axiom of digital photography,
namely, RAW is independent of WB.

It's simple to prove for your particular camera. Just shoot a colour chart using RAW under tungsten illumination and daylight/tungsten WB settings then check if the reds block up on the daylight setting or shoot in daylight using tungsten light and daylight/tungsten WB settings and see if the blues block up on the tungsten setting

Be your own practitioner, don't trust quacks.

-- hide signature --

Rob

 distudio's gear list:distudio's gear list
Sony RX100 II Pentax K-5 IIs Pentax K-3 Pentax 645Z Nikon D750 +19 more
Jonas B Forum Pro • Posts: 14,596
I took 15 pictures...

and found nothing.

Subject: A Kodak grey card that filled the viewfinder
Light: So called daylight bulbs
Tripod: yupp
ISO: 200, 400, 800, 1600, 3200
f-value: manually, set to 1.2 (wide open) for all pictures
Shutter time: manual, 1/250, 1/500, 1/1000, 1/2000 and 1/4000
White balance: first serie: daylight, then tube light D, finally ordinary bulbs
Camera: *ist DS
format: raw

I used the little Russian command line utility and got 15 of these pictures:

Here are the pictures with white balance set to daylight:
ISO200: http://photos.imageevent.com/jonas_b/dprmonthly/0611/2135.jpg
ISO400: http://photos.imageevent.com/jonas_b/dprmonthly/0611/2136.jpg
ISO800: http://photos.imageevent.com/jonas_b/dprmonthly/0611/2137.jpg
ISO1600: http://photos.imageevent.com/jonas_b/dprmonthly/0611/2138.jpg
ISO3200: http://photos.imageevent.com/jonas_b/dprmonthly/0611/2139.jpg
-----
Here are the pictures with white balance set to Fluoroscent D:
ISO200: http://photos.imageevent.com/jonas_b/dprmonthly/0611/2140.jpg
ISO400: http://photos.imageevent.com/jonas_b/dprmonthly/0611/2141.jpg
ISO800: http://photos.imageevent.com/jonas_b/dprmonthly/0611/2142.jpg
ISO1600: http://photos.imageevent.com/jonas_b/dprmonthly/0611/2143.jpg
ISO3200: http://photos.imageevent.com/jonas_b/dprmonthly/0611/2144.jpg
-----
Here are the pictures with white balance set to Tungsten:
ISO200: http://photos.imageevent.com/jonas_b/dprmonthly/0611/2145.jpg
ISO400: http://photos.imageevent.com/jonas_b/dprmonthly/0611/2146.jpg
ISO800: http://photos.imageevent.com/jonas_b/dprmonthly/0611/2147.jpg
ISO1600: http://photos.imageevent.com/jonas_b/dprmonthly/0611/2148.jpg
ISO3200: http://photos.imageevent.com/jonas_b/dprmonthly/0611/2149.jpg

To me it was interesting to see that I got "striped" histograms allready at ISO800. And, as expected, no difference in the distribution between the channelswhen using different settings for the white balance. I write expected as this is what Í have been believing in for nearly 2 years now. But, I'm still open for an altered reality. Did I something wrong with my test setup? Let me know and I'll redo it tomorrow. For now I assume nothing has changed. I'll have another look at that striped histogram using Photoshop later.

Comments?

-- hide signature --

regards,

Jonas

blende8 Senior Member • Posts: 1,494
Re: I took 15 pictures...

Well, one could say that the source target is not optimal, because everything is squeezed in a small area of the histogram, therefore differences are difficult to discern. Small differences are visible.

philzucker
philzucker Veteran Member • Posts: 8,721
Re: I took 15 pictures...

Hi, Jonas!

Jonas B wrote:

and found nothing.
To me it was interesting to see that I got "striped" histograms
allready at ISO800. And, as expected, no difference in the
distribution between the channelswhen using different settings for
the white balance. I write expected as this is what Í have been
believing in for nearly 2 years now. But, I'm still open for an
altered reality.

Me too, Jonas Open for altered reality, that is. Would have been delighted to find some advanced color preconditioning on my good old Ds ...

You saved me from doing the same tests. Thanks for that. Of course I somehow feel forced to photograph something red in tungsten light with different WB presets to see if my Ds has preconditioning or not.

Apart from that the "digital amplifcation" for ISOs above 800 (or in your case already at ISO 800) is interesting enough to know. Will think about the consequences of this info later ...
--
Phil

GMT +1

distudio Veteran Member • Posts: 3,913
Re: I took 15 pictures...

Jonas B wrote:

and found nothing.

Good test, I wonder if we would see the same given an analysis of the K10D RAW files? If the operation of the image pre-processor is as suggested I'd assume not.

-- hide signature --

Rob

 distudio's gear list:distudio's gear list
Sony RX100 II Pentax K-5 IIs Pentax K-3 Pentax 645Z Nikon D750 +19 more
CurtisR Regular Member • Posts: 263
Gordon, you're wrong on a critical point

GordonBGood wrote:

Yes, for the D cameras that has been observed several times, and I
think I saw some similar results posted for the K100D. However,
whether you do the conversion yourself in raw or use the camera
settings won't make any difference to the final result. Since the
lost bits are pretty much below the noise floor, there won't be
anything significant lost in the way of Dynamic Range and as for
banding, even if one could notice these gaps in tonality, they will
pretty will be filled in by the dithering effect of further
processing.

The bits lost by using ISO3200 instead of 800 are the Most Significant Bits - representing the brightest parts of the image. You are clipping highlights, not chopping noise into blackness. Shooting in ISO 800 instead of 3200 gives you the same low-intensity data, but doesn't throw away two stops of high-intensity data. (Though the jpg conversion, and the tone curve that the camera uses to do this conversion tends to throw away a lot of shadow detail, but this is regardless of the camera's sensitivity setting - it's just the nature of jpg vs. raw. You'd need to show some proof to make me believe that all the bits thrown away in this fashion are below the noise floor in all cases.)

Regards, GordonBGood

Just keepin' it real.

CurtisR Regular Member • Posts: 263
Re: I took 15 pictures...

distudio wrote:

Jonas B wrote:

and found nothing.

Good test, I wonder if we would see the same given an analysis of
the K10D RAW files? If the operation of the image pre-processor is
as suggested I'd assume not.

First off.. look at the images more closely. There is a change in distribution - it's just not obvious because of where the exposure is.

When I was perusing the presentation docs for the readout chip in the K10D, it had a programmable per-pixel gain adjustment for the A/D converter that is designed to be used for just this sort of thing. If memory serves me correct, the thing can do a very crude tone curve for each color channel at readout time, in analog. Whether or not Pentax uses this, or does the WB while the data is still 22 bits, it will still show up exactly like this in the raw file (as the raw file is a reduced bit-depth version of the data from the A/D, and the reason for bothering with the extra bits is to prevent quantization artifacts for doing WB and/or tone curves.)

distudio Veteran Member • Posts: 3,913
Re: I took 15 pictures...

CurtisR wrote:

When I was perusing the presentation docs for the readout chip in
the K10D, it had a programmable per-pixel gain adjustment for the
A/D converter that is designed to be used for just this sort of
thing. If memory serves me correct, the thing can do a very crude
tone curve for each color channel at readout time, in analog.
Whether or not Pentax uses this, or does the WB while the data is
still 22 bits, it will still show up exactly like this in the raw
file (as the raw file is a reduced bit-depth version of the data
from the A/D, and the reason for bothering with the extra bits is
to prevent quantization artifacts for doing WB and/or tone curves.)

I must admit I didn't look at the links presented as I'm really not bothered either way, I always pre-set WB when shooting RAW even if crudely. The A/D converter and pre-processor used in the K10D had a 16 point gamma pre-curve adjustment capability I believe. The inference that the data is pre-adjusted in K10D RAW files is what I was alluding to, whether it's the case (due to rudimentary channel gain adjustments) in my *ist D RAW files I have no idea. I for one will still be pre-setting WB when shooting RAW using a K10D regardless.

-- hide signature --

Rob

 distudio's gear list:distudio's gear list
Sony RX100 II Pentax K-5 IIs Pentax K-3 Pentax 645Z Nikon D750 +19 more
feltmate Regular Member • Posts: 414
Re: RAW vs White balance

Interesting. I've learned a bit about the inner-workings of my camera by reading this. I will continue to try to adjust my white balance as I shoot, which improves workflow time anyway. Ultimately, I don't find this bothersome though, since I don't usually print my histograms and I have been satisfied with my pictures so far. Anyway, interesting discussion nonetheless.

CurtisR Regular Member • Posts: 263
Re: I took 15 pictures... I took three more

There is a subtle difference in your histograms, Jonas. I did three (admittedly not retentive enough to post, but easilly acurate enough for the task at hand) shots, and when the exposure is cranked up a bit, your subtle differences become pronounced.

Clearly a good policy to use an apropriate white ballance for the situation - and even potentially use an incorrect one if you know you're going to be photographing specific bright colors.

I just wonder if the correction is digital or analog. Need to dive into the numbers more closely to figure that out, though. Very interesting stuff.

Jonas B Forum Pro • Posts: 14,596
Curtis, Blende8, Rob, others...

CurtisR wrote:

There is a subtle difference in your histograms, Jonas. I did
three (admittedly not retentive enough to post, but easilly acurate
enough for the task at hand) shots, and when the exposure is
cranked up a bit, your subtle differences become pronounced.

Clearly a good policy to use an apropriate white ballance for the
situation - and even potentially use an incorrect one if you know
you're going to be photographing specific bright colors.

I just wonder if the correction is digital or analog. Need to dive
into the numbers more closely to figure that out, though. Very
interesting stuff.

Curtis, Blende8, Rob, others...

Yes there small differences. Or subtle. I don't know what they are showing but I guess part of the differences are due to minor fluctuations with the shutter and ISO settings.

Still, there is a small difference when itcomes to the Tungsten setting in my series.
As the histogram we get from the tool linked to by vortout
( http://malykh.com/temp/2006-11-19/malykh-pef-2006-11-19.zip )

shows the data without gamma applied everything is to the left. The pictures themselves showed up normally on the camera display with the luminosity (grenn channels I guess) having a top at the center of the screen.

To get a better "picture" I loaded ACR and developed the three ISO200 pictures with all settings set to neutral, or zero, or straight, and the White balance to "custom" which in my case means "Daylight". I developed to aRGB color space as it is what I usualyy works with. I checked the histogram in ACR setting the color space to Pro Photo and sRGB and for Pro Photo you can see that the red channel moves to the right while aRGB and sRGB stayed nearly the same. Anyway, in Photoshop I made a selection of approx 200*200 pixels at the center of the firt picture and loaded it in the other pictures as well. Then I checked th channel histogram for the three pictures. Here:

My own conclusion for now is that I can safely leave the WhiBal setting at Sunshine all the time (as I shoot raw only) and take care of the colors at home when editng the pictures. The small differences I have been able to find are really small.

I may be wrong. It's easy to miss something and any comment or suggestion to look at this from another angle of view is welcome to me.

regards!

Jonas

distudio Veteran Member • Posts: 3,913
Re: Curtis, Blende8, Rob, others...

Jonas B wrote:

My own conclusion for now is that I can safely leave the WhiBal
setting at Sunshine all the time (as I shoot raw only) and take
care of the colors at home when editng the pictures. The small
differences I have been able to find are really small.

I may be wrong. It's easy to miss something and any comment or
suggestion to look at this from another angle of view is welcome to
me.

The variation is indeed small and could well be attributed to exposure related factors so my guess is that you are correct. What I'm keen to see now is the same set of tests performed using RAW files from a K10D as the source.

-- hide signature --

Rob

 distudio's gear list:distudio's gear list
Sony RX100 II Pentax K-5 IIs Pentax K-3 Pentax 645Z Nikon D750 +19 more
Keyboard shortcuts:
FForum MMy threads