Get to learn your 40D - which ISO to use

Started Nov 14, 2007 | Discussions
GaborSch Veteran Member • Posts: 7,203
Get to learn your 40D - which ISO to use

The question of ISO and noise has been discussed to death and past.

However, there is a lesser known but not less important fact to know about the 40D's characteristics:

the clipping point depends on the ISO.

Following is the list of clipping points, depending on the ISO, but first a few notes:

1. the clipping points may vary. I saw images from another copy with slight variations,

2. the clipping points may depend on the color, though not too much. Even more, the two greens may have different clipping points,

3. the clipping points do not reflect the black current level, which is between 1025 and 1035 (again with variations).

However, the above variations mean only a small difference between the different values. I menationed these only to barr further fruitless discussions about irrelevant details.

So, the clipping points are

ISO 0100: 13824
ISO 0125: 16383
ISO 0160: 12743

ISO 0200: 16224
ISO 0250: 16383
ISO 0320: 12743

ISO 0400: 16224
ISO 0500: 16383
ISO 0640: 12743

ISO 0800: 16224
ISO 1000: 16383
ISO 1250: 12743

ISO 1600: 16224
ISO 3200: 16383

What does this mean for the photographer?

It means, that the highlights may be clipped simply due to the bad choice of ISO. The difference between the worse, 12743 and the best, 16383 is 31%, after discounting the black current level.

This is * 1/3 of a stop * one can lose, simply by selecting ISO 100 or some other ISOs.

I shot a scenery for demonstration, the dynamic range of which exceeds the 40D's capturing capability at good light:

The test serie consists of 10 shots with ISO 100 and ISO 125, each. Note, that there are a few specular spots on the vase. Their effect is visible on the histograms only at very close look, as a thin, short line at the end of the respective range. This is irrelevant for the following test serie, as only a few pixels are effected by this extreme overexposure.

The histograms show clearly, what is happening. The ISO and exposure is in the head of the histograms, which are shown pairwise, the left one is with ISO 100, the right one with ISO 125 but 1/3 EV lower exposure.

These histograms reflect the RAW data, i.e. no adjustments have been applied.

What are the consequences?

1. DO NOT use a "bad ISO", namely ISO 100, ISO 160, ISO 320, ISO 640, ISO 1250,

2. DO NOT use ISO 200 with HTP On, as this ends up with using ISO 100.

In case of sufficient interest, I will post detailed analysis of the effect of these phenomena on the raw processing, with particular attention to DPP and ACR.

JimH Forum Pro • Posts: 12,911
What I don't get:

OK, all of the "real" ISOs have the same clipping point (16224) except for ISO 100. But in previous Canons, it's been known that ISO 100 had less highlight headroom.

It's not surprising to me to see that things are essentially the same with the 40D. ISO 100 has a lower clipping point than the others and this might be because we're actually seeing the full-well capacities of the sensor sites at ISO 100 but at the higher "real" ISOs, the well capacities are no longer the limit.

So far, everything seems logical and expected.

But since the intermediate ISOs are simply calculated from RAW data actually captured at one of the nearby "real" ISOs, I'd expect to see the following:

Observed Expected(by me)

ISO 0100: 13824 Fine, whatever the full wells will give
ISO 0125: 16383 Fine, max allowed by 14 bits
ISO 0160: 12743 10816

ISO 0200: 16224 Fine, not a full-well limit, but some arbitrary cutoff
ISO 0250: 16383 Fine again, max allowed by 14 bits
ISO 0320: 12743 10816

ISO 0400: 16224 Fine again, as with 200
ISO 0500: 16383 Fine again, max allowed by 14 bits
ISO 0640: 12743 10816

ISO 0800: 16224 Fine again, as with 200
ISO 1000: 16383 Fine again, max allowed by 14 bits
ISO 1250: 12743 10816

ISO 1600: 16224 Fine again, as with 200
ISO 3200: 16383 Fine yet again, max allowed by 14 Bits

The fake ISOs just above a "real" ISO should be the result of multiplying the data from that real ISO upwards by 1.333... And by doing that, we simply run into the limit of a 14 Bit number or 16383. So no surprise there.

But what's odd are the fake ISOs achieved from the real ISO just above them. If we take 16224 * 0.666..., we should get 10816 NOT 12743. Even if we started with 16384 and multiplied by 0.666... we'd have a max possible value of 10922.

Maybe I'm missing something, but I thought the intemediate ISOs were always derived from the nearest "real" ISO. If so, from where does this 12743 figure arise?

That's what's got me wondering.

And it is ironic that when we shoot at ISO 2oo (HTP Enabled), we end up shooting at the ISO that has the lowest DR and is the most likely to actually clip the highlights

-- hide signature --

Jim H.

The Digital Mage Regular Member • Posts: 147
Re: Get to learn your 40D - which ISO to use

this is useful information - thank you.

really don't care about the techno-speak. guess i'm a techie burnout.

i wrote your list of bad ISO numbers down. if i can add anything, it's that there is little difference indeed in the noise between ISO 400 and 500.

AdamJRed Contributing Member • Posts: 545
Re: What I don't get:

Jim, that is indeed very odd. I thought exactly the same about the highligh clipping values for the +2/3rd ISO values.

I believe you are correct in that the +2/3rds ISO values are actually using the next 'full' ISO stop at the sensor and reducing the values after the fact, hence the output of my discussion on noise vs. ISO -> http://forums.canonphotogroup.com/showthread.php?t=958

My guess is that something more then a simple calculation ( x 0.666..) is taking place and that is the reason why the +2/3rds highlight clipping occurs where it does...

 AdamJRed's gear list:AdamJRed's gear list
Fujifilm X-T1 Fujifilm XF 18-55mm F2.8-4 R LM OIS Lytro Light Field 8GB Fujifilm 16-55mm F2.8R LM WR
mmullen Veteran Member • Posts: 4,289
Re: Get to learn your 40D - which ISO to use

GaborSch wrote:

It means, that the highlights may be clipped simply due to the bad
choice of ISO. The difference between the worse, 12743 and the best,
16383 is 31%, after discounting the black current level.

There are no "bad" ISO's. Each ISO has different characteristics with regard to noise and highlight headroom. For example, most of the ISO's you think should be avoided, also have lower noise than the next lowest ISO. This tells me for lower contrast scenes ISO's 160, 320, 640 and 1250 will provide the best image quality for the exposure (less noise and enough dynamic range) while for very contrasty scenes you may want to avoid these ISO's for the most dynamic range at the expense of more noise.

Interesting how the two are related.

-- hide signature --

Mike Mullen

 mmullen's gear list:mmullen's gear list
Canon EOS 40D Canon EOS 7D Canon EF 85mm F1.8 USM Canon EF 300mm f/2.8L IS USM Canon EF-S 10-22mm F3.5-4.5 USM +6 more
jlalor Contributing Member • Posts: 790
Interesting analysis... thank you! (nt)
-- hide signature --

Jonathan

'Photography is a money-sucking black hole, and I've crossed the event horizon'

bcatwilly Regular Member • Posts: 221
Re: Get to learn your 40D - which ISO to use

Interesting stuff, thanks.

-- hide signature --

Canon EOS 40D, Olympus C-5050z, Kodak DC-4800

Chief Senior Member • Posts: 1,531
How did you determine the clipping points?

I don't see 31% difference in real life shots ... what did you use to determine these numbers?

-- hide signature --

Regards,
John

OP GaborSch Veteran Member • Posts: 7,203
Calculation

JimH wrote:

in previous Canons, it's been known that ISO
100 had less highlight headroom

Yes with the 30D, but no with the 20D, which goes up to 4095 even w. ISO 100

The fake ISOs just above a "real" ISO should be the result of
multiplying the data from that real ISO upwards by 1.333...

The factor should be cuberoot(2), which is 1.26

If we take 16224 * 0.666..., we should get 10816 NOT
12743. Even if we started with 16384 and multiplied by 0.666... we'd
have a max possible value of 10922

If you go backwards, you have to take the reciprocate of the "upwards factor". That would be 0.75 with 1.333. With 1.26 it is 0.79...

A further error is to calculate directly with the pixel values of the raw file. One has to remove the black level before calculation and add it afterwards.

Thus (16224 - 1028) / 1.26 + 1028 = 13088 (at least this is, how it is "supposed to be").

And it is ironic that when we shoot at ISO 2oo (HTP Enabled), we end
up shooting at the ISO that has the lowest DR and is the most likely
to actually clip the highlights

That was my point. In fact, the camera offers a "mini HTP" by selecting the 1/3 stop higher ISO (but only for raw shooters).

OP GaborSch Veteran Member • Posts: 7,203
Difference in real life shot

Chief wrote:

I don't see 31% difference in real life shots

That's normal. The difference is in the very highlights, when exposing to the right (or when the dynamic range is in fact very large). This range gets heavily "compressed" by the mapping over the RGB space, except if you are increasing the contrast of the highlights. For example if you want to see contrasty clouds, you have to adjust the brightness and contrast (perhaps with curves), and then the 1/3 stop makes a big difference.

what did you use to determine these numbers?

I shot a white board with very high overexposure (three-four stops over the exposure the camera calculated) and analyzed the raw images; see http://www.cryptobola.com/Photobola/rawnalyze.htm . The histogram shows the clipping points only raughly, the exposure analysis reveals it pixel-wise, even that half of the greens (those in the top right position of the filter pattern) have a different clipping point, than the other half. This difference is only 1 with my camera, but I saw one with a much higher difference. (This is irrelevant, but I find it interesting.)

John down under Veteran Member • Posts: 8,238
noise for 'good' ISOs?

GaborSch wrote:

So, the clipping points are

ISO 0100: 13824
ISO 0125: 16383
ISO 0160: 12743

ISO 0200: 16224
ISO 0250: 16383
ISO 0320: 12743

ISO 0400: 16224
ISO 0500: 16383
ISO 0640: 12743

ISO 0800: 16224
ISO 1000: 16383
ISO 1250: 12743

ISO 1600: 16224
ISO 3200: 16383

Gabor, thanks for an interesting analysis. Have you looked at noise between the sets of good ISOs, ie 200 vs 250, 400 vs 500 and 800 vs 1000? White clipping point is of course one consideration, but do the 'in-between' ISOs of 250, 500 and 1000 suffer from noticeably more noise than the 'even' ISOs of 200, 400 and 800? Are those in-between ISOs achieved by calculating from the even ISOs rather than from adjusting amplification as used to achieve different even ISOs?

Also, John Sheehy's analysis from a while back showed that the 20D also has white clipping problems with ISO100, so I use ISO200 instead as my default unless ISO100 seems to be my best option for slowing things down at any time. John's noise analysis was also interesting to show how the in-between ISOs produce more noise than the even ISOs.

-- hide signature --

Cheers from John from Adelaide, South Australia
John Harvey Photography http://johnharvey.com.au
Canon 20D & Fuji F10

JimH Forum Pro • Posts: 12,911
OK, that makes sense.

I'm not sure why I was thinking at 1/3 stop would be 1/3

D'oh! I knew that a few weeks ago when I was calculating the f/stop for my pinhole lens

And the bit about subtracting the black point and then adding it back in later escaped me as well.

OK, all is well with the world

I also thought that the 20D shared the lower clipping point for ISO 100 with the 30D, but if it doesn't, that makes me wonder if Canon has set these calibrations a bit different for the cameras with the "fake" ISOs on purpose.

-- hide signature --

Jim H.

RawDogg Senior Member • Posts: 1,029
interesting indeed, thanks for the info (nt)
 RawDogg's gear list:RawDogg's gear list
Canon EOS M Canon EOS M3 Canon EOS 77D Canon EF 50mm F1.8 II Canon EF-S 18-135mm F3.5-5.6 IS STM +6 more
PopL Junior Member • Posts: 43
Re: Get to learn your 40D - which ISO to use

Ok, interresting. Just a question though, exactly what do you mean by "clipping point", and "black currant level"? Aren't these matter of interpretation, i.e. what looks white and what looks black, or wont the sensor record values higher or lower than those given?

-- hide signature --

Peter

GaborSch wrote:

The question of ISO and noise has been discussed to death and past.

the clipping point depends on the ISO.

3. the clipping points do not reflect the black current level, which
is between 1025 and 1035 (again with variations).

JimH Forum Pro • Posts: 12,911
Re: What I don't get:

I think Gabor has shown where my calculations were incorrect. The main bug was that I was, for some reason, totally forgetting that we're dealing with fractions of a "stop" and not simple linear fractions.

So 1/3 of a stop doesn't equate to 1/3. Instead, it's the cube root of 2 (1.26) as half stops are the square root of 2 (1.414) for the factor. I had completely spaced that out even though I used the right calculation just a few weeks ago when figuring the effective aperture of my nifty pinhole camera Go figure!

He also points out that the black levels are probably subtracted first, then added back in after the "gain" calculations are applied.

So it all makes sense.

But I read your post on the other site and that's all interesting.

Have you seen John Sheehy's posts about this on this forum?

Here's one I consider to be something of a "landmark":

http://forums.dpreview.com/forums/read.asp?forum=1019&message=19721647&changemode=1

I believe John was the first person to demonstrate that the intermediate ISOs in the 30D are "fake" and then that the noise levels at the different ISOs follow that pattern.

I am not sure, but I believe that the reason that the 5D may behave differently than the 30D or 40D in this respect is that the 50D does employ some form of auxiliary analog amplification to achieve the intermediate ISOs where the 30D does it purely in software (as the 40D probably also does).

But I've read conflicting analyses of the 5D, so I'm left wondering exactly how it does achieve its intermediate ISOs. Sometimes I read that it's all numeric manipulation and other times I read that it's done at least partially in the analog domain. If it is done via analog means, that may explain the fact that the graphs are similar, but offset from each other.

Perhaps John might pop in and enlighten us in this thread. I feel that he has an excellent grasp of this and I credit him with being the first person to discover that Canon was playing games to achieve the intermediate ISOs on the 30D.

-- hide signature --

Jim H.

theberkin8or Contributing Member • Posts: 832
Re: Get to learn your 40D - which ISO to use

This is why Dpreview is so useful. I would have never discovered this on my own. Thanks.
--

OP GaborSch Veteran Member • Posts: 7,203
Noise

John down under wrote:

Have you looked at noise
between the sets of good ISOs, ie 200 vs 250, 400 vs 500 and 800 vs
1000?

Honestly, I virtually never shoot with higher ISO. I shoot almost exclusively panos, mostly landscape; as such, low light is seldom a consideration for me.

I shot a white board with different ISOs for finding the clipping points, in order to complate the list (my raw analyzer has to kow these values).

Also, John Sheehy's analysis from a while back showed that the 20D
also has white clipping problems with ISO100

The 20D fills the full range (up to 4095) in ISO 100. I think what you refer to relates to the 30D.

OP GaborSch Veteran Member • Posts: 7,203
Black current, clipping

PopL wrote:

Ok, interresting. Just a question though, exactly what do you mean by
"clipping point", and "black currant level"? Aren't these matter of
interpretation, i.e. what looks white and what looks black, or wont
the sensor record values higher or lower than those given?

These are not what you can set in the raw processor as black point/white point.

1. The black current or black level is the value, what pixels yield even without being exposed. The Canon raw pixel values include the black level, and the raw processor has to reduce the pixel values. The camera measures these by reading so-called masked pixels. There are tens of thousands of them on the sensor: dozens for each row and each column (yes, cross-wise), and the raw processor has to figure out, which value to substract from which column.

On the raw histograms above you can see a gap at the left side; with the 40D, this is from 0 to around 1030 (it is not a fixed value, but it is within a tight range, about 1020 to 1040).

2. The clipping point is the maximum pixel value of a given color at a given ISO. Even though there are 14 bits, i.e. the value range is 0 to 16383, the actual pixel values are between 1028 (or so) and the clipping point.

In other words, the number of levels of the 40D with ISO 100 is 13824 - 1028, far from the 16384.

DRG Veteran Member • Posts: 5,217
Re: Calculation

GaborSch wrote:

JimH wrote:

in previous Canons, it's been known that ISO
100 had less highlight headroom

Yes with the 30D, but no with the 20D, which goes up to 4095 even w.
ISO 100

Don't forget John Sheehy's now-ancient posting in which he showed that the highlight tones were nonlinear in the 20D. The data "wanted" to clip at a lower value (likely due to well saturation, just as here and in the 30D) but Canon applied a superlinear stretch to pull the lower tone value up to 4095. It's as though Canon felt compelled to keep the highest tone value constant across ISO despite the sensor saturation actually conferring a lower clipping point. So, there may be little difference between the 20D, 30D and 40D models beyond this 20D stretching of the highlight tones (I don't recall if this stretching was digitally, after A/D conversion, or was implemented as a nonlinearity in the ISO 100 amplification stage).

The fake ISOs just above a "real" ISO should be the result of
multiplying the data from that real ISO upwards by 1.333...

The factor should be cuberoot(2), which is 1.26

The ratio is actually staring us in the face without needing to calculate the cube root. The ISO values themselves are listed in the table. If you want the correct ratio, just use the ISO value ratios (400/320 = 1.25, 250/200 = 1.25, etc.). Canon has apparently rounded off slightly.

David

DRG Veteran Member • Posts: 5,217
Re: Get to learn your 40D - which ISO to use

The behavior described here is consistent with what John Sheehy reported for the 30D. As in that case, I don't think the 160, 320, 640, 1250 ISO values should be considered the bad ones. Quite the contrary. Their behavior is simply a slight overexposure with a cranking back of the tone values digitally before storage as a raw file. Thus, shooting at ISO 320 is the same as shooting at ISO 400 with EC = +1/3 and then applying EC=-1/3 during conversion. I often shoot with EC = +1/3 or +2/3 because I find that Canon's metering typically leaves highlight headroom unused when it could actually be used to expose more to the right to improve noise. These full stop - 1/3 stop ISO values do exactly this. Just as you wouldn't use a positive EC if this would result in highlight clipping, you wouldn't use these ISO values in the same circumstances. But, for scenes that have no extreme highlight content and which thus "waste" the opportunity to collect more photons, these values work fine.

I consider the full stop + 1/3 stop ISO values to be the "bad" ones. That is, ISO 125, 250, 500 and 1000. These underexpose by a stop and then adjust the values digitally. They may seem to have a high clipping point, but this is really just a lower clipping point that's been digitally boosted up. The underexposure plus digital boost results in worse noise, however. You may as well shoot at, say, ISO 200 and use EC=-1/3 instead of ISO 250. At least then you'd be gaining something for your extra-noise troubles ... you'd be obtaining 1/3 stop highlight extension. You lose this when shooting at full stop + 1/3 stop because the camera digitally multiplies by 1.25 and then clips at 16383 even if that top 1/3 stop contains information.

David

GaborSch wrote:

What are the consequences?

1. DO NOT use a "bad ISO", namely ISO 100, ISO 160, ISO 320, ISO 640,
ISO 1250,

2. DO NOT use ISO 200 with HTP On, as this ends up with using ISO 100.

So you don't get a full stop extra highlight headroom, but you still do better headroom-wise, if shooting JPEG, than shooting at (ISO - 1 stop) without HTP.

Of course, there's no point using HTP at all if you shoot raw, as it results in the same raw data as the ISO one stop down, but with EC = -1 (plus a metadata flag that gets set so that DPP knows to apply the appropriate tone curve).

David

Keyboard shortcuts:
FForum MMy threads