ISO setting and noise

[Ellis Vener]

From my experience, I know that the LrC's highlights warnings (clipping) do not correspond to the raw data clipping.

You evaded my question: exactly how and by how much do they not correspond?

 

[SrMi]

From my experience, I know that the LrC's highlights warnings (clipping) do not correspond to the raw data clipping.

You evaded my question: exactly how and by how much do they not correspond?

LrC shows clipping when there is none, for example.

--
To see my work, please visit http://www.ellisvener.com
Or on Instagram @EllisVenerStudio

 

[Bill Ferris]

A test to see if noise increases or decreases with ISO numbers.

...

Keep in mind that, at best, your test only applies that model of camera. Not all cameras have lower noise at higher ISO settings.

I'm very much aware of that.

Indeed, I'm beginning to think that "lower noise at higher ISO numbers" is simply a result of some cameras applying noise reduction to raw files.

it might be for a few cameras, but not for most.

In fact, with some cameras, the raw files would be identical, except for the metadata stating what the ISO setting was.

I suspect that's the case here, but I don't know a way to edit the metadata. Surely all raw files with identical exposure and subject should be identical except for an ISO tag in the metadata.

No. When variable gain is used to implement an increase in ISO setting, on most digital cameras, the RAW pixel values for a given exposure will vary proportionately with the ISO setting.

Unfortunately Raw Digger seems to be applying the ISO tag as it loads the file, so that the ones with high ISO numbers are "overexposed", -- that is, over-lightened.

That would be correct outcome on camera where the ISO value affects the RAW pixel data values. Those pixels are already overly light, and some might be blown . It would also be the correct behaviour with cameras which don't use variable gain to affect image lightness, but you have RAW digger set to display as per the ISO setting.

To illustrate, here's a post from a couple of years ago using D500 files to show how variable ISO settings at the same exposure affects Nikon NEFs made by this camera: https://www.dpreview.com/forums/post/63481107

 

[knickerhawk]

From my experience, I know that the LrC's highlights warnings (clipping) do not correspond to the raw data clipping.

You evaded my question: exactly how and by how much do they not correspond?

The classic case is red flowers. Histogram displays of red flowers shown in-camera and in many raw processing tools such as LR and ACR are frequently way off-base with how much of the red channel is truly blown in raw.

Here is an excellent article on the issue (by Iliah Borg) that appears in the FastRawViewer blog. FastRawViewer is a companion program to Rawdigger. Both of these tools allow you to see accurate raw histograms. The article also dives into how these raw histograms and related tools can help you make decisions about which files to process and how to process them to avoid unnecessary clipping.

 

[tony field]

From my experience, I know that the LrC's highlights warnings (clipping) do not correspond to the raw data clipping.

You evaded my question: exactly how and by how much do they not correspond?

The classic case is red flowers. Histogram displays of red flowers shown in-camera and in many raw processing tools such as LR and ACR are frequently way off-base with how much of the red channel is truly blown in raw.

One of the problems with many Reds such as flowers and stage performance with artificial light is that they become clipped when converting to one off the RGB work spaces. This may give the illusion of a blown channel In the original image.

Here is an excellent article on the issue (by Iliah Borg) that appears in the FastRawViewer blog. FastRawViewer is a companion program to Rawdigger. Both of these tools allow you to see accurate raw histograms. The article also dives into how these raw histograms and related tools can help you make decisions about which files to process and how to process them to avoid unnecessary clipping.

--
Charles Darwin: "ignorance more frequently begets confidence than does knowledge."
tony

 

[Iliah Borg]

Surely all raw files with identical exposure and subject should be identical except for an ISO tag in the metadata.

ISO is not just a tag. Changing ISO results in changes to the pixel data in the RAW file.

If you shoot a scene at f/4 and 1/30s that requires ISO 1600 for a 'correct exposure' but you choose ISO 400 instead, the pixel data in the file will be 'underexposed'. If you choose ISO 6400 instead, the pixel data in the file will be 'overexposed'.

You can prove this to yourself. Grossly overexpose a scene using an ISO so high that large areas in the scene are saturated and blown, even the ones that are midtones in the scene itself. Then try to recover all of them in post using any tools at your disposal. You'll soon see that the pixel data is nowhere near the same as it would have been if you used something close to the 'correct' ISO.

Unfortunately Raw Digger seems to be applying the ISO tag as it loads the file, so that the ones with high ISO numbers are "overexposed", -- that is, over-lightened.

It's showing the actual pixel data.

I just read this thread:

https://www.dpreview.com/forums/thread/4556012

If that's factual information, your comments might be applicable to your camera, the Sigma fp ... but certainly not to all cameras. You should go to the source (Iliah Borg) to see how RawDigger handles the Sigma fp. My comments apply to the majority of other cameras.

Thanks for finding that discussion. The BaselineExposure tag is mostly a constant one stop (zero to one stop) while real ISO increases as the ISO setting increases. It is not only the ISO tag in metadata that is changing with increasing ISO.

See this article about tags in Fuji's ISO bracketing:A Curious Case of ISO Bracketing with Fujifilm X-series:

If someone wants to see the real RAW exposure, then one will need to uncheck the 'Apply Adobe hidden exposure correction' checkbox (like for other tags).

To enable linear mode in FastRawViewer:

Preferences -> Image Display -> "Ignore exposure correction/baseline exposure in linear mode"

To switch linear mode on/off, press Shift-L.

Works with both v.1 and v.2:

New View Mode, "without Beautifications"

So, to clarify: Using the testing scenario I described, is D Cox correct or incorrect in saying that the actual pixel data generated by his Sigma fp would remain essentially unchanged regardless of ISO?

To answer, I would need to examine original raw files

The test I do: flat grey target filling the frame, camera focused at infinity, fixed light, fixed aperture, fixed shutter speed, vary ISO on the camera. If the raw values are the same for each ISO setting ISO is just a tag.

To clarify: by "same raw values" you mean the histograms and not the brightness in the viewer.

Yes, raw histogram, and statistics.

 

[alfn]

I know that my experience here is of no value to most folks on the dpr forums, but I still wonder if anybody else utilizes jpegs sooc in the audience. Reason being that among the large amount of digital cameras that I have owned since their first release 20+ years ago, one thing has been prevalent. I have never owned a single one that did not provide the best sooc jpeg images at the lowest iso setting available. This includes cameras from the simplest point and shoots to professional dslr's. And without exception, the best quality sooc images from every one of them came from using the lowest iso settings appropriate for the situation. And actually the same experience existed for film cameras for the 40 or so years preceeding that. Anybody else have this belief? It makes life simpler for me in that it's one setting that leaves no doubt where it should be in my camera setup. "Use the lowest iso possible for the lighting and action situation at hand" if I want the highest quality images possible.

The best quality generally comes from using the highest exposure that doesn't blow out important highlights. If you are using the camera in an automatic mode, you can come close to this by using the base ISO setting (typically ISO 100).

If subject lighting, depth of field concerns, or motion blur issues keep you from hitting that exposure, you can maximize your exposure by:

• Choosing the widest aperture that yields sufficient depth of field
• Choose the slowest shutter speed that does not result in unwanted motion blur
• Use Auto-ISO to set the corresponding ISO

The good old exposure triangle.

 

[SrMi]

I know that my experience here is of no value to most folks on the dpr forums, but I still wonder if anybody else utilizes jpegs sooc in the audience. Reason being that among the large amount of digital cameras that I have owned since their first release 20+ years ago, one thing has been prevalent. I have never owned a single one that did not provide the best sooc jpeg images at the lowest iso setting available. This includes cameras from the simplest point and shoots to professional dslr's. And without exception, the best quality sooc images from every one of them came from using the lowest iso settings appropriate for the situation. And actually the same experience existed for film cameras for the 40 or so years preceeding that. Anybody else have this belief? It makes life simpler for me in that it's one setting that leaves no doubt where it should be in my camera setup. "Use the lowest iso possible for the lighting and action situation at hand" if I want the highest quality images possible.

The best quality generally comes from using the highest exposure that doesn't blow out important highlights. If you are using the camera in an automatic mode, you can come close to this by using the base ISO setting (typically ISO 100).

If subject lighting, depth of field concerns, or motion blur issues keep you from hitting that exposure, you can maximize your exposure by:

• Choosing the widest aperture that yields sufficient depth of field
• Choose the slowest shutter speed that does not result in unwanted motion blur
• Use Auto-ISO to set the corresponding ISO

The good old exposure triangle.

In case you were not kidding: since ISO is not part of the exposure, there is no such thing as exposure triangle.

--
When the fun stops, stop.

 

[Iliah Borg]

The good old exposure triangle.

And quite poor.

 

[Michael Fryd]

I know that my experience here is of no value to most folks on the dpr forums, but I still wonder if anybody else utilizes jpegs sooc in the audience. Reason being that among the large amount of digital cameras that I have owned since their first release 20+ years ago, one thing has been prevalent. I have never owned a single one that did not provide the best sooc jpeg images at the lowest iso setting available. This includes cameras from the simplest point and shoots to professional dslr's. And without exception, the best quality sooc images from every one of them came from using the lowest iso settings appropriate for the situation. And actually the same experience existed for film cameras for the 40 or so years preceeding that. Anybody else have this belief? It makes life simpler for me in that it's one setting that leaves no doubt where it should be in my camera setup. "Use the lowest iso possible for the lighting and action situation at hand" if I want the highest quality images possible.

The best quality generally comes from using the highest exposure that doesn't blow out important highlights. If you are using the camera in an automatic mode, you can come close to this by using the base ISO setting (typically ISO 100).

If subject lighting, depth of field concerns, or motion blur issues keep you from hitting that exposure, you can maximize your exposure by:

• Choosing the widest aperture that yields sufficient depth of field
• Choose the slowest shutter speed that does not result in unwanted motion blur
• Use Auto-ISO to set the corresponding ISO

The good old exposure triangle.

The three main factors in exposure are subject lighting, aperture and shutter speed.

It's been that way since the days of film. When shooting film, over exposure resulted in a dense negative, and under exposure resulted in a thin negative. How dark or light the print looked was determined in the darkroom when you printed. A dense negative could be printed to produce a dark print, a light print, or a normal print.

With digital, you still have the same three factors determining exposure (subject lighting, aperture and shutter). The ISO setting provides alters the context for interpreting that exposure. The same exposure can result in a dark, light, or normal JPEG, depending on the ISO setting.

I have yet to hear a coherent explanation of how a Triangle is supposed to help people understand exposure. It's an additive process. More subject lighting increases exposure. A wider aperture increases exposure. A longer shutter increases exposure. Combine the contributions of these three and you get your total exposure.

 

[Bob A L]

Which combined with the senitivity of the recording medium to light affects the brightness of the photo as observed on a print or screen.

 

[knickerhawk]

Which combined with the senitivity of the recording medium to light affects the brightness of the photo as observed on a print or screen.

So does the brightness of the monitor being used to display the image or the wattage of the bulb illuminating the print. Your point being?

 

[Iliah Borg]

Which combined with the senitivity of the recording medium to light affects the brightness of the photo as observed on a print or screen.

Sensitivity doesn't affect brightness.

Sensitivity is the lowest useful signal level, given the minimum allowed SNR.

Brightness happens when there is light.

The word is "responsivity", and it is a constant for a given photodiode in a sensor.

--
http://www.libraw.org/

 

[Michael Fryd]

Which combined with the senitivity of the recording medium to light affects the brightness of the photo as observed on a print or screen.

Not really.

Consider a typical modern digital camera. If the sensor doesn't detect a signal at ISO 100, it won't detect that signal at ISO 128000. ISO doesn't change the sensitivity of the sensor.

In fact, a number of knowledgeable people have cited cameras that produce the same raw data at ISO 100 as ISO 1600.

How dark or light a JPEG looks is dependent on how the raw data is processed. A typical raw file can be processed to produce a dark, light, or normal looking JPEG. One cannot change the sensitivity of the physical sensor by changing how you process the raw data.

 

[Dem Bell]

I have yet to hear a coherent explanation of how a Triangle is supposed to help people understand exposure.

It is not supposed to do that and it doesn’t.

It's an additive process. More subject lighting increases exposure. A wider aperture increases exposure.

and decreases depth of field.

A longer shutter increases exposure.

and increases motion blur.

Combine the contributions of these three and you get your total exposure.

I am not interested in "exposure", I am interested in image properties such as "depth of field", "motion blur" and "noise". The triangle demonstrates how these 3 are connected and can be traded one against another by adjusting 3 camera controls: f-stop, shutter speed and ISO setting while keeping the same image lightness under the given constant lighting.

This is how autoexposure works in any camera. The triangle is a list of (f-stop, ss, ISO) options which will give the same image lightness under given lighting. Which option is chosen will NOT affect the image histogram but will affect how the image looks in terms of ("depth of field", "motion blur", "noise").

If one shoots raw, they might find the triangle confusing because they can no longer control image noise with camera's ISO setting. There is an additional step of adjusting "Exposure" in post processing which affects both image lightness and the amount of noise in each tone.

 

[D Cox]

A test to see if noise increases or decreases with ISO numbers.

How did you control for variations in other variabes that might conceivably affect noise?

The camera is a Sigma fp, which is believed to use a Sony sensor, probably the same as the one in the A72 or possibly A73.

It was mounted on a copy stand and a release cable was used to fire the shutter.

The scene consists of some very small print on a CD sleeve, a white ceramic glazed tile as used in kitchens, and a Color Checker. The lens is a Topcon 58mm Macro, set at f/16. The shutter speed was set to 1/50 and not changed.

The raw files (DNG format) were processed in Adobe Camera Raw. The "Exposure" (sic) slider was used to adjust lightness so that the histogram of each shot filled the width. There is an ISO tag in these raw files to tell the software to adjust lightness -- the higher the ISO the lighter the image when first loaded in.

The test area is a square part of the white tile. This was selected in Photoshop by using snap-to-grid to pick a 3x3 squares measuring area. The Histogram display in Photoshop includes a measurement of the Standard Deviation, which indicates the noise level.

Results:

ISO SD

100 1.73

400 1.63

800 1.72

1600 2.00

3200 2.31

6400 2.20

1280 2.17

25600 2.29

I see no obvious sign of the Standard Deviation going down with higher ISO settings. However, owners of other cameras should do a similar test with raw files.

And perhaps use RAW Digger, and a uniform exposure across the frame

You can use a selected area in Raw Digger.

So why might one expect SNR to improve at higher ISO values? This will only occur if two conditions are met:

1. The camera uses variable gain to implement an increase in ISO setting.

2. The camera adds some noise at or after the variable gain stage.

Together, these to conditions see the following results from an increase in ISO setting:

The signal is multiplied by the gain factor, The noise present just before the variable gain stage is multiplied by the same factor, The noise added afterwards is nor multiplied. As a result, since all the signal but only most of the noise was multiplied, so we expect the SNR to increase.

The vast majority of digital cameras fulfil both conditions. Newer cameras add much less noise at or after the gain stage so the effect is not nearly so pronounced as it was on, for instance, older Canon models. You can see this effect of increased variable gain on SNR in Bill Claff's Shadow Improvement charts

However, a few digital cameras do not fulfil either condition. For instance, RED digital video cameras produce the same RAW pixel data regardless of ISO setting, but the ISO value is stored as a tag in the RAW file and when the RAW file is processed, the ISO value is used to adjust the image lightness.

You say "There is an ISO tag in these raw files to tell the software to adjust lightness -- the higher the ISO the lighter the image when first loaded in." This would strongly imply that your Sigma camera does not use increased variable gain to produce an image with a higher ISO setting. Therefor we would not expect to see any increase in SNR as ISO value is increased in photos from this model of camera.

And we don't.

However, I think those who have been saying that "Raising the ISO setting reduces noise" are now admitting that this is only because some cameras apply noise reduction to raw files (as well as to JPGs), with stronger NR at higher ISO settings.

Which is not what I want my cameras to do. I prefer to do NR, when I want it, with a specialist program. Algorithms in cameras are designed for speed, not for quality.

Don

Raising ISO must not mean that NR is added to the raw files. For example, it does not occur with my Leica, Sony, and Nikon cameras, AFAIK. Please let me know if you have measurements that show otherwise. AFAIK, only long exposure NR affects raw files. All high ISO NR is only applied to JPEGs.

No, I do not think anyone says that raising the ISO settings reduces noise because of stronger NR at higher ISOs.

That's the conclusion I have come to. It doesn't apply to all cameras.

Do you have any other explanation ? Or is it just magic ?

P2P measurements show that raising ISO reduces noise. If you look at ISO alone (no automatic metering from the camera involved), you will observe that increasing ISO, and everything else remaining equal, reduces noise (increases SNR). That is why some like to use ITTR (ISO to the right).

In my book, two stops ISO movements show not enough SNR improvements to be used for improving IQ. Instead, and at higher ISOs, I'd rather apply one-stop negative EC to the ISO to protect highlights. This works especially well in the Misomatic mode.

How do you think a camera can reduce noise at high ISO settings ?

The method used is may be different for shot noise and read noise.

Don

 

[D Cox]

Bob, my guess is that the catalyst for this thread was a recent discussion in a different thread about the amount of noise an ISO setting contributes to a photo. As you know, the topic of noise in photos is a contentious one. Disagreements are sometimes the product of miscommunication between the participants.

To illustrate, one person might say, "High ISO images are noisier," and somebody might respond, "High ISO images are not noisier." The immediate impression one might take from this exchange, is that the two participants fundamentally disagree. However, there is a scenario in which the two people may largely be in agreement.

Suppose the underlying and unspoken context of the first comment is that it's an observation about the prominence of shot noise in a photo made with a low exposure. A high ISO was used to deliver a photo having a pleasing lightness but the underlying weak exposure results in an image displaying prominent shot noise.

Suppose the second comment is made from an underlying but unspoken context as a clarification that noise gets more prominent in low light photos but it's not the high ISO that's the primary contributor. It's shot noise that we see. Read noise decreases as ISO increases so, isn't the culprit.

In this scenario, it's possible the participants agree that shot noise is what we're seeing. But they may be misunderstanding each other. The second commenter may assume (incorrectly) that the first commenter is making an observation about the contribution of read noise to an image's appearance. The first commenter may misunderstand the response as denying that shot noise becomes more prominent as exposure (and the total light used to make a photo) decreases.

This "disagreement" arises from miscommunication; an absence of context or clarifying info in the written comments. The unfortunate outcome is an appearance of a fundamental disagreement about a topic on which the participants are broadly in agreement.

Now, I can't speak for the OP or any participants in this thread other than myself. My sense - and I may be mistaken - is that there's some miscommunication at play, here. That's why I posted a series of graphs illustrating that read noise does, in fact, tend to decrease as ISO increases.

https://www.dpreview.com/forums/post/66429775

So, when some folks talk about the common misperception that noise increases as ISO increases, those comments can be misunderstood as a denial that shot noise becomes more prominent as exposure decreases and less total light is used to make a photo. In fact, just the opposite is true. Calling out the high ISO noise misperception is an acknowledgement that photos look noisier as ISO increases. However, the reason for that noisier look is less total light being used (more prominent shot noise) and not, as some mistakenly believe, the result of increased read noise at higher ISOs.

The structure of this argument goes something like this:

• Shot noise, which is determined by the total light used to make a photo, is typically the most prominent kind of noise we see in a photo.
• Shot noise becomes more prominent as exposure (and total light used) decreases.
• While the ISO used tends to increase as exposure decreases, the resulting correlation between high ISOs and noise visibility, is not evidence of causation.
• It's not the read noise at high ISOs that makes photos look noisier. In fact, the amount of read noise is probably lower than the read noise in a well-exposed photo made at a low ISO.
• It's the reduced signal-to-noise ratio (SNR) and corresponding increase in the prominence of shot noise that we see.
• At very high ISOs, pattern noise and other artifacts can make read noise stand out and be more prominent. However, these are outlier situations and atypical of most photography.

So, yes, photos made at low ISOs typically do look best and have the highest quality. They're well-exposed with a high SNR and minimally visible shot noise. As exposure gets lower and less total light is used to make a photo, shot noise becomes more prominent and ultimately annoying. We use higher ISOs when making these photos but it's not read noise that compromises image quality. It's the more prominent shot noise that we see.

The general claim is that noise is less at higher ISO settings at the same exposure (aperture and speed). Nobody denies that when you raise the ISO the metering system will give less exposure, and hence more noise. The claim is that, at any given manually set exposure, the higher the ISO number the better. This doesn't seem to be the case with my tests.

Don

 

[D Cox]

Which combined with the senitivity of the recording medium to light affects the brightness of the photo as observed on a print or screen.

Sensitivity doesn't affect brightness.

Sensitivity is the lowest useful signal level, given the minimum allowed SNR.

Brightness happens when there is light.

The word is "responsivity", and it is a constant for a given photodiode in a sensor.

I don't think all users of digital cameras realise that.

The only "film" you can use in your camera is Ektachrome 100.

Don

 

[Dem Bell]

The general claim is that noise is less at higher ISO settings at the same exposure (aperture and speed). Nobody denies that when you raise the ISO the metering system will give less exposure, and hence more noise. The claim is that, at any given manually set exposure, the higher the ISO number the better.

No, the claim is that you get lower noise by shooting raw at a higher ISO compared to using an ISO say 5 stops lower and then pushing the raw file 5 stops up in post processing. On some (isoinvariant) cameras this difference will be negligible. Canon 5D, for example, was not one of those, but most modern cameras to a good approximation are isoinvariant, so using a slightly lower ISO and then pushing the raw file a couple of stops up in post will not be detrimental to image quality, in fact, it might save some blown highlights and thus increase dynamic range.

Your test indicates that you get lower noise when pushing a raw file from ISO 100 (should it be ISO 200?) to match the image taken at ISO 25600. This can't be right.

This doesn't seem to be the case with my tests.

My interpretation of the results is that the method is not accurate enough to tell the difference between noise in the range ISO 3200 - ISO 25600 (all SD values are about 2.25) and then the method breaks down at lower ISOs (all SD values are about 1.70).

 

[FingerPainter]

The general claim is that noise is less at higher ISO settings at the same exposure (aperture and speed).

Well that's half of it. You left out something in the middle

A more correct description would be:

The general claim is that cameras that use an increase in gain to implement a higher ISO setting, and which add some noise at or rafter the variable gain stage will produce images with a higher SNR when using a higher ISO setting for a given exposure.

Nobody denies that when you raise the ISO the metering system will give less exposure, and hence more noise. The claim is that, at any given manually set exposure, the higher the ISO number the better. This doesn't seem to be the case with my tests.

Well, according to you, your camera doesn't use variable gain to implement ISO changes. If you were correct, we'd expect the same SNR at all ISO settings for a given exposure. And according to you, there is no difference in SNR at various different ISO settings for the same exposure on your camera.

Bill Claff's tests seem to show that SNR does indeed change with ISO setting for a given exposure on your camera model. They also seem to indicate use of variable gain. Can you tell us why we should believe you rather than Bill?