This is odd... dss problem please help

Sir Canon said:

i did post the whole image. on the subject of longer integrations i absolutely believe you, honestly its incredible that rogger is getting nearly that good image quality. however to a point more subs aren't helping. the amount noise is decreased is an exponential decay. meaning that if you have 4 subs you will half the noise but you need 16 to reduce it by a factor of 4. so for my intents and purposes over 100 or so frames isnt going to make a difference. i restacked it and learned that i had a wrong combination method on. so here is what it looks like stacked up right. there is still a large lack of detail because of the lack of integration time however.



--
I tend to overdo things

Click to expand...

You are absolutely right that there is a point of diminishing returns. Read my recent reply to Roger here, where I explain why imaging under light polluted skies is inefficient and will usually be the most limiting factor. The best way to get better results is to move out of the city a bit to darker skies. Darker skies are radically more efficient. You usually don't have to go all that far most of the time, either. Most suburban areas are "red zone" (look up the Bortle scale if you don't know what that means, it's pretty self explanatory once you read about it ).

If you could find a decent yellow zone area and set up there, the difference would be like night and day. You wouldn't have to get hundreds of subs...in fact, you could probably get a lot fewer, just make each one longer. Longer subs at a darker location have better signal, so your imaging efficiency improves (details in my reply to Roger). When I was still imaging in my back yard, I kept going for more and more and more subs. I eventually got up to the point where I'd spend a month or even longer imaging the same object, such as Pleiades or Andromeda Galaxy, and I'd gather 260 to 280 subs! The results ended up pretty nice...but man it was a lot of work! My first dark site image, my own Orion image (which I think I shared in one of my previous posts) was a mere 2h 20m. It was 40x 210 second subs. Far more reasonable than over 260. Back then I wasn't as skilled at managing my guiding performance, and these days I'd probably have gone with 300 second subs, which would have allowed me to reduce my sub count to a mere 28.

I know that getting out to a dark site isn't always easy. I sometimes end up foregoing a clear night myself sometimes because I just don't have the time to get out to my dark site. There are other options to find a middle ground. One is to use a light pollution filter. With an LP filter, like an IDAS LPS-P2 (or LPS-D1 if you have a modded DSLR) or the IDAS LPS-V4, you can expose for longer in a light polluted zone. Usually not as long as if you were at a dark site, but it is a way to get better image signal in each sub and allow you to stack fewer subs and get a better result. The caveat to using filters is they throw away certain parts of the visible spectrum, and that affects color quality. An LPS-P2/D1 (which is mostly the same as an Orion SkyGlow Astrophotography filter) throws away less of the spectrum, while an LPS-V4 (which is mostly the same as an Orion SkyGlow BroadBand or an Astronomik CLS filter) throws away large chunks of the spectrum. The LPS-P2/D1 will give you better color, but you won't be able to expose as long per sub as with an LPS-V4, however the latter will have a pretty large impact on your color fidelity (you'll lose almost all star color, you can't really image galaxies or star clusters with it, and you might not even like the nebula color.)

jammeymc said:

There have been a lot of discussion regarding exposure max referencing 1/3 to 1/4. Just so I, and maybe others, are clear as to what is being stated here - is the attached what you would consider the max 1/3 exposure (such that read noise is met and so as to not loose DR)?

Click to expand...

That histogram is a great example. The 1/3rd rule is mostly a safety rule. It's easy, you don't have to think too hard about it, and you can check it quite simply on the back of your camera.

That said, I like this histogram because it actually shows exactly why you need to expose enough to "swamp read noise". Note the tails on the histogram. It peaks at around 1/3rd, but it has a long tail going off towards the right where it meets the right hand edge and then starts to climb it a bit. It also has another tail going off towards the left. That tail going off towards the left is the darkest pixels in the noise. That is the range of tones you have used in your image, from the tip of the left tail all the way to the tip of the right tail.

The key is that you get the tail that trails off to the left of the peak off the left-hand edge of the histogram. If the tail is touching the left hand edge, then you have under-exposed, and you are likely losing some detail that otherwise might have been recoverable if you had exposed a little longer. It's also best to give yourself a decent bit of room between the end of that left hand tail and the left edge. The reason for that is histograms represent the entire numeric range supported by the bit depth of your camera. In the case of a 14-bit camera, that is levels 0-16383. Most histograms do not have 16384 columns to accurately display every level in the image in the histogram, so the information displayed is "compressed". There are often some pixels that have even darker tones than where you see the left tail of the histogram end...so having some headroom back their is important to make sure you aren't clipping any information.

In this particular histogram, I would say that you probably could have gotten away with less exposure. This might have been just fine with a 1/4 histogram exposure. The back of camera histogram isn't a particularly accurate one, so in the end it's really mostly a guess where you should expose to in order to get the best exposure. Hence the recommendation to expose somewhere between 1/4 to 1/3rd histogram.

It should be noted that the width of the peak in the histogram will often vary. Sometimes it will vary throughout a single imaging session. The width of the peak is indicative of how much noise you have. The wider it is, the more noise you have...the narrower it is, the less noise you have. There is a lower limit on how little noise you can have...and that would be dictated by how strong the signal is. You also have noise from read noise as well as dark current, though. You also get additional noise from light pollution. Read noise is fixed for a given camera, and at higher ISO's with a DSLR, it is usually a small term, so it doesn't matter a lot (caveats there, I can explain if anyone is interested).

The bigger issues are dark current and light pollution. Dark current increases as temperature increases, so the hotter the camera sensor is, the more dark current noise you will usually have. That will cause the histogram peak to fatten, and the fatter the peak is, the more likely you will need to expose to 1/3rd rather than only 1/4. Light pollution adds unwanted signal, and like any other signal, it will also contain noise. The more light pollution you have, the fatter your histogram peak will usually be. The only thing you can really do about that is either add a light pollution filter, or better, find and use a site under darker skies. Same as with dark current noise, the more LP you have, the more likely you will need to expose to 1/3rd rather than only 1/4.

One final point. The reason you don't want to expose beyond 1/3rd histogram is to maximize your use of the available dynamic range. This histogram is a great example of why. The RIGHT-hand tail of the histogram stretches all the way to the right edge. That right hand tail is all your stars. They tend to expose much faster than the faint nebula and galaxies. In this particular histogram you will notice that the right hand tail actually climbs up the right edge a small amount. When you see that, it is an indication that your signal started to clip. The brightest stars in the image all exposed so much that the center pixels became pure white. If you were to expose beyond the 1/3rd point in the histogram, you would only be clipping more and more stars, and more heavily. With this particular histogram, a 1/4 exposure would have probably been better for the stars...however the difference would have been fairly small, and some stars would have still undoubtedly clipped (just fewer.)

FWIW, clipping stars a little bit is usually not a huge problem. IMO, it's preferable to underexposing too much, which tends to bury more details in noise. I myself usually opt to clip a few stars and get better SNR on the faint details than the other way around...but sometimes it depends on what your imaging. Regions with lots of bright stars might do better with 1/4 exposures than 1/3 exposures.

Jon Rista said:

As for read noise itself. If you stuck with the 1 minute exposures at the dark site, you would need to stack more than 24 subs to get the same SNR as with a single 24 minute sub. I know you know that, Roger, as you noted it in your comment above...but for other readers here. You would need around 42 subs, because of the compounding of read noise. SNR would be (10 * 42)/SQRT(42* (10 + 1 + 3^9)) = 14.5:1. A single 24 minute sub or 42 one minute subs. The choice there should be pretty easy. Personally, if your willing to image for 42 minutes, you might as well go for two 24 minute subs and your SNR will be up to 20.5:1.

Click to expand...

Jon, this really is getting tiring and I'm going to bow out as I really don't have the time to keep making corrections after this one, but for the sake of other astrophotographers, I'll try one more time.

Here is what you forgot. You keep harping on 1-minute subs. If you actually read what I write, I say expose to get 1/4 to 1/3 histogram peak on the sky. So my 7D2 that is around 65 photons / exposure. On a camera with larger pixels it would be more electrons. You conveniently forgot to include that. So here is what you would actually get with 24 minutes total exposure:

Say 1-minute exposure gave sky of 60 photons, subject 10 photons, read noise 2.4 electrons, dark current 5 electrons (0.083 e/sec).

In 24 1-minute subs S/N = 24 * 10 / sqrt(24* (60 + 10 + 2.4^2 + 5)) = 5.45

Now assuming no clipping or dynamic range issues, a single 24 minute exposure would have 1440 sky electrons, 240 subject electrons, read noise 2.4 electrons and 120 dark current electrons. Thus:

S/N = 240 / sqrt(1440 + 240 + 2.4^2 + 120) = 5.65, thus less than 4% difference. Again, you would not see any difference in the image.

How hard is this? The key is at 1/4 to 1/3 histogram, sky is much greater than read noise because 60 is much greater than 2.4 squared. Once you reach that level, red noise and thus sub exposure length is irrelevant.

And as read noise gets lower, you can do even shorter subs. For example, raise ISO, do 30 second subs. On the 7d2, raise ISO to 3200 and get 1.9 electron read noise, and we have the same dynamic range in 30 seconds as 60 second at iso 1600. We now do 42 subs to get to 24 minutes total integration time. We now get:

S/N = 42 * 5 / sqrt(42* ( 30 + 5 + 1.9^2 + 2.5)) = 5.05, or only 12% worse than a single long exposure. You would be hard pressed to tell the difference.

Go another factor of two shorter, raise ISO, then you would get:

S/N = 84 * 2.5 / sqrt(84* ( 15 + 2.5 + 1.7^2 + 1.25)) = 4.92, so only 15% worse than a single exposure. Again, it would be difficult to tell.

And of course in practice, people generally do not do 24-minute subs, so the real-world difference would be less. With 8-minute subs, you get S/N =80 * 3 /sqrt(3*(480 + 80 + 2.4^2 + 40)) = 5.63

So we now have a simple table

sub exposure___S/N for the above parameters

24 min_______5.65

_8 min_______5.63

_1 min_______5.45

0.5 min______5.05

0.25 min_____4.92

Jon Rista said:

This is really only viable if you are either A) using fast camera lenses, or B) using specialized ultra fast telescopes like the Celestron with Hyperstar, where you can get around f/2 optics.

Click to expand...

Again, you are not getting it. Nowhere do I say you should do minute exposures. I say exposure length to get to 1/4 to 1/3 histogram, whatever time that takes. If you follow this then read noise with modern cameras will have little significant impact. If you do less exposure than 1/4 histogram, read noise becomes a larger contributor, but it has small impacts until sub exposure times are very short.

Roger

rnclark said:

Jon Rista said:

As for read noise itself. If you stuck with the 1 minute exposures at the dark site, you would need to stack more than 24 subs to get the same SNR as with a single 24 minute sub. I know you know that, Roger, as you noted it in your comment above...but for other readers here. You would need around 42 subs, because of the compounding of read noise. SNR would be (10 * 42)/SQRT(42* (10 + 1 + 3^9)) = 14.5:1. A single 24 minute sub or 42 one minute subs. The choice there should be pretty easy. Personally, if your willing to image for 42 minutes, you might as well go for two 24 minute subs and your SNR will be up to 20.5:1.

Click to expand...

Jon, this really is getting tiring and I'm going to bow out as I really don't have the time to keep making corrections after this one, but for the sake of other astrophotographers, I'll try one more time.

Here is what you forgot. You keep harping on 1-minute subs. If you actually read what I write, I say expose to get 1/4 to 1/3 histogram peak on the sky. So my 7D2 that is around 65 photons / exposure. On a camera with larger pixels it would be more electrons. You conveniently forgot to include that. So here is what you would actually get with 24 minutes total exposure:

Say 1-minute exposure gave sky of 60 photons, subject 10 photons, read noise 2.4 electrons, dark current 5 electrons (0.083 e/sec).

In 24 1-minute subs S/N = 24 * 10 / sqrt(24* (60 + 10 + 2.4^2 + 5)) = 5.45

Now assuming no clipping or dynamic range issues, a single 24 minute exposure would have 1440 sky electrons, 240 subject electrons, read noise 2.4 electrons and 120 dark current electrons. Thus:

S/N = 240 / sqrt(1440 + 240 + 2.4^2 + 120) = 5.65, thus less than 4% difference. Again, you would not see any difference in the image.

How hard is this? The key is at 1/4 to 1/3 histogram, sky is much greater than read noise because 60 is much greater than 2.4 squared. Once you reach that level, red noise and thus sub exposure length is irrelevant.

And as read noise gets lower, you can do even shorter subs. For example, raise ISO, do 30 second subs. On the 7d2, raise ISO to 3200 and get 1.9 electron read noise, and we have the same dynamic range in 30 seconds as 60 second at iso 1600. We now do 42 subs to get to 24 minutes total integration time. We now get:

S/N = 42 * 5 / sqrt(42* ( 30 + 5 + 1.9^2 + 2.5)) = 5.05, or only 12% worse than a single long exposure. You would be hard pressed to tell the difference.

Go another factor of two shorter, raise ISO, then you would get:

S/N = 84 * 2.5 / sqrt(84* ( 15 + 2.5 + 1.7^2 + 1.25)) = 4.92, so only 15% worse than a single exposure. Again, it would be difficult to tell.

And of course in practice, people generally do not do 24-minute subs, so the real-world difference would be less. With 8-minute subs, you get S/N =80 * 3 /sqrt(3*(480 + 80 + 2.4^2 + 40)) = 5.63

So we now have a simple table

sub exposure___S/N for the above parameters

24 min_______5.65

_8 min_______5.63

_1 min_______5.45

0.5 min______5.05

0.25 min_____4.92

rnclark said:

This is really only viable if you are either A) using fast camera lenses, or B) using specialized ultra fast telescopes like the Celestron with Hyperstar, where you can get around f/2 optics.

Click to expand...

Again, you are not getting it. Nowhere do I say you should do minute exposures. I say exposure length to get to 1/4 to 1/3 histogram, whatever time that takes. If you follow this then read noise with modern cameras will have little significant impact. If you do less exposure than 1/4 histogram, read noise becomes a larger contributor, but it has small impacts until sub exposure times are very short.

Roger

Click to expand...

Wow, so arrogant, when you are entirely missing the point. For the sake of other astrophotographers, one last time. It's not specifically about 1-minute exposures. It's not about read noise (and you ironically quoted the one thing in my rather lengthy previous post where I said anything about read noise. *rolleyes*) It's about light pollution, how light pollution adds noise, and how light pollution is NOT a constant flux everywhere. DARK SKIES vs. LIGHT POLLUTED SKIES. It's about how higher photon flux from light polluted skies implicitly limit your maximum exposure time (if your trying to avoid exposing the skyfog peak in the histogram beyond 1/3rd.)

It's not the 1-minute subs that I'm "harping" on. It's the difference between a 1/3rd histogram exposure in a light polluted zone, and a 1/3rd histogram exposure in a dark zone. If your FWC is ~2230e- @ ISO 1600, then a 24 minute sub exposed in the light polluted zone would be 1800e-. That is an 80% histogram! Such a sub would be MASSIVELY overexposed. Therefor, in order to get a 24 minute exposure that is not exposed beyond 1/4-1/3rd histogram, you have to find darker skies (or use a filter). The only reason I've REFERENCED 1-minute subs is because that was what you used in your Orion image...9 one-minute subs. It's just a reference point, and one I find quite appropriate because you stated that with those 60-second subs your histogram was at 1/3rd. Everything else I've explained is derived from that reference point...for consistency sake. Don't get so hung up on that, because it's making you completely miss the point. I've tried making this point with you before, and you missed it back then as well.

So, for the sake of other astrophotographers...

Under darker skies (or with a filter), the background sky photon flux IS REDUCED. There is LESS light pollution. Roger has not accounted for that. You might have 60 photons per minute in the city...however at a dark site, you might have only 6 photons per minute...or just one photon per minute...or even less than a photon per minute if you get over a hundred miles or more from any nearby cities (not really necessary most of the time...getting 30 miles out will usually do). Light pollution is not the same in every location, and that is what Roger is missing. This is something that can be easily measured, then correlated back to photon flux levels. I use an SQM meter to measure my skies. There are bortle scale charts that map zones and magnitude measurements to artificial light/object light levels that can be used to determine how much of the light your exposing is useful object signal vs. unwanted light pollution and airglow signal, for those who are interested in figuring out how good their skies are, and how much better darker skies might be.

A 24 minute exposure in the city where there is a lot of light pollution would, if your 60ph/m rate is correct, very likely have 1440e- signal from light pollution itself. However a 24 minute exposure out of the city...away from the light pollution...is going to have SIGNIFICANTLY LESS signal from light pollution itself. Your object signal, the signal derived from photons from space, is going to be the same regardless. THIS is what Roger is missing. He is only accounting for the difference in exposure time...and ignoring the fact that you cannot get a 1/4-1/3rd histogram exposure with a 24 minute exposure in the city (too much additional light reflected from ground sources off the atmosphere back into your scope, which results in an over-exposed sub if you actually tried.) You could only get a 24 minute 1/4-1/3rd histogram exposure out where the skies are much darker (or...added a light pollution filter. ;P)

Lets just say that a good dark site is 10x darker than wherever it was Roger acquired the subs for his 7D II Orion image. Instead of 60ph/m skyfog flux, you have 6ph/m skyfog flux, for a total of 144ph skyfog signal. Now run the SNR calculation with that:

S/N = 240 / sqrt(144 + 240 + 2.4^2 + 120) = 10.62:1

That is an SNR 94% larger than the 5.45:1 he gets with 24 1-minute subs you used with 60ph/m light pollution. If we move into a gray zone, 22mag/sq", we might have less than 1ph/m skyfog flux, for a total of (at most) 24ph skyfog signal:

S/N = 240 / sqrt(24 + 240 + 2.4^2 + 120) = 12.62:1

Now our SNR is 132% larger than his original 24x1m subs with higher light pollution.

This really does matter. You cannot ignore the difference in sky brightness as you move away from a light pollution center as Roger has been doing. If we aim for 1/3rd histogram, we cannot get a 24 minute exposure in the city. The light pollution of the city is a key LIMITING FACTOR in how long we can expose for. To expose longer, we must remove some or all of that unwanted photon flux. You do that either with a filter (light pollution filter with DSLR, or possibly narrow band filter with DSLR)...or by driving out of the city to darker skies.

This is the same math that told me I should drive out to a dark site and do some imaging out there in December 2014. I listened to the math, hauled my gear out to a dark site 35 minutes east of my home, and the differences were MASSIVE. It's a significant part of the reason I was able to produce this image:

http://www.astrobin.com/full/142576/F/

I had tried through most of October, November and the earlier part of December in 2014 to create a similar image from my light polluted back yard with an Astronoik CLS filter and even longer subs (which at the time was measuring around 17.5mag/sq" due to icy inversion layers...the results were rather horrid, if I do say so myself.) There isn't any comparing the quality of an exposure, the quality of the object signal, between the average light polluted back yard (which is usually somewhere in a red zone, by the number of people I've discussed this very same issue with over the last 14 months) and a decent dark site.

It's not just about increasing exposure length in the same location where you have heavy light pollution...it's about finding skies that will support longer exposures, and making the most of them. The difference between 24x1m subs and 1x24m subs is not a mere 4%...because in order to actually get a 24m sub that still peaks at or less than 1/3rd histogram...you can't keep imaging from the same location...with the same old light pollution.

Both Jon and Roger make valid (but different) points.
However, good points can often go completely missed in long paragraphs of technical discussion.
I'm a mathematician but I admit I lost the will to read through all this

Mark

sharkmelley said:

Both Jon and Roger make valid (but different) points.
However, good points can often go completely missed in long paragraphs of technical discussion.
I'm a mathematician but I admit I lost the will to read through all this

Mark

Click to expand...

I guess to distill it down:

SNR = (ObjectFlux * QuantumEfficiency * ExposureTime * SubCount)/SQRT(SubCount * ((ObjectFlux * QuantumEfficiency * ExposureTime) + (SkyFogFlux * QuantumEfficiency * ExposureTime) + (DarkCurrentRate * ExposureTime) + ReadNoise^2))

From the above formula, SkyFogFlux is not a constant. It can be extremely high in the city, tens to hundreds of photons per minute, where lots of artificial light is reflected back to Earth off the atmosphere (light pollution). It can also be extremely low far from any nearby cities or towns, where the only unwanted photons come from airglow, and it may only be few photons per minute or less at an exceptional dark site (~22mag/sq").

In the city, you have a very large noise term from an unwanted signal, light pollution. Near the edges of the city you have a large noise term from an unwanted signal. Out of the city you have a small noise term from an unwanted signal. Very far from a city, you have a tiny noise term from an unwanted signal.

If you stick with the 1/3rd histogram guideline, assuming your using around an f/4 scope and ISO 800-1600, in the city you will reach a 1/3rd histogram exposure in very little time, maybe 15-30 seconds, and most of the signal is unwanted, from light pollution, and will need to be subtracted out. Near the edges of a city/suburban area, you will reach 1/3rd histogram exposure with longer exposures, but still relatively short in the grand scheme of things, say 60-120 seconds, and your signal will be more balanced...better object signal and only slightly more light pollution signal. Out of the city you will require much longer exposures to reach 1/3rd histogram, anywhere from 2-8 minutes or so, and the majority of your signal will be object signal, or at worst object and skyfog signals will be about the same. Very far from any cities, and you may find it starts to get difficult to reach 1/3rd histogram even with 10 minute exposures, and most of your signal should be from space, very little should be from any artificial source or airglow.

The farther from the city you get, each 1/3rd histogram exposure gets richer, deeper, with less total noise (from any and all sources). The closer to the city you get, each 1/3rd histogram exposure gets weaker, shallower, with more total noise (from any and all sources).

Could you and roger please stop arguing over this, you both have slightly different ideas based on the same principles, and its both getting you really good images. thanks for your imput guys!

Jon Rista said:

rnclark said:

Jon Rista said:

As for read noise itself. If you stuck with the 1 minute exposures at the dark site, you would need to stack more than 24 subs to get the same SNR as with a single 24 minute sub. I know you know that, Roger, as you noted it in your comment above...but for other readers here. You would need around 42 subs, because of the compounding of read noise. SNR would be (10 * 42)/SQRT(42* (10 + 1 + 3^9)) = 14.5:1. A single 24 minute sub or 42 one minute subs. The choice there should be pretty easy. Personally, if your willing to image for 42 minutes, you might as well go for two 24 minute subs and your SNR will be up to 20.5:1.

Click to expand...

Jon, this really is getting tiring and I'm going to bow out as I really don't have the time to keep making corrections after this one, but for the sake of other astrophotographers, I'll try one more time.

Here is what you forgot. You keep harping on 1-minute subs. If you actually read what I write, I say expose to get 1/4 to 1/3 histogram peak on the sky. So my 7D2 that is around 65 photons / exposure. On a camera with larger pixels it would be more electrons. You conveniently forgot to include that. So here is what you would actually get with 24 minutes total exposure:

Say 1-minute exposure gave sky of 60 photons, subject 10 photons, read noise 2.4 electrons, dark current 5 electrons (0.083 e/sec).

In 24 1-minute subs S/N = 24 * 10 / sqrt(24* (60 + 10 + 2.4^2 + 5)) = 5.45

Now assuming no clipping or dynamic range issues, a single 24 minute exposure would have 1440 sky electrons, 240 subject electrons, read noise 2.4 electrons and 120 dark current electrons. Thus:

S/N = 240 / sqrt(1440 + 240 + 2.4^2 + 120) = 5.65, thus less than 4% difference. Again, you would not see any difference in the image.

How hard is this? The key is at 1/4 to 1/3 histogram, sky is much greater than read noise because 60 is much greater than 2.4 squared. Once you reach that level, red noise and thus sub exposure length is irrelevant.

And as read noise gets lower, you can do even shorter subs. For example, raise ISO, do 30 second subs. On the 7d2, raise ISO to 3200 and get 1.9 electron read noise, and we have the same dynamic range in 30 seconds as 60 second at iso 1600. We now do 42 subs to get to 24 minutes total integration time. We now get:

S/N = 42 * 5 / sqrt(42* ( 30 + 5 + 1.9^2 + 2.5)) = 5.05, or only 12% worse than a single long exposure. You would be hard pressed to tell the difference.

Go another factor of two shorter, raise ISO, then you would get:

S/N = 84 * 2.5 / sqrt(84* ( 15 + 2.5 + 1.7^2 + 1.25)) = 4.92, so only 15% worse than a single exposure. Again, it would be difficult to tell.

And of course in practice, people generally do not do 24-minute subs, so the real-world difference would be less. With 8-minute subs, you get S/N =80 * 3 /sqrt(3*(480 + 80 + 2.4^2 + 40)) = 5.63

So we now have a simple table

sub exposure___S/N for the above parameters

24 min_______5.65

_8 min_______5.63

_1 min_______5.45

0.5 min______5.05

0.25 min_____4.92

Jon Rista said:

This is really only viable if you are either A) using fast camera lenses, or B) using specialized ultra fast telescopes like the Celestron with Hyperstar, where you can get around f/2 optics.

Click to expand...

Again, you are not getting it. Nowhere do I say you should do minute exposures. I say exposure length to get to 1/4 to 1/3 histogram, whatever time that takes. If you follow this then read noise with modern cameras will have little significant impact. If you do less exposure than 1/4 histogram, read noise becomes a larger contributor, but it has small impacts until sub exposure times are very short.

Roger

Click to expand...

Wow, so arrogant, when you are entirely missing the point. For the sake of other astrophotographers, one last time. It's not specifically about 1-minute exposures. It's not about read noise (and you ironically quoted the one thing in my rather lengthy previous post where I said anything about read noise. *rolleyes*) It's about light pollution, how light pollution adds noise, and how light pollution is NOT a constant flux everywhere. DARK SKIES vs. LIGHT POLLUTED SKIES. It's about how higher photon flux from light polluted skies implicitly limit your maximum exposure time (if your trying to avoid exposing the skyfog peak in the histogram beyond 1/3rd.)

It's not the 1-minute subs that I'm "harping" on. It's the difference between a 1/3rd histogram exposure in a light polluted zone, and a 1/3rd histogram exposure in a dark zone. If your FWC is ~2230e- @ ISO 1600, then a 24 minute sub exposed in the light polluted zone would be 1800e-. That is an 80% histogram! Such a sub would be MASSIVELY overexposed. Therefor, in order to get a 24 minute exposure that is not exposed beyond 1/4-1/3rd histogram, you have to find darker skies (or use a filter). The only reason I've REFERENCED 1-minute subs is because that was what you used in your Orion image...9 one-minute subs. It's just a reference point, and one I find quite appropriate because you stated that with those 60-second subs your histogram was at 1/3rd. Everything else I've explained is derived from that reference point...for consistency sake. Don't get so hung up on that, because it's making you completely miss the point. I've tried making this point with you before, and you missed it back then as well.

So, for the sake of other astrophotographers...

Under darker skies (or with a filter), the background sky photon flux IS REDUCED. There is LESS light pollution. Roger has not accounted for that. You might have 60 photons per minute in the city...however at a dark site, you might have only 6 photons per minute...or just one photon per minute...or even less than a photon per minute if you get over a hundred miles or more from any nearby cities (not really necessary most of the time...getting 30 miles out will usually do). Light pollution is not the same in every location, and that is what Roger is missing. This is something that can be easily measured, then correlated back to photon flux levels. I use an SQM meter to measure my skies. There are bortle scale charts that map zones and magnitude measurements to artificial light/object light levels that can be used to determine how much of the light your exposing is useful object signal vs. unwanted light pollution and airglow signal, for those who are interested in figuring out how good their skies are, and how much better darker skies might be.

A 24 minute exposure in the city where there is a lot of light pollution would, if your 60ph/m rate is correct, very likely have 1440e- signal from light pollution itself. However a 24 minute exposure out of the city...away from the light pollution...is going to have SIGNIFICANTLY LESS signal from light pollution itself. Your object signal, the signal derived from photons from space, is going to be the same regardless. THIS is what Roger is missing. He is only accounting for the difference in exposure time...and ignoring the fact that you cannot get a 1/4-1/3rd histogram exposure with a 24 minute exposure in the city (too much additional light reflected from ground sources off the atmosphere back into your scope, which results in an over-exposed sub if you actually tried.) You could only get a 24 minute 1/4-1/3rd histogram exposure out where the skies are much darker (or...added a light pollution filter. ;P)

Lets just say that a good dark site is 10x darker than wherever it was Roger acquired the subs for his 7D II Orion image. Instead of 60ph/m skyfog flux, you have 6ph/m skyfog flux, for a total of 144ph skyfog signal. Now run the SNR calculation with that:

S/N = 240 / sqrt(144 + 240 + 2.4^2 + 120) = 10.62:1

That is an SNR 94% larger than the 5.45:1 he gets with 24 1-minute subs you used with 60ph/m light pollution. If we move into a gray zone, 22mag/sq", we might have less than 1ph/m skyfog flux, for a total of (at most) 24ph skyfog signal:

S/N = 240 / sqrt(24 + 240 + 2.4^2 + 120) = 12.62:1

Now our SNR is 132% larger than his original 24x1m subs with higher light pollution.

This really does matter. You cannot ignore the difference in sky brightness as you move away from a light pollution center as Roger has been doing. If we aim for 1/3rd histogram, we cannot get a 24 minute exposure in the city. The light pollution of the city is a key LIMITING FACTOR in how long we can expose for. To expose longer, we must remove some or all of that unwanted photon flux. You do that either with a filter (light pollution filter with DSLR, or possibly narrow band filter with DSLR)...or by driving out of the city to darker skies.

This is the same math that told me I should drive out to a dark site and do some imaging out there in December 2014. I listened to the math, hauled my gear out to a dark site 35 minutes east of my home, and the differences were MASSIVE. It's a significant part of the reason I was able to produce this image:

http://www.astrobin.com/full/142576/F/

I had tried through most of October, November and the earlier part of December in 2014 to create a similar image from my light polluted back yard with an Astronoik CLS filter and even longer subs (which at the time was measuring around 17.5mag/sq" due to icy inversion layers...the results were rather horrid, if I do say so myself.) There isn't any comparing the quality of an exposure, the quality of the object signal, between the average light polluted back yard (which is usually somewhere in a red zone, by the number of people I've discussed this very same issue with over the last 14 months) and a decent dark site.

It's not just about increasing exposure length in the same location where you have heavy light pollution...it's about finding skies that will support longer exposures, and making the most of them. The difference between 24x1m subs and 1x24m subs is not a mere 4%...because in order to actually get a 24m sub that still peaks at or less than 1/3rd histogram...you can't keep imaging from the same location...with the same old light pollution.

Click to expand...

Roger, I only WISH I could expose for 30 sec. , let alone 1 min. from my backyard and keep it below 1/3 BOC histo, shooting a Sigma 85mm f1.4, @ f4 and only ISO 800 !!! I could do it by shooting at Unity Gain, ~ISO 400 but you've already written enough volumes about how "Unity Gain" can't capture the faintest signals, so I'm sunk there.

But with my LP there's no signal below that level to capture with 30 sec subs !!! Nine minutes of White Noise (18 X 30") will never capture any more signal below that floor, that is, unless I go to the extremes of trying to stack ~ 85,000/ 30" subs to replicate just a few short hours at a dark site to capture that rare signal photon that even makes it into my 18.0 mag arc sec ^ 2 backyard.

Jon is completely correct about the effects of LP ! 99% of the users here don't have the funds to have their own satellite orbiting Saturn as their main "work" scope, or have the funds to jet off to whatever dark site looks fun, while using the latest/greatest f2.8 "CANON" lens on the latest FF body to get 30" shots.

The OP took his photo with a kit zoom, from his local conditions, with far too short an exposure in any of his subs. Suggesting to him, Jon, and everyone else, that 30 sec subs at f2.8 is what all of "real" astrophotographers do, (beats chest) is braggadocios at best , but is more so, fundamentally detached from the reality of those 99.999% of astrophotographers that are using f 5.6-f8 kit zooms, f7 APO's, f8 RC's, or f10 SCT's. Like the political debates like to point out, you live in the 1% world.

Roger, if you could, please take a 9 min photo of the Orion Nebula, shot using any Canon Rebel body and the EF-S 18-200mm f/3.5-5.6 IS lens, taken from Cheesman Park, and post what you get. Then, from the same site, post your best photo of Abell 2151, using your longest f2.8 lens.

''

'

,

swimswithtrout said:

Roger, I only WISH I could expose for 30 sec. , let alone 1 min. from my backyard and keep it below 1/3 BOC histo, shooting a Sigma 85mm f1.4, @ f4 and only ISO 800 !!! I could do it by shooting at Unity Gain, ~ISO 400 but you've already written enough volumes about how "Unity Gain" can't capture the faintest signals, so I'm sunk there.

But with my LP there's no signal below that level to capture with 30 sec subs !!! Nine minutes of White Noise (18 X 30") will never capture any more signal below that floor, that is, unless I go to the extremes of trying to stack ~ 85,000/ 30" subs to replicate just a few short hours at a dark site to capture that rare signal photon that even makes it into my 18.0 mag arc sec ^ 2 backyard.

Jon is completely correct about the effects of LP ! 99% of the users here don't have the funds to have their own satellite orbiting Saturn as their main "work" scope, or have the funds to jet off to whatever dark site looks fun, while using the latest/greatest f2.8 "CANON" lens on the latest FF body to get 30" shots.

The OP took his photo with a kit zoom, from his local conditions, with far too short an exposure in any of his subs. Suggesting to him, Jon, and everyone else, that 30 sec subs at f2.8 is what all of "real" astrophotographers do, (beats chest) is braggadocios at best , but is more so, fundamentally detached from the reality of those 99.999% of astrophotographers that are using f 5.6-f8 kit zooms, f7 APO's, f8 RC's, or f10 SCT's. Like the political debates like to point out, you live in the 1% world.

Roger, if you could, please take a 9 min photo of the Orion Nebula, shot using any Canon Rebel body and the EF-S 18-200mm f/3.5-5.6 IS lens, taken from Cheesman Park, and post what you get. Then, from the same site, post your best photo of Abell 2151, using your longest f2.8 lens.

Click to expand...

First, you seem to be misunderstanding exposure and collecting light from the subject. Increased light pollution not only erases the fast optical system advantage, but works against it and favors slower f/ratios. Don't you image with an 8-inch f/4? That is a huge advantage over a 107 mm f/2.8 at a dark site but even more so in a light polluted site.

Because of the massive confusion here, I have added a lot of information on exposure, sub-exposure and light pollution effects here:


Note, I describe how one can win against light pollution's short exposure times by increasing focal lengths with the same aperture (e.g. add a teleconverter) and binning. It doesn't solve all the issues, but mitigates the problem in many cases.

Roger

rnclark said:

swimswithtrout said:

Roger, I only WISH I could expose for 30 sec. , let alone 1 min. from my backyard and keep it below 1/3 BOC histo, shooting a Sigma 85mm f1.4, @ f4 and only ISO 800 !!! I could do it by shooting at Unity Gain, ~ISO 400 but you've already written enough volumes about how "Unity Gain" can't capture the faintest signals, so I'm sunk there.

But with my LP there's no signal below that level to capture with 30 sec subs !!! Nine minutes of White Noise (18 X 30") will never capture any more signal below that floor, that is, unless I go to the extremes of trying to stack ~ 85,000/ 30" subs to replicate just a few short hours at a dark site to capture that rare signal photon that even makes it into my 18.0 mag arc sec ^ 2 backyard.

Jon is completely correct about the effects of LP ! 99% of the users here don't have the funds to have their own satellite orbiting Saturn as their main "work" scope, or have the funds to jet off to whatever dark site looks fun, while using the latest/greatest f2.8 "CANON" lens on the latest FF body to get 30" shots.

The OP took his photo with a kit zoom, from his local conditions, with far too short an exposure in any of his subs. Suggesting to him, Jon, and everyone else, that 30 sec subs at f2.8 is what all of "real" astrophotographers do, (beats chest) is braggadocios at best , but is more so, fundamentally detached from the reality of those 99.999% of astrophotographers that are using f 5.6-f8 kit zooms, f7 APO's, f8 RC's, or f10 SCT's. Like the political debates like to point out, you live in the 1% world.

Roger, if you could, please take a 9 min photo of the Orion Nebula, shot using any Canon Rebel body and the EF-S 18-200mm f/3.5-5.6 IS lens, taken from Cheesman Park, and post what you get. Then, from the same site, post your best photo of Abell 2151, using your longest f2.8 lens.

Click to expand...

First, you seem to be misunderstanding exposure and collecting light from the subject. Increased light pollution not only erases the fast optical system advantage, but works against it and favors slower f/ratios. Don't you image with an 8-inch f/4? That is a huge advantage over a 107 mm f/2.8 at a dark site but even more so in a light polluted site.

Because of the massive confusion here, I have added a lot of information on exposure, sub-exposure and light pollution effects here:

http://www.clarkvision.com/articles/astrophotography.and.exposure/

Note, I describe how one can win against light pollution's short exposure times by increasing focal lengths with the same aperture (e.g. add a teleconverter) and binning. It doesn't solve all the issues, but mitigates the problem in many cases.

Click to expand...

Now that you are actually addressing the question of noise from light pollution, which is what I've been talking about all along.

One can sort of "win" against light pollution...however, why fight the battle at all? That is the question I have. You can overcome light pollution by integrating to an excessive degree, or by using a massive aperture...but why?

I do know some imagers who use 14" Celestron EdgeHD SCT's with Hyperstar to overcome light pollution. They are certainly able to acquire deep exposures on background details, but it is also usually at the cost of fairly extreme clipping of brighter stars. As much as that giant aperture gathers more light for faint background details, it also gathers more light for stars. Stars already have a significantly higher flux, and when you use such a large aperture, maintaining a balance between well-exposed stars and decently exposed background/DSO becomes more difficult.

Even with my 600mm f/4 lens, I have trouble avoiding clipped stars at my image scale of around 2.14"/px, because stars saturate so fast relative to the fainter nebula and background details. One way or another, light pollution is a problem. One which we FIGHT against...in my opinion, needlessly.

There is a much easier solution to that battle: Avoid it altogether. Eliminate light pollution as a problem at all, then you don't have to worry about overcoming it in the first place. Then you are no longer restricted to using shorter subs. Then your capable of getting deeper exposure and greater contrast and higher SNR in each and every single sub, and when you integrate many of them, the results are significantly better than the integrations you could create with that large scope while overcoming light pollution (especially if you only "overcome" with less than an hour of total integration time.)

Now, your article is mostly based on the results from a night of high airglow, but in a green/blue zone. While there is certainly some light pollution there, that is nothing remotely close to the kind of light pollution you get from an average suburban back yard. Your article makes the claim that you can relatively easily overcome light pollution, but the LP levels you have described are still relatively low in the grand scheme of things. In the average RED ZONE back yard, which is much more common for your average suburban home, they can be ten times higher or more. Your SNR calculations are fine for a green zone, but you have not given anyone an adequate understanding of how bad the noise from LP in a red zone is. It isn't 7.7e- noise from LP vs. 3e- read noise...it's more like 30e- noise from LP vs. 3e- read noise. I think that is an important distinction, a critically important one. With 30e- noise from skyfog, you should be integrating significantly more subs. You shouldn't be exposing longer per sub in heavy LP...however you should be gathering significantly more exposure time in total (total integration time) to effectively overcome all the extra noise added by light pollution. Your article does not address that...and I think it really should. Because that's what I've been trying to address all along (you seem to have mistakenly assumed that I've been harping on read noise...I have not; my primary concern is the noise from light pollution in the AVERAGE suburban resident's back yard...which is usually a bortle red zone, with a growing number falling into white zones, and a shrinking number falling into orange zones, as light pollution continues to grow at a sadly terrifying rate.)

-----

To demonstrate why it is light pollution that I rail against, these two unfiltered single exposure images are identical exposures (with the exception that dark site image was at a higher ISO, which only changes read noise, which you have made eminently clear is something one need hardly worry about). One is from my 18.8mag/sq" red zone back yard, and one from my 21.3mag/sq" dark site (with an inversion layer in place, which hurt transparency a bit):

[IMG width="400px" alt="18.8mag/sq" red zone left; 21.3mag/sq" green zone right"] 18.8mag/sq" red zone left; 21.3mag/sq" green zone right

The back yard sub was just a little shy of 1/3rd histogram, the dark site sub was significantly less exposed (less than 1/6th when normalized with the red zone image, and could have withstood significantly more exposure if I'd wanted to get more detail on the nebula in the background). These two images have been identically "screen stretched" with PixInsight to demonstrate the impact of light pollution. The impact should be beyond obvious...and it is quite severe. For total clarity here...this is a RED ZONE vs. a GREEN ZONE, Bortle class.

Here are the same two images, however in this case, I let PixInsight automatically attempt to maximize the quality of the light polluted image (which effectively offsets the additional signal from light pollution, without offsetting any actual nebula data):

[IMG width="400px" alt="18.8mag/sq" red zone left; 21.3mag/sq" green zone right"] 18.8mag/sq" red zone left; 21.3mag/sq" green zone right

Despite the offsetting (subtracting out) of the light pollution, the dark site sub is significantly better than the one from my back yard. I know imagers who would LOVE to have just that one single dark site sub as an image, because even after integrating 30, 45, maybe even 60 minutes of 30-second subs from their red or white zone, their images still don't look as good! The dark site sub could have been exposed much more deeply as well, greatly improving the contrast of the data in each and every sub. (The only reason I did not expose longer is I'd somehow forgotten an extension tube for my guidescope, and was unable to guide! )

-----

Also, one point that I think might be important, since you recommend people use the lightpollutionmap.info site. If you wish to avoid confusion and discrepancies: The color scale on that site is based on direct radiance levels from satellite measurements. A green zone on the lightpollutionmap site is NOT the same as a green zone on a standard Bortle scale map. My own dark site is in the blue/gray areas of lightpollutionmap.info...however based in recent SQM-L measurements (20.8-21.2mag/sq", beginning of the night at around 6:30pm to the end of my imaging session around 2am), it is right on the border of a green/yellow bortle zone. (I attribute the increased brightness vs. a year ago, when I was measuring 21.5mag/sq", to the rather rapid growth and expansion of Denver population and the consequential increase in LP that has had around the Denver area. :''( ) The Bortle scale is not based on any actual measurements of LP, either from the ground or from space. The Bortle scale is a visually based scale that is roughly guestimated based on the proximity of light pollution bubbles, and the visible appearance of various stellar objects:

https://en.wikipedia.org/wiki/Bortle_scale

" John E. Bortle created the scale and published it in the February 2001 edition of Sky & Telescope magazine to help amateur astronomers evaluate the darkness of an observing site, and secondarily, to compare the darkness of observing sites."

Personally, I much prefer the way lightpollutionmap.info works to a standard bortle scale map. It's a far better gauge for finding decent imaging dark sites (in my experience, anything cyan or darker offers skies orders of magnitude better for imaging than the average red or orange bortle zone back yard). However I think it is confusing to recommend that map moments after explaining the Bortle Scale, as the color grading is the same, but the colors actually mean a different thing.

-----

If I could make one recommendation, while your battling the evil armies of the "internet experts": Try not to assume everyone has, is capable of, nor is interested in buying one of these fancy new "low noise" DSLRs. For one, as you are a Canon guy, the only camera like that (both low read noise AND very low dark current) available from Canon at the moment is the 7D II. The dark current is still relatively high (outside of the dead of winter) on most other Canon DSLRs (I think it was you who measured that the 6D has over 3e-/s at ~25C, and still has ~0.3e-/s at ~10C? Over even a five minute exposure @ 10C, that is 9.5e- dark current noise, more than read noise, more even than skyfog at a dark site...and during the summer it is quite common to have sensor temps reach over 30C, where dark current would be at least 6e-/s, resulting in a whopping 42e- dark current noise.) I do not yet know what the 5Ds dark current is like, although I would hope it's about the same as the 7D II. That is two Canon DSLRs that have (or might have) truly low dark current...neither of them cheap.

I talk with a lot of astrophotography beginners, and if there is ever a constant among them, it's that none of them want to spend money. A $1500 camera is usually totally out of the question, let alone a $2000 6D, let alone a CCD camera. Spending $1500 in total is usually out of the question, although every so often a beginner comes along willing to spend $1500-$2000 in total on a full set of gear, including a mount, software, maybe a laptop for computer control, etc. Most are looking for a very cheap DSLR, and most bring up $175-$400 used DSLRs like older Canon Rebels or possibly a Nikon D3300...possibly already astro modded.

Having ultra low read noise and ultra low dark current is NOT a common thing among I would say a majority of astrophotographers. Even among those who use CCD, it's 5-7e- most of the time for those using a KAF-8300 (however because they can cool to -25C or even more, they effectively don't have to worry about dark current noise, so noise levels are still usually lower, or at least balanced with, even most newer DSLRs.) If your articles are intended to help the entire astrophotography community, always assuming that everyone has a 7D II and uses it for astrophotography is probably not going to help as many as you otherwise might.

sharkmelley said:

Jon Rista said:

sharkmelley said:

rnclark wrote: a single 24 minute exposure would have 1440 sky electrons, 240 subject electrons, read noise 2.4 electrons and 120 dark current electrons. Thus:

S/N = 240 / sqrt(1440 + 240 + 2.4^2 + 120) = 5.65...

With 8-minute subs, you get S/N =80 * 3 /sqrt(3*(480 + 80 + 2.4^2 + 40)) = 5.63

Click to expand...

If we move into a gray zone, 22mag/sq", we might have less than 1ph/m skyfog flux, for a total of (at most) 24ph skyfog signal:

S/N = 240 / sqrt(24 + 240 + 2.4^2 + 120) = 12.62:1

Click to expand...

Both Jon and Roger make valid (but different) points.
However, good points can often go completely missed in long paragraphs of technical discussion.
I'm a mathematician but I admit I lost the will to read through all this

Click to expand...

Hi Mark,

I must say that this has been a most informative exchange. Simplifying for clarity, if I understand correctly for a given scene, location, camera setup and total integration time there is in this context just one and only one variable in the SNR equation that is affected when individual exposure times are varied: read noise variance, which gets multiplied by the number of frames taken (n). Everything else stays exactly the same (well, you know).

So for a total integration time of 24 minutes with the metrics from rnclark's post his SNR formula is heavily dominated by sky glow/pollution/fog noise worth 1440e- and simplifies to

S/N = 240 / sqrt(1440 + 240 + n*2.4^2 + 120),

n representing the number of frames chosen to achieve a total integration time of 24*60s. If on the other hand we look at Jon's situation, SNR is no longer dominated by sky noise of only 24e- and read noise becomes more relevant as the number of frames is piled on:

S/N = 240 / sqrt(24 + 240 + n*2.4^2 + 120).


Camera, subject and location metrics as described in the posts above

Clearly depending on equipment, location and subject matter the number of frames chosen to achieve a set integration time may matter more or less. Thank you Jon and Roger for a most informative exchange.

Jack

Jack Hogan said:

sharkmelley said:

Jon Rista said:

Jack Hogan said:

rnclark wrote: a single 24 minute exposure would have 1440 sky electrons, 240 subject electrons, read noise 2.4 electrons and 120 dark current electrons. Thus:

S/N = 240 / sqrt(1440 + 240 + 2.4^2 + 120) = 5.65...

With 8-minute subs, you get S/N =80 * 3 /sqrt(3*(480 + 80 + 2.4^2 + 40)) = 5.63

Click to expand...

If we move into a gray zone, 22mag/sq", we might have less than 1ph/m skyfog flux, for a total of (at most) 24ph skyfog signal:

S/N = 240 / sqrt(24 + 240 + 2.4^2 + 120) = 12.62:1

Click to expand...

Both Jon and Roger make valid (but different) points.
However, good points can often go completely missed in long paragraphs of technical discussion.
I'm a mathematician but I admit I lost the will to read through all this

Click to expand...

Hi Mark,

I must say that this has been a most informative exchange. Simplifying for clarity, if I understand correctly for a given scene, location, camera setup and total integration time there is in this context just one and only one variable in the SNR equation that is affected when individual exposure times are varied: read noise variance, which gets multiplied by the number of frames taken (n). Everything else stays exactly the same (well, you know).

So for a total integration time of 24 minutes with the metrics from rnclark's post his SNR formula is heavily dominated by sky glow/pollution/fog noise worth 1440e- and simplifies to

S/N = 240 / sqrt(1440 + 240 + n*2.4^2 + 120),

n representing the number of frames chosen to achieve a total integration time of 24*60s. If on the other hand we look at Jon's situation, SNR is no longer dominated by sky noise of only 24e- and read noise becomes more relevant as the number of frames is piled on:

S/N = 240 / sqrt(24 + 240 + n*2.4^2 + 120).


Camera, subject and location metrics as described in the posts above

Clearly depending on equipment, location and subject matter the number of frames chosen to achieve a set integration time may matter more or less. Thank you Jon and Roger for a most informative exchange.

Jack

Click to expand...

Thanks for the chart, Jack. I decided to get some actual measurements from two sets of subs of the Pleiades. I am not exactly certain of the sky darkness for each...however the backyard subs were around 18.8-18.9mag/sq" and the dark site was 21.2-21.3mag/sq". The dark site had poor transparency.

I measured the dark site photon level for the background sky at 347γ, and the backyard photon level for the background sky at 1760γ. In terms of electrons, it's about 173.5e- and 880e-. The region around Pleiades is packed with dust, and there are gradients in the unprocessed subs...so I don't know that I found the "true" deepest background sky level, but I think this is good enough for a basic comparison. This is for roughly identical exposures (ISO changed, but actual EV did not, and the way PI loads the data, ISO doesn't matter anyway.) So the only real difference between the two subs is read noise. Backyard data was ISO 400, dark site was ISO 1600, Canon 5D III.

I measure the dark site signal at around 120γ, and the backyard signal at around 100γ. In terms of electrons that is about 60e- and 50e- (assuming ~50% Q.E. for the 5D III...I'm going off of sensorgen.info...honestly don't know how accurate that is). My dark current at these temperatures is around 0.02e-/s/px. Exposures were 135s.

SNRdark = 60/SQRT(60+ 173.5 + 2.7 + 3.6^2) = 60/SQRT(249.16) = 60/15.8 = 3.8:1

SNRcity = 50/SQRT(50 + 880 + 2.7 + 9.8^2) = 50/SQRT(1028.74) = 50/33.07 = 1.56:1

Even if I had been using ISO 1600:

SNRcity = 50/SQRT(50 + 880 + 2.7 + 3.6^2) = 50/SQRT(945.66) = 50/30.75 = 1.62:1

Using a higher ISO for lower read noise in the city changed the SNR by 3.9% (1.0385x stronger signal) which is actually in line with Roger's claims on his site), whereas moving to the dark site (even with poor transparency) resulted in a change of 144% (2.44x stronger signal).



Now, if I hadn't forgotten my extension tube for my guide scope, I would have gone with double length subs at ISO 800 at the dark site. That would have changed the SNR to:

SNRdark = 120/SQRT(120 + 347 + 5.4 + 5.6^2) = 120/SQRT(503.76) = 120/22.45 = 5.35:1

And if I'd gone for 4x longer subs at ISO 400 (same ISO as in my back yard):

SNRdark = 240/SQRT(240+ 694 + 10.8 + 9.8^2) = 120/SQRT(1040.84) = 240/32.26 = 7.44:1

Jack Hogan said:

sharkmelley said:

Jon Rista said:

Jack Hogan said:

rnclark wrote: a single 24 minute exposure would have 1440 sky electrons, 240 subject electrons, read noise 2.4 electrons and 120 dark current electrons. Thus:

S/N = 240 / sqrt(1440 + 240 + 2.4^2 + 120) = 5.65...

With 8-minute subs, you get S/N =80 * 3 /sqrt(3*(480 + 80 + 2.4^2 + 40)) = 5.63

Click to expand...

If we move into a gray zone, 22mag/sq", we might have less than 1ph/m skyfog flux, for a total of (at most) 24ph skyfog signal:

S/N = 240 / sqrt(24 + 240 + 2.4^2 + 120) = 12.62:1

Click to expand...

Both Jon and Roger make valid (but different) points.
However, good points can often go completely missed in long paragraphs of technical discussion.
I'm a mathematician but I admit I lost the will to read through all this

Click to expand...

Hi Mark,

I must say that this has been a most informative exchange. Simplifying for clarity, if I understand correctly for a given scene, location, camera setup and total integration time there is in this context just one and only one variable in the SNR equation that is affected when individual exposure times are varied: read noise variance, which gets multiplied by the number of frames taken (n). Everything else stays exactly the same (well, you know).

So for a total integration time of 24 minutes with the metrics from rnclark's post his SNR formula is heavily dominated by sky glow/pollution/fog noise worth 1440e- and simplifies to

S/N = 240 / sqrt(1440 + 240 + n*2.4^2 + 120),

n representing the number of frames chosen to achieve a total integration time of 24*60s. If on the other hand we look at Jon's situation, SNR is no longer dominated by sky noise of only 24e- and read noise becomes more relevant as the number of frames is piled on:

S/N = 240 / sqrt(24 + 240 + n*2.4^2 + 120).


Camera, subject and location metrics as described in the posts above

Clearly depending on equipment, location and subject matter the number of frames chosen to achieve a set integration time may matter more or less. Thank you Jon and Roger for a most informative exchange.

Jack

Click to expand...

But the second equation (Jon's) violates the basic rule in astrophotography: expose long enough so that the sky histogram peak is 1/4 to 1/3 from left to right on the camera LCD. So of course S/N shows a dependence on sub-exposure time. Even then, the difference between 1 and 8 minute sub frames is small (less than 15% difference in S/N).

The 2.4 read noise came from a 7D2 and iso 1600. If I were to image from a dark site at sub 1-minute exposure times, I would up the iso, e.g. at 0.5 minute go to iso 3200 so I maintain the dynamic range and drop the read nise to 1.9 electrons. At 0.25 minute, go to iso 6400 and read noise is 1.7 electrons.

Now drop read noise into the 1-electron range, which we see in some cameras, and the effect is even less.

Roger

Jon Rista said:

Now, your article is mostly based on the results from a night of high airglow, but in a green/blue zone.

Click to expand...

No Jon, if you actually read it, I say:

"The times are derived from observations made in the western continental US, Hawaii and Africa. The red zone data are from within a metro area of 2.7 million people, about 8 miles from the center. Green zones are 40 to 50 miles out, blue zones 60 to 80 miles out, and gray zones more than about 80 miles out from the population of 2.7 million."

I made multiple measurements from multiple sites and multiple dates.

Jon Rista said:

Your SNR calculations are fine for a green zone, but you have not given anyone an adequate understanding of how bad the noise from LP in a red zone is. It isn't 7.7e- noise from LP vs. 3e- read noise...it's more like 30e- noise from LP vs. 3e- read noise.

Click to expand...

If you actually read what is there you would see factors of 16 between red zone and dark site, and you missed another point: you expose to 1/4 to 1/3 histogram, so it should never be 7.7 electrons from LP, but more like 50. 7.7 electrons would be about 1/30th histogram level, which no experienced astrophotographer would image at.

Jon Rista said:

-

The impact should be beyond obvious...and it is quite severe. For total clarity here...this is a RED ZONE vs. a GREEN ZONE, Bortle class.

Click to expand...

[IMG width="400px" alt="18.8mag/sq" red zone left; 21.3mag/sq" green zone right"] 18.8mag/sq" red zone left; 21.3mag/sq" green zone right

Your example shows incomplete processing. One can do much better from a red zone. Here is an example with more complete processing. One can do pretty well from a red zone, relatively speaking, with good processing. This image is now Figure 7a at:


and 7b shows a full resolution crop.


red zone, green zone, very dark zone compare.

rnclark said:

Jon Rista said:

Now, your article is mostly based on the results from a night of high airglow, but in a green/blue zone.

Click to expand...

No Jon, if you actually read it, I say:

"The times are derived from observations made in the western continental US, Hawaii and Africa. The red zone data are from within a metro area of 2.7 million people, about 8 miles from the center. Green zones are 40 to 50 miles out, blue zones 60 to 80 miles out, and gray zones more than about 80 miles out from the population of 2.7 million."

I made multiple measurements from multiple sites and multiple dates.

rnclark said:

Your SNR calculations are fine for a green zone, but you have not given anyone an adequate understanding of how bad the noise from LP in a red zone is. It isn't 7.7e- noise from LP vs. 3e- read noise...it's more like 30e- noise from LP vs. 3e- read noise.

Click to expand...

If you actually read what is there you would see factors of 16 between red zone and dark site, and you missed another point: you expose to 1/4 to 1/3 histogram, so it should never be 7.7 electrons from LP, but more like 50. 7.7 electrons would be about 1/30th histogram level, which no experienced astrophotographer would image at.

Click to expand...

The 7.7e- is the noise from the signal, which you derived yourself in your article: SQRT(60e-) = 7.745e-. The noise from the light pollution signal in a red zone is SIGNIFICANTLY higher, because the signal in a red zone is SIGNIFICANTLY higher...I mean, at 16x it would be 960e-, with noise of...31e-.

Also, the point I am trying to make is, your actual SNR comparisons are only comparing different sub counts and exposure times at the same location...at a dark site. You have not done any direct comparisons of exposures/SNRs between a blue/green zone dark site and a red zone. It's the comparison of subs between a dark site and a red zone that I think are most valuable. Of course there is going to be a minimal change in SNR from the same location (although personally I'll take the 5.3% if I have the option (we are talking about long exposures here...so 5.3% factored across many subs amounts to many many minutes of time spent imaging.) My own testing demonstrates very significant differences, factor of two at the very least, if not much more, when moving out of the heavily light polluted city and moving to a dark site.

Yes, you can fight against light pollution...but doing so means taking on many additional challenges, and requires a significant increase in total integration times. With sub counts up in the hundreds, pre-processing times jump considerably as well. I just posted measurements from the Pleiades images below...given the same EV used for both images, the difference in the two subs was 144% in favor of the dark site image...with 2x or 4x the exposure time at lower ISOs (despite the increase in read noise) the difference jumps to over 300%. Given my own experiences, and given that those experiences only require a 30 minute drive out of town, I have no option but to recommend to astrophotographers who are interested in getting the most out of their imaging time to find a nearby imaging dark site and use it as often as they can. One single night at a dark site out in the middle of a green zone is worth many nights of imaging in a red zone. On one single winter night, you could easily come away with several 2-4 hours integrations of several targets, and every image would look incredible compared to the single image you might be able to compile after imaging for four to six solid nights in a red or white zone in the city.

rnclark said:

rnclark said:

-

The impact should be beyond obvious...and it is quite severe. For total clarity here...this is a RED ZONE vs. a GREEN ZONE, Bortle class.

Click to expand...

[IMG width="400px" alt="18.8mag/sq" red zone left; 21.3mag/sq" green zone right"]

https://imgur.com/muTqZaj

18.8mag/sq" red zone left; 21.3mag/sq" green zone right

Your example shows incomplete processing. One can do much better from a red zone. Here is an example with more complete processing. One can do pretty well from a red zone, relatively speaking, with good processing.

Click to expand...

There is no processing at all, for either image. These are single subs, not full integrated images. These are just "screen stretched", which is a feature of PixInsight that allows you to see the image data, but all measurements and processing is still done on the linear data. There is no processing outside of the stretch for either, as I wanted people to see what the out of camera subs looked like with a basic (but as ideal as possible) stretch to reveal the information within.

Even with removal of the gradients from both images and normalization of nebula brightness, the difference is still clear (click for larger size to see difference in noise levels):



For the record, it would take about five subs from the red zone to match the single sub from the dark site, at ISO 1600 where RN is 3.6e-.

rnclark said:

But the second equation (Jon's) violates the basic rule in astrophotography: expose long enough so that the sky histogram peak is 1/4 to 1/3 from left to right on the camera LCD. So of course S/N shows a dependence on sub-exposure time. Even then, the difference between 1 and 8 minute sub frames is small (less than 15% difference in S/N).

The 2.4 read noise came from a 7D2 and iso 1600. If I were to image from a dark site at sub 1-minute exposure times, I would up the iso, e.g. at 0.5 minute go to iso 3200 so I maintain the dynamic range and drop the read nise to 1.9 electrons. At 0.25 minute, go to iso 6400 and read noise is 1.7 electrons.

Click to expand...

The graph above applies to the situation discussed in the previous posts, without taking into consideration changing read noise/ISO. Here is one that does, assuming the same maximum integration time of 24 minutes. Each curve describes expected SNR performance at the given sky glow/pollution/fog level assuming for convenience that the 24min integration time is contained within one frame at ISO 100, two frames of 12 min each at ISO200, 16 frames of 90s at ISO1600 etc.

Read noise, gain and clipping from Roger's site, prnu not taken into consideration, other metrics from posts above.


7DII SNR at various sky fog levels and a constant total integration time of 24 minutes

Looks like all the SNR curves have a definite peak independently of sky fog level, say 8 frames at ISO 800 for 3 minutes each. Or even 4 frames of 6 minutes each at ISO400. Who would've thunk, eh Roger?

Since I was at it I plugged some Nikon D7200 figures into the model because its pixels are roughly of the same size as the 7DII's (I did not correct for the difference and I don't know its dark current so I left it equal to the 7DII's, does this sound right?). This is what came out:


D7200 SNR at various sky fog levels and a constant total integration time of 24 minutes

Makes sense, the longer the better for D7200 frames given their almost ISO invariant performance, so a single frame of 24 minutes at ISO 100, or two frames of 12 minutes each at ISO 200 to keep FPN at bay would do it for this situation according to this model.

And here is also data for the Sony a7S, definitely not an ISOless camera, with the same per pixel assumptions (also left dark current equal to the 7DII's in the model, don't know if that's right)*:


a7S SNR at various sky fog levels and a constant total integration time of 24 minutes

Thanks to Aptina's contribution the model here suggests that there is a sweet spot around ISO3200, 32 frames by 45 seconds each for this total integration time and assumptions. ISO's higher than that do not seem to improve IQ while killing DR. a7S figures of merit come from bill claff's site .

Of course I know that this is just a model and we are not dealing with the nitty gritty of the real world like FPN and prnu. Still, interesting food for thought when you are out there in the cold. Assuming I did not make any mistakes (always a possibility when I am involved the SNR peak of each curve is clearly camera/situation dependent. The location of the peak seems to be dependent on ISO but apparently not, however, on the level of sky fog. So this begs the question: for your astro purposes, what's the sweetspot ISO of your camera?

Jack

*BTW, the a7S's graph is not directly comparable to the other two because of the different format and because the a7S's pixels are about four times the area of the others'. To make it comparable I think one would need to boost the signal and fog by 4x, think about angle of views and start making all sorts of other 'equivalence' corrections. On the other hand the smaller-pixel cameras would have the option of trading off some detail for SNR.

Jack Hogan said:

rnclark said:

But the second equation (Jon's) violates the basic rule in astrophotography: expose long enough so that the sky histogram peak is 1/4 to 1/3 from left to right on the camera LCD. So of course S/N shows a dependence on sub-exposure time. Even then, the difference between 1 and 8 minute sub frames is small (less than 15% difference in S/N).

The 2.4 read noise came from a 7D2 and iso 1600. If I were to image from a dark site at sub 1-minute exposure times, I would up the iso, e.g. at 0.5 minute go to iso 3200 so I maintain the dynamic range and drop the read nise to 1.9 electrons. At 0.25 minute, go to iso 6400 and read noise is 1.7 electrons.

Click to expand...

The graph above applies to the situation discussed in the previous posts, without taking into consideration changing read noise/ISO. Here is one that does, assuming the same maximum integration time of 24 minutes. Each curve describes expected SNR performance at the given sky glow/pollution/fog level assuming for convenience that the 24min integration time is contained within one frame at ISO 100, two frames of 12 min each at ISO200, 16 frames of 90s at ISO1600 etc.

Read noise, gain and clipping from Roger's site, prnu not taken into consideration, other metrics from posts above.


7DII SNR at various sky fog levels and a constant total integration time of 24 minutes

Looks like all the SNR curves have a definite peak independently of sky fog level, say 8 frames at ISO 800 for 3 minutes each. Or even 4 frames of 6 minutes each at ISO400. Who would've thunk, eh Roger?

Since I was at it I plugged some Nikon D7200 figures into the model because its pixels are roughly of the same size as the 7DII's (I did not correct for the difference and I don't know its dark current so I left it equal to the 7DII's, does this sound right?). This is what came out:


D7200 SNR at various sky fog levels and a constant total integration time of 24 minutes

Makes sense, the longer the better for D7200 frames given their almost ISO invariant performance, so a single frame of 24 minutes at ISO 100, or two frames of 12 minutes each at ISO 200 to keep FPN at bay would do it for this situation according to this model.

And here is also data for the Sony a7S, definitely not an ISOless camera, with the same per pixel assumptions (also left dark current equal to the 7DII's in the model, don't know if that's right)*:


a7S SNR at various sky fog levels and a constant total integration time of 24 minutes

Thanks to Aptina's contribution the model here suggests that there is a sweet spot around ISO3200, 32 frames by 45 seconds each for this total integration time and assumptions. ISO's higher than that do not seem to improve IQ while killing DR. a7S figures of merit come from bill claff's site .

Of course I know that this is just a model and we are not dealing with the nitty gritty of the real world like FPN and prnu. Still, interesting food for thought when you are out there in the cold. Assuming I did not make any mistakes (always a possibility when I am involved the SNR peak of each curve is clearly camera/situation dependent. The location of the peak seems to be dependent on ISO but apparently not, however, on the level of sky fog. So this begs the question: for your astro purposes, what's the sweetspot ISO of your camera?

Jack

*BTW, the a7S's graph is not directly comparable to the other two because of the different format and because the a7S's pixels are about four times the area of the others'. To make it comparable I think one would need to boost the signal and fog by 4x, think about angle of views and start making all sorts of other 'equivalence' corrections. On the other hand the smaller-pixel cameras would have the option of trading off some detail for SNR.

Click to expand...

Hi Jack,

Very nice presentation. Of course there are significant factors that would change the curves at low iso in real imaging situations. 1) Fixed pattern noise. There is never 100% correction, even with bias and dark frames. When the problem is bad, one needs to resort to dithering and that favors more sub exposures. 2) As gain (iso) drops, intensity sampling noise increases, so even with no fixed pattern noise, all the curves will turn down when gain (ISO) drops (higher than 1/3 electron/DN sampling). In practice, for 14-bit cameras with 4 to 5 micron pixels, that means noise in the final image increases as ISO drops below about 400. The sampling effects become larger as ISO drops so digitization is > 1/2 e/DN, and is worse for lower read noise. (16-bit A/D will help this.)

So for faint signal detection and 14-bit cameras, ISOs should remain above about 400, and above 800 for canon until they improve the fixed pattern noise.

Roger

"Jon, this really is getting tiring and I'm going to bow out as I really don't have the time to keep making corrections after this one, but for the sake of other astrophotographers, I'll try one more time."

As my first post, I wanted to thank rnclarck for sticking it out, I totally get it.

I'm perplexed as to the straw-man arguments that are presented to him.

Even being a new guy to astrophotogpraphy, I can usually understand what Roger is saying, yet find the arguments against him confusing and full of words.

So far I am still doing fixed tripod stacking, pixinsight and PS trials expired...but when I get my barndoor up and running I will have a leg up from reading Roger's posts.

It is not a hard concept to grasp, aim for 1/3rd histogram and open that shutter as often as possible! Integration time!

This is very relevant for us new guys as Roger's technique will have less tracking errors. This is a pretty big deal as it will allow us to delay buying that very expensive mount.

rnclark said:

rnclark wrote: Of course there are significant factors that would change the curves at low iso in real imaging situations. 1) Fixed pattern noise. There is never 100% correction, even with bias and dark frames. When the problem is bad, one needs to resort to dithering and that favors more sub exposures. 2) As gain (iso) drops, intensity sampling noise increases, so even with no fixed pattern noise, all the curves will turn down when gain (ISO) drops (higher than 1/3 electron/DN sampling). In practice, for 14-bit cameras with 4 to 5 micron pixels, that means noise in the final image increases as ISO drops below about 400. The sampling effects become larger as ISO drops so digitization is > 1/2 e/DN, and is worse for lower read noise. (16-bit A/D will help this.)

So for faint signal detection and 14-bit cameras, ISOs should remain above about 400, and above 800 for canon until they improve the fixed pattern noise.

Click to expand...

Yes, your point about FPN makes intuitive sense.

As for quantization noise the rule of thumb is that light information is properly encoded into the raw data (thus accurately retrievable) as long as the random noise at the input of the ADC is greater than 1 LSB (or DN)*. For instance every Canon DSLR in existence shows read noise greater than 4DN (LSB) at base ISO, four times the dithering target, so quantization would likely not be an issue for them at any ISO.

Things are not as easy for very clean sensors. For instance the D7200 sports read noise in a dark frame of about 0.8DN* at base ISO - that's danger territory and as you know it shows some symptoms of quantization in the deepest shadows of the weakest channels (red and blue) at that ISO. It's about 1.2DN at ISO200 and 2DN by ISO400.

Jim kasson performed an interesting exercise with a D810, which has a base ISO read noise of about 1 DN* at 14-bits: with the green channel not showing symptoms of quantization at base ISO he switched the camera to 12-bit mode. This in theory corresponds to feeding data to the 12-bit ADC with a read noise of 1/4LSB, insufficient for proper information capture. Sure enough the measured green channel showed ringing, one of the symptoms of quantization. He then increased ISO a couple of stops on the 12-bit captures, bringing read noise back to about 1LSB, and re-measured: quantization effects gone.

Note that e- or pixel pitch did not enter the discussion. I believe there is no 'safe' ISO as far as quantization error is concerned: it's 'safe' at whatever ISO the >1LSB-rn-before-ADC condition is met. I think in any case that these low level effects, if present, are virtually impossible to recognize in a normal capture, as shown by Jim's related visual tests. I wonder if they would instead be observable when stacking 100 images.

Jack

*Of course only the manufacturers' engineers know the value of read noise before the ADC, but we can guess at it by measuring the output of the ADC stored in the raw data of, say, a dark frame as a proxy for the amount of noise just before it, hoping the ADC is not too noisy itself. Based on what I have seen 1.5-2 output referred DNs should be safe.

Jack Hogan said:

Jack Hogan said:

rnclark wrote: Of course there are significant factors that would change the curves at low iso in real imaging situations. 1) Fixed pattern noise. There is never 100% correction, even with bias and dark frames. When the problem is bad, one needs to resort to dithering and that favors more sub exposures. 2) As gain (iso) drops, intensity sampling noise increases, so even with no fixed pattern noise, all the curves will turn down when gain (ISO) drops (higher than 1/3 electron/DN sampling). In practice, for 14-bit cameras with 4 to 5 micron pixels, that means noise in the final image increases as ISO drops below about 400. The sampling effects become larger as ISO drops so digitization is > 1/2 e/DN, and is worse for lower read noise. (16-bit A/D will help this.)

So for faint signal detection and 14-bit cameras, ISOs should remain above about 400, and above 800 for canon until they improve the fixed pattern noise.

Click to expand...

Yes, your point about FPN makes intuitive sense.

As for quantization noise the rule of thumb is that light information is properly encoded into the raw data (thus accurately retrievable) as long as the random noise at the input of the ADC is greater than 1 LSB (or DN)*. For instance every Canon DSLR in existence shows read noise greater than 4DN (LSB) at base ISO, four times the dithering target, so quantization would likely not be an issue for them at any ISO.

Things are not as easy for very clean sensors. For instance the D7200 sports read noise in a dark frame of about 0.8DN* at base ISO - that's danger territory and as you know it shows some symptoms of quantization in the deepest shadows of the weakest channels (red and blue) at that ISO. It's about 1.2DN at ISO200 and 2DN by ISO400.

Jim kasson performed an interesting exercise with a D810, which has a base ISO read noise of about 1 DN* at 14-bits: with the green channel not showing symptoms of quantization at base ISO he switched the camera to 12-bit mode. This in theory corresponds to feeding data to the 12-bit ADC with a read noise of 1/4LSB, insufficient for proper information capture. Sure enough the measured green channel showed ringing, one of the symptoms of quantization. He then increased ISO a couple of stops on the 12-bit captures, bringing read noise back to about 1LSB, and re-measured: quantization effects gone.

Note that e- or pixel pitch did not enter the discussion. I believe there is no 'safe' ISO as far as quantization error is concerned: it's 'safe' at whatever ISO the >1LSB-rn-before-ADC condition is met. I think in any case that these low level effects, if present, are virtually impossible to recognize in a normal capture, as shown by Jim's related visual tests. I wonder if they would instead be observable when stacking 100 images.

Jack

*Of course only the manufacturers' engineers know the value of read noise before the ADC, but we can guess at it by measuring the output of the ADC stored in the raw data of, say, a dark frame as a proxy for the amount of noise just before it, hoping the ADC is not too noisy itself. Based on what I have seen 1.5-2 output referred DNs should be safe.

Click to expand...

It gets a little more complex when you start stacking to detect sub one photon per exposure. We had a thread about this a while back about unity gain. I made this model showing the effects of sampling gain in a 100 image stack. Clearly, there is a difference in going from unity gain to 0.5e/dn and a slight gain to 0.3 e/dn. Of course as sky noise, or read noise, or dark current noise increases per exposure, the effect becomes less. The model includes Poisson noise from photons.