Worst Long Exposure Noise Yet - Sony a6300 Surprise

[tradesmith45]

After encountering problems from high LEN from Olympus m43 cams & reading Brendan Davey's LEN sensor tests, I've been doing this Q&D LEN test of various cameras.

I do the same for all tests: room temp, ISO3200, in camera LENR turned off, body or lens cap, cam in a dark container, 2 & 4 min. exposures. I take the captures, crop to about 1/4, boost exposure by 4 EV & use Adobe Standard Camera Profile in LRCC w/ no other adjustments.

Up until now, the Panasonic GH4 has shown the highest LEN. The Oly E-M5 has the lowest LEN of all the m43 cams tested but its still pretty bad. Here's a comparison w/ the a6300:



I have no explanation for why this is so bad. Brendan Davey reported the Sony a7RII has lots of LEN but I had speculated that might be a result of using sensor based IS which means the sensor is floating in the camera so perhaps can't dissipate heat too well.

Also, there is speculation the new Fuji cams use the same Sony sensor but w/ the X-Trans filter arrangement. The Fuji's have the least LEN I've tested so far:



Perhaps one of you with more knowledge about sensor engineering has an explanation but I'm sure surprised.

 

[rnclark]

I have no explanation for why this is so bad. Brendan Davey reported the Sony a7RII has lots of LEN but I had speculated that might be a result of using sensor based IS which means the sensor is floating in the camera so perhaps can't dissipate heat too well.

I have written about Davey's web site here in this forum, and I have written him twice about it, both this year and in 2014. He has never responded. For those interested, this is the web site: http://www.brendandaveyphotography.com/?page_id=726

This is from my latest email to him (Sept 30, 2016):

There are many things wrong with your site that lead to bad conclusions. For example, each camera has a different gain, which means the signal from a subject will be different in each camera. Second, at long exposures, 300 seconds especially, noise from dark current can dominate depending on the camera and camera temperature. You do not report temperature for each camera and about 10 degrees difference can result in a factor of 2 change in noise. Your statistics are on demosaicked data, so comparison between cameras is also a reflection on how that raw converter handled the data, and that can vary from camera to camera. For example, some cameras have no offset (noise gets clipped), some cameras have a large offset, which it looks like your site may report as a large signal. Clipped data will have low noise (because many pixels are just zero), underreporting noise, and giving a false impression of better performance. Some cameras filter the raw data, again giving a false impression of noise performance.

With all those unknowns, let's look at 3 cameras. The Canon 6D with a great reputation for low light, the 7D2 which has the lowest measured dark current of any DSLR that I have seen data for, and a Nikon D7200. Here are the standard deviations at 300 sec, green channel from your site:

6D: 28.69

7D2: 31.79

D7200: 37.80

So your data indicates the 6D is the best low light camera of the 3.

Now based on the known gains from my reviews (estimated for the D7200):

Clarkvision Reviews

I translate the above standard deviations to electrons (in an exposure 1 photon gets 1 electron). (Note photoshop is 15 bits internally, so the gains from my reviews are divided by 2 from the camera raw 14-bit numbers. DN = data number.) I estimated it for the D7200 gain based on the sensor's pixels--probably accurate to ~20%).

6D: 2.7 electrons (gain = 0.093 electron/photoshop DN)

7D2: 1.3 electrons (gain = 0.042 electron/photoshop DN)

D7200 estimate: 1.4 electrons (gain = 0.038 electron/photoshop DN)

So a supposedly noisy camera (the Nikon 7200) actually shows excellent performance, and the 6D significantly worse than the 7D2 and 7200.

Now let's look at real calibrated data. Examine the Canon 6d and 7D2 here:

Canon 6D Review Clarkvision.com: Camera Review and Sensor Analysis Series

Canon 7DII Review Clarkvision.com: Camera Review and Sensor Analysis Series, Canon 7D Mark II

Go down to Table 4a on each page and look at the data for
T= 4C for the 7D2, and T= 6C for the 6D (the closest two temperatures).

The standard deviations using raw data from the camera,
no raw converter (I use dcraw), are:

7D2 standard deviation = 3.27 electrons
6D standard deviation = 13.32 electrons

So the 7D2 has over 4 times lower noise at this temperature and
exposure time in calibrated electrons, which translates directly to
comparison with photon noise.

I am sorry to say that your uncalibrated data are giving people the
wrong information.

Now there is a way to at least estimate the gains to get to electrons for your database, but you will need to extract the raw data with dcraw or rawdigger with no modification of the raw data. If you are interested I can help you.

If you want to learn more on this subject and real low light performance, please see my series on night photography,

Characteristics of Best cameras and lenses for nightscape and astro photography, Clarkvision.com

Roger Clark

 

[Sir Canon]

I have no explanation for why this is so bad. Brendan Davey reported the Sony a7RII has lots of LEN but I had speculated that might be a result of using sensor based IS which means the sensor is floating in the camera so perhaps can't dissipate heat too well.

I have written about Davey's web site here in this forum, and I have written him twice about it, both this year and in 2014. He has never responded. For those interested, this is the web site: http://www.brendandaveyphotography.com/?page_id=726

This is from my latest email to him (Sept 30, 2016):

There are many things wrong with your site that lead to bad conclusions. For example, each camera has a different gain, which means the signal from a subject will be different in each camera. Second, at long exposures, 300 seconds especially, noise from dark current can dominate depending on the camera and camera temperature. You do not report temperature for each camera and about 10 degrees difference can result in a factor of 2 change in noise. Your statistics are on demosaicked data, so comparison between cameras is also a reflection on how that raw converter handled the data, and that can vary from camera to camera. For example, some cameras have no offset (noise gets clipped), some cameras have a large offset, which it looks like your site may report as a large signal. Clipped data will have low noise (because many pixels are just zero), underreporting noise, and giving a false impression of better performance. Some cameras filter the raw data, again giving a false impression of noise performance.

With all those unknowns, let's look at 3 cameras. The Canon 6D with a great reputation for low light, the 7D2 which has the lowest measured dark current of any DSLR that I have seen data for, and a Nikon D7200. Here are the standard deviations at 300 sec, green channel from your site:

6D: 28.69

7D2: 31.79

D7200: 37.80

So your data indicates the 6D is the best low light camera of the 3.

Now based on the known gains from my reviews (estimated for the D7200):

http://www.clarkvision.com/reviews/

I translate the above standard deviations to electrons (in an exposure 1 photon gets 1 electron). (Note photoshop is 15 bits internally, so the gains from my reviews are divided by 2 from the camera raw 14-bit numbers. DN = data number.) I estimated it for the D7200 gain based on the sensor's pixels--probably accurate to ~20%).

6D: 2.7 electrons (gain = 0.093 electron/photoshop DN)

7D2: 1.3 electrons (gain = 0.042 electron/photoshop DN)

D7200 estimate: 1.4 electrons (gain = 0.038 electron/photoshop DN)

So a supposedly noisy camera (the Nikon 7200) actually shows excellent performance, and the 6D significantly worse than the 7D2 and 7200.

Now let's look at real calibrated data. Examine the Canon 6d and 7D2 here:

http://www.clarkvision.com/reviews/evaluation-canon-6d/
http://www.clarkvision.com/reviews/evaluation-canon-7dii/

Go down to Table 4a on each page and look at the data for
T= 4C for the 7D2, and T= 6C for the 6D (the closest two temperatures).

The standard deviations using raw data from the camera,
no raw converter (I use dcraw), are:

7D2 standard deviation = 3.27 electrons
6D standard deviation = 13.32 electrons

So the 7D2 has over 4 times lower noise at this temperature and
exposure time in calibrated electrons, which translates directly to
comparison with photon noise.

I am sorry to say that your uncalibrated data are giving people the
wrong information.

Now there is a way to at least estimate the gains to get to electrons for your database, but you will need to extract the raw data with dcraw or rawdigger with no modification of the raw data. If you are interested I can help you.

If you want to learn more on this subject and real low light performance, please see my series on night photography,
http://www.clarkvision.com/articles...-and-lenses-for-nightscape-astro-photography/

Roger Clark

Roger deserves an award for his dedication to helping and inforning people on this fourm.

 

[Tristimulus]

Have to agree Sir Canon!

 

[Rudy Pohl]

I have no explanation for why this is so bad. Brendan Davey reported the Sony a7RII has lots of LEN but I had speculated that might be a result of using sensor based IS which means the sensor is floating in the camera so perhaps can't dissipate heat too well.

I have written about Davey's web site here in this forum, and I have written him twice about it, both this year and in 2014. He has never responded. For those interested, this is the web site: http://www.brendandaveyphotography.com/?page_id=726

This is from my latest email to him (Sept 30, 2016):

....

With all those unknowns, let's look at 3 cameras. The Canon 6D with a great reputation for low light, the 7D2 which has the lowest measured dark current of any DSLR that I have seen data for, and a Nikon D7200. Here are the standard deviations at 300 sec, green channel from your site:

6D: 28.69

7D2: 31.79

D7200: 37.80

So your data indicates the 6D is the best low light camera of the 3.

Now based on the known gains from my reviews (estimated for the D7200):

http://www.clarkvision.com/reviews/

...

I estimated it for the D7200 gain based on the sensor's pixels--probably accurate to ~20%).

6D: 2.7 electrons (gain = 0.093 electron/photoshop DN)

7D2: 1.3 electrons (gain = 0.042 electron/photoshop DN)

D7200 estimate: 1.4 electrons (gain = 0.038 electron/photoshop DN)

So a supposedly noisy camera (the Nikon 7200) actually shows excellent performance, and the 6D significantly worse than the 7D2 and 7200.

Roger Clark

Hi Roger:

As an owner of the NIkon D7200 and having used it extensively for Milky Way photography last season (and I will again this coming season), you wrote a similar post to the one above in one of my threads back in August 2016. Unfortunately, I was as lost in the technical understanding back then as I am reading your post today. So I was wondering if it were possible for you to translate your findings comparing these three cameras into a format for technically challenged laypersons like myself?

What I had in mind in something like this: If these cameras were rated on a scale of 1 to 10 with 10 being the best, how would you numerically rate these three cameras in relation to one another (7DMKII, 7D and D7200)?

For example, are we able to say something like "The 7DMKII gets 10 out of 10, the D7200 gets 7/10 and the 7D gets 5/10 when it comes to sensor noise." Is this kind of simple, layperson comparison possible and if so what would the actual comparison numbers be? If not, no problem and thanks anyway,

Best regards,
Rudy

 

[rnclark]

6D: 2.7 electrons (gain = 0.093 electron/photoshop DN)

7D2: 1.3 electrons (gain = 0.042 electron/photoshop DN)

D7200 estimate: 1.4 electrons (gain = 0.038 electron/photoshop DN)

So a supposedly noisy camera (the Nikon 7200) actually shows excellent performance, and the 6D significantly worse than the 7D2 and 7200.

Roger Clark

Hi Roger:

As an owner of the NIkon D7200 and having used it extensively for Milky Way photography last season (and I will again this coming season), you wrote a similar post to the one above in one of my threads back in August 2016. Unfortunately, I was as lost in the technical understanding back then as I am reading your post today. So I was wondering if it were possible for you to translate your findings comparing these three cameras into a format for technically challenged laypersons like myself?

What I had in mind in something like this: If these cameras were rated on a scale of 1 to 10 with 10 being the best, how would you numerically rate these three cameras in relation to one another (7DMKII, 7D and D7200)?

For example, are we able to say something like "The D7200 is approximately 70% as good as the 7DMKII and is twice as good as the 7D when it comes to sensor noise." Is this kind of simple, layperson comparison possible and if so what would the actual comparision numbers be? If not, no problem and thanks anyway,

Best regards,
Rudy

Hi Rudy,

We don't know the temperatures of the cameras, but if we assume they were at the same temperature, the key numbers are:

Noise:

6D: 2.7 electrons

7D2: 1.3 electrons

D7200 estimate: 1.4

So the D7200 is only 1.4/1.3 =1.08 or only 8% worse that the 7D2 in a 300 second exposure, while the 6D is 2.7/1.3 = 2.08 or over a factor of 2 worse. Because we don't know the temperatures, this assessment might be off by factors of more than 2 to 4 or more from the Davey data.

If we use my data for 600 second exposures with the cameras at close to the same temperature, we see

7D2 standard deviation = 3.27 electrons
6D standard deviation = 13.32 electrons

Thus the 7D2 is 13.32 / 3.27 = 4.1 or more than 4 times better than the 6D at long exposure low light. This also implies the Davey data can be off by factors of 2, and that is after scaling the Davey data to uniform units like electrons.

Roger

 

[Rudy Pohl]

6D: 2.7 electrons (gain = 0.093 electron/photoshop DN)

7D2: 1.3 electrons (gain = 0.042 electron/photoshop DN)

D7200 estimate: 1.4 electrons (gain = 0.038 electron/photoshop DN)

So a supposedly noisy camera (the Nikon 7200) actually shows excellent performance, and the 6D significantly worse than the 7D2 and 7200.

Roger Clark

Hi Roger:

As an owner of the NIkon D7200 and having used it extensively for Milky Way photography last season (and I will again this coming season), you wrote a similar post to the one above in one of my threads back in August 2016. Unfortunately, I was as lost in the technical understanding back then as I am reading your post today. So I was wondering if it were possible for you to translate your findings comparing these three cameras into a format for technically challenged laypersons like myself?

What I had in mind in something like this: If these cameras were rated on a scale of 1 to 10 with 10 being the best, how would you numerically rate these three cameras in relation to one another (7DMKII, 7D and D7200)?

For example, are we able to say something like "The D7200 is approximately 70% as good as the 7DMKII and is twice as good as the 7D when it comes to sensor noise." Is this kind of simple, layperson comparison possible and if so what would the actual comparision numbers be? If not, no problem and thanks anyway,

Best regards,
Rudy

Hi Rudy,

We don't know the temperatures of the cameras, but if we assume they were at the same temperature, the key numbers are:

Noise:

6D: 2.7 electrons

7D2: 1.3 electrons

D7200 estimate: 1.4

So the D7200 is only 1.4/1.3 =1.08 or only 8% worse that the 7D2 in a 300 second exposure, while the 6D is 2.7/1.3 = 2.08 or over a factor of 2 worse. Because we don't know the temperatures, this assessment might be off by factors of more than 2 to 4 or more from the Davey data.

Can we therefore say that with respect to the above data the 7DMKII scores a 10/10 (the best of our 3 cameras is automatically set to 10/10), the 7D is about 5/10 (factor of 2 worse), and the D7200 is about a 9/10 (about 10% worse than the 7DMKII)?

Rudy

If we use my data for 600 second exposures with the cameras at close to the same temperature, we see

7D2 standard deviation = 3.27 electrons
6D standard deviation = 13.32 electrons

Thus the 7D2 is 13.32 / 3.27 = 4.1 or more than 4 times better than the 6D at long exposure low light. This also implies the Davey data can be off by factors of 2, and that is after scaling the Davey data to uniform units like electrons.

Roger

 

[tradesmith45]

Thanks Roger for your technically well grounded info. I'm trying to address a different set of problems however. Let me explain.

As we all know many folks are now trying night landscapes including several members of my photo club. Both they and many posters here are trying to recover from a previous (or about to make a future- poor equipment choice for this kind of shooting. All the major equipment review articles and sites are mute on the topic of long exposure noise. My posts here and other locations are an attempt to bring more attention to this subject that I've seen be a surprise to many general photographers. I do this in the hopes that consumers will be provided with more information on this topic from at least one of the major review organizations. We need something like what LensTip.com has provided us about lenses.

Currently we only have either limited coverage of a few cameras using highly technical evaluations and language common to astronomy, a few reviews of specific cameras by starry landscape sites or Brendan Davey's simplistic coverage of a broad range of cameras. From watching members of my photo club and the local astronomy club I'm also a member of, its clear that the language of astronomy is completely opaque for the vast majority of amateur photographers.

A club member has loaned me the Sony a6300 after being surprised by his images. Having tried Davey's simple dark frame test over a half dozen times, I've found it gives a good indication of how much noise to expect in a long night landscape exposure. Precision and technical detail aren't needed to gain this useful information.

Given all this, my current view is Davey's work should be applauded and even promoted (w/ caveats). Ultimately we need something different from either what he's doing or what you are doing. We need the long exposure-high ISO equivalent of DPR's Studio Shot that anyone can download or look at online that is a standard part of all camera reviews.

From this perspective, I thought Davey's addition of numerical analysis to his simple display of dark frames was a step backwards for 3 reasons: it obscured to much of the images, it was obviously technically fraught and a better step would be to show the whole dark frame in addition to the crop because there's more to be seen.

Some compact mirrorless cameras & perhaps small DSLRs seem to be bringing a new set of issues to this topic - localized in-camera heating & noise patterns from the on-sensor PDAF detectors. Both of those can be seen here:



Hope this clarifies my purposes.

 

[rnclark]

6D: 2.7 electrons (gain = 0.093 electron/photoshop DN)

7D2: 1.3 electrons (gain = 0.042 electron/photoshop DN)

D7200 estimate: 1.4 electrons (gain = 0.038 electron/photoshop DN)

So a supposedly noisy camera (the Nikon 7200) actually shows excellent performance, and the 6D significantly worse than the 7D2 and 7200.

Roger Clark

Hi Roger:

As an owner of the NIkon D7200 and having used it extensively for Milky Way photography last season (and I will again this coming season), you wrote a similar post to the one above in one of my threads back in August 2016. Unfortunately, I was as lost in the technical understanding back then as I am reading your post today. So I was wondering if it were possible for you to translate your findings comparing these three cameras into a format for technically challenged laypersons like myself?

What I had in mind in something like this: If these cameras were rated on a scale of 1 to 10 with 10 being the best, how would you numerically rate these three cameras in relation to one another (7DMKII, 7D and D7200)?

For example, are we able to say something like "The D7200 is approximately 70% as good as the 7DMKII and is twice as good as the 7D when it comes to sensor noise." Is this kind of simple, layperson comparison possible and if so what would the actual comparision numbers be? If not, no problem and thanks anyway,

Best regards,
Rudy

Hi Rudy,

We don't know the temperatures of the cameras, but if we assume they were at the same temperature, the key numbers are:

Noise:

6D: 2.7 electrons

7D2: 1.3 electrons

D7200 estimate: 1.4

So the D7200 is only 1.4/1.3 =1.08 or only 8% worse that the 7D2 in a 300 second exposure, while the 6D is 2.7/1.3 = 2.08 or over a factor of 2 worse. Because we don't know the temperatures, this assessment might be off by factors of more than 2 to 4 or more from the Davey data.

Can we therefore say that with respect to the above data the 7DMKII scores a 10/10 (the best of our 3 cameras is automatically set to 10/10), the 7D is about 5/10 (factor of 2 worse), and the D7200 is about a 9/10 (about 10% worse than the 7DMKII)?

Rudy

Yes, that works. (Note 6D is about 5/10, not 7D). But with the caveat that we don't know the temperatures so where the D7200 falls could be off a factor or 2 or so (either way--better or worse).

 

[rnclark]

Thanks Roger for your technically well grounded info. I'm trying to address a different set of problems however. Let me explain.

As we all know many folks are now trying night landscapes including several members of my photo club. Both they and many posters here are trying to recover from a previous (or about to make a future- poor equipment choice for this kind of shooting. All the major equipment review articles and sites are mute on the topic of long exposure noise. My posts here and other locations are an attempt to bring more attention to this subject that I've seen be a surprise to many general photographers. I do this in the hopes that consumers will be provided with more information on this topic from at least one of the major review organizations. We need something like what LensTip.com has provided us about lenses.

I agree completely. When I launched the 7D2 review in the fall of 2014, I had intended to do a camera review every month or two. Two things prevented that from happening. 1) My spacecraft work got more complicated and I had to recalibrate an instrument which took until fall this year, and 2) I took on Nikon D800 and D810 sensor reviews. Something is fishy in the raw data and I have been working to unravel any raw data filtering that is being applied. I will not do any new reviews until that is sorted out. Filtering impacts derived parameters including gains, read noise and dark current and makes things look better than they really are.

I have no desire to corner these analyses. That is why in my email to Brendan I offered to help him put his data on better footing. Gains can be estimated based on pixel area, then the raw data extracted with DCRAW. Some cameras report the camera temperature.

Currently we only have either limited coverage of a few cameras using highly technical evaluations and language common to astronomy, a few reviews of specific cameras by starry landscape sites or Brendan Davey's simplistic coverage of a broad range of cameras. From watching members of my photo club and the local astronomy club I'm also a member of, its clear that the language of astronomy is completely opaque for the vast majority of amateur photographers.

A club member has loaned me the Sony a6300 after being surprised by his images. Having tried Davey's simple dark frame test over a half dozen times, I've found it gives a good indication of how much noise to expect in a long night landscape exposure. Precision and technical detail aren't needed to gain this useful information.

But like I have outlined earlier in the thread, if you just look at uncalibrated dark frames stretched hard, you can't compare between cameras.

Given all this, my current view is Davey's work should be applauded and even promoted (w/ caveats).

If put on a scale where cameras could actually be compared I would agree. But at present, one is led to a conclusion that can point to cameras with poor low light performance. For example, by Brendan's web site, one would never choose a Canon 7D2 for low light even though from the calibrated data we have, it is the top performing long exposure low light digital camera on the market.

Ultimately we need something different from either what he's doing or what you are doing. We need the long exposure-high ISO equivalent of DPR's Studio Shot that anyone can download or look at online that is a standard part of all camera reviews.

Yes, I agree, but it is not that simple. The manufacturers should give us the information. They certainly have it.

But to assess it is difficult if they are filtering the raw data. I liken it to cheating, like Volkswagen did using software to cheat on emissions tests so the cars looked like they performed better than they really do. Assessing filtering is difficult. For example, a Nikon D800 gets great review specs, but in a side by side test with am old Canon 7D Mark 1, they recorded about the same level of faint stars. By the specs that shouldn't be.

From this perspective, I thought Davey's addition of numerical analysis to his simple display of dark frames was a step backwards for 3 reasons: it obscured to much of the images, it was obviously technically fraught and a better step would be to show the whole dark frame in addition to the crop because there's more to be seen.

Some compact mirrorless cameras & perhaps small DSLRs seem to be bringing a new set of issues to this topic - localized in-camera heating & noise patterns from the on-sensor PDAF detectors. Both of those can be seen here:



Hope this clarifies my purposes.

That dark frame looks really bad by today's standards. But we don't know how bad (if at all in practice) unless the frame is calibrated. We need to know the min and max in electrons. In theory, that could be better than a 7D2 (though I doubt it).

See this comparison (Figure 1) with two images at the same electron intensity scale:

On-Sensor Dark Current Suppression, Clarkvision.com

That is a factor of 10 difference in low light. Plus, by calibrating the high end, pixel efficiency has improved 2.5x between the two cameras. Thus low light performance between the two cameras is different by a factor of about 25! But I could present the data so that that poor performing camera actually looked better than the 7D2.

Roger

 

[Rudy Pohl]

6D: 2.7 electrons (gain = 0.093 electron/photoshop DN)

7D2: 1.3 electrons (gain = 0.042 electron/photoshop DN)

D7200 estimate: 1.4 electrons (gain = 0.038 electron/photoshop DN)

So a supposedly noisy camera (the Nikon 7200) actually shows excellent performance, and the 6D significantly worse than the 7D2 and 7200.

Roger Clark

Hi Roger:

As an owner of the NIkon D7200 and having used it extensively for Milky Way photography last season (and I will again this coming season), you wrote a similar post to the one above in one of my threads back in August 2016. Unfortunately, I was as lost in the technical understanding back then as I am reading your post today. So I was wondering if it were possible for you to translate your findings comparing these three cameras into a format for technically challenged laypersons like myself?

What I had in mind in something like this: If these cameras were rated on a scale of 1 to 10 with 10 being the best, how would you numerically rate these three cameras in relation to one another (7DMKII, 7D and D7200)?

For example, are we able to say something like "The D7200 is approximately 70% as good as the 7DMKII and is twice as good as the 7D when it comes to sensor noise." Is this kind of simple, layperson comparison possible and if so what would the actual comparision numbers be? If not, no problem and thanks anyway,

Best regards,
Rudy

Hi Rudy,

We don't know the temperatures of the cameras, but if we assume they were at the same temperature, the key numbers are:

Noise:

6D: 2.7 electrons

7D2: 1.3 electrons

D7200 estimate: 1.4

So the D7200 is only 1.4/1.3 =1.08 or only 8% worse that the 7D2 in a 300 second exposure, while the 6D is 2.7/1.3 = 2.08 or over a factor of 2 worse. Because we don't know the temperatures, this assessment might be off by factors of more than 2 to 4 or more from the Davey data.

Can we therefore say that with respect to the above data the 7DMKII scores a 10/10 (the best of our 3 cameras is automatically set to 10/10), the 7D is about 5/10 (factor of 2 worse), and the D7200 is about a 9/10 (about 10% worse than the 7DMKII)?

Rudy

Yes, that works. (Note 6D is about 5/10, not 7D). But with the caveat that we don't know the temperatures so where the D7200 falls could be off a factor or 2 or so (either way--better or worse).

Thanks Roger. Here's hoping that you might consider becoming an unofficial astro camera rating service on some of the most popular models so people like us who can't grasp the math can make confident buying decisions when choosing a camera for astrophotography.

Thanks again for all your help and best regards,
Rudy

 

[tradesmith45]

Hi Roger, sorry holidays have taken me away from this conversation for awhile.

Still trying to figure out a good way to do these comparisons so camera buyers can get useful information.

Thanks Roger for your technically well grounded info. I'm trying to address a different set of problems however. Let me explain.

As we all know many folks are now trying night landscapes including several members of my photo club. Both they and many posters here are trying to recover from a previous (or about to make a future- poor equipment choice for this kind of shooting. All the major equipment review articles and sites are mute on the topic of long exposure noise. My posts here and other locations are an attempt to bring more attention to this subject that I've seen be a surprise to many general photographers. I do this in the hopes that consumers will be provided with more information on this topic from at least one of the major review organizations. We need something like what LensTip.com has provided us about lenses.

I agree completely. When I launched the 7D2 review in the fall of 2014, I had intended to do a camera review every month or two. Two things prevented that from happening. 1) My spacecraft work got more complicated and I had to recalibrate an instrument which took until fall this year, and 2) I took on Nikon D800 and D810 sensor reviews. Something is fishy in the raw data and I have been working to unravel any raw data filtering that is being applied. I will not do any new reviews until that is sorted out. Filtering impacts derived parameters including gains, read noise and dark current and makes things look better than they really are.

I have no desire to corner these analyses. That is why in my email to Brendan I offered to help him put his data on better footing. Gains can be estimated based on pixel area, then the raw data extracted with DCRAW. Some cameras report the camera temperature.

Imagine if you can't understand what's going on under the hood of the D800/810 what the average amateur is up against! I'm not feeling so bad about my struggles understanding the latest Fuji & Sony.

On that score, I'm thinking Fuji filters or clips the low end but does a great job heat sinking while Sony may be using an offset but not so good heat sinks in their little cams. Based on my comparison of Sony sensors in older vs latest Fuji, the newest sensor seems to have more thermal noise during a long exposure than the older sensor. But DXO data says read noise levels from the new Sony are very low 2.77 vs 3.5 DN in 2 yr old chip. So the X-T2 should be better than the X-T10 but sure doesn't look it though they both yield really clean images.

I'm tempted to get a copy of DCraw to do a more in-depth eval but would rather spend my time on other things. I've been playing w/ a demo version of Iridient for other reasons & may try that.

Currently we only have either limited coverage of a few cameras using highly technical evaluations and language common to astronomy, a few reviews of specific cameras by starry landscape sites or Brendan Davey's simplistic coverage of a broad range of cameras. From watching members of my photo club and the local astronomy club I'm also a member of, its clear that the language of astronomy is completely opaque for the vast majority of amateur photographers.

A club member has loaned me the Sony a6300 after being surprised by his images. Having tried Davey's simple dark frame test over a half dozen times, I've found it gives a good indication of how much noise to expect in a long night landscape exposure. Precision and technical detail aren't needed to gain this useful information.

But like I have outlined earlier in the thread, if you just look at uncalibrated dark frames stretched hard, you can't compare between cameras.

I've got lots of thoughts but no conclusions here. My photo club has a monthly Forum night where members bring images to ask for help or just share. Several night landscape issue have been brought in the last few months. That's how I got the Sony on loan. So my thinking is evolving based on seeing how other photographers are responding to these problems.

Also I've been really surprised how much difference there is in the images from different high performing cameras - D750, A7Rii, D500, X-T2, 6D, a6xxx. The D800/810 aren't the only cameras where something unique is being done under the hood.

Watching all this, I'm inclined to want to approach this like a skilled amateur rather than an advanced astronomer. Folks select a camera based on the best recommendations they can get, shot it w/ settings from a workshop or someone w/ some experience & then do some limited post most often in LR/PS.

Given all this, my current view is Davey's work should be applauded and even promoted (w/ caveats).

If put on a scale where cameras could actually be compared I would agree. But at present, one is led to a conclusion that can point to cameras with poor low light performance. For example, by Brendan's web site, one would never choose a Canon 7D2 for low light even though from the calibrated data we have, it is the top performing long exposure low light digital camera on the market.

The problem I've run into w/ Davey's site is that its pretty much meaningless to an average amateur. Their basic question: will noise intrude into their night landscape too much for their taste. Looking at small crops or even crops w/ a numerical rating can never tell them that. Calibrated or not there is no reference for what good should look like. That's why I'm thinking we need a revealing long exposure scene so that folks can see what they'll get. Problem is, that assumes some simple standard post processing can be used. And that's just not possible for all sorts of reasons.

So here's something that's complete heresy. Since the camera makers know how their product is designed to work, maybe we should shot the standard scene and look at the OOC JPGs? The camera makers will have adjusted all the image parameters to get the best from their camera, right?

I've started playing w/ this idea just to see. So far my (indoor) night scene isn't good enough. The most noise revealing night scenes include star light + air glow/lite LP reflecting off water, rocks & tress. There has to be a way to do this to get standardized test images that are accessible to an average amateur photographer. Again, not sure how much time I'm willing to put into this.

Of course this approach is worthless for an astronomer who wants quantative noise data to estimate minimum capture times.

Ultimately we need something different from either what he's doing or what you are doing. We need the long exposure-high ISO equivalent of DPR's Studio Shot that anyone can download or look at online that is a standard part of all camera reviews.

Yes, I agree, but it is not that simple. The manufacturers should give us the information. They certainly have it.

But to assess it is difficult if they are filtering the raw data. I liken it to cheating, like Volkswagen did using software to cheat on emissions tests so the cars looked like they performed better than they really do. Assessing filtering is difficult. For example, a Nikon D800 gets great review specs, but in a side by side test with am old Canon 7D Mark 1, they recorded about the same level of faint stars. By the specs that shouldn't be.

From this perspective, I thought Davey's addition of numerical analysis to his simple display of dark frames was a step backwards for 3 reasons: it obscured to much of the images, it was obviously technically fraught and a better step would be to show the whole dark frame in addition to the crop because there's more to be seen.

Some compact mirrorless cameras & perhaps small DSLRs seem to be bringing a new set of issues to this topic - localized in-camera heating & noise patterns from the on-sensor PDAF detectors. Both of those can be seen here:



Hope this clarifies my purposes.

That dark frame looks really bad by today's standards. But we don't know how bad (if at all in practice) unless the frame is calibrated. We need to know the min and max in electrons. In theory, that could be better than a 7D2 (though I doubt it).

See this comparison (Figure 1) with two images at the same electron intensity scale:

http://www.clarkvision.com/articles//dark-current-suppression-technology/

That is a factor of 10 difference in low light. Plus, by calibrating the high end, pixel efficiency has improved 2.5x between the two cameras. Thus low light performance between the two cameras is different by a factor of about 25! But I could present the data so that that poor performing camera actually looked better than the 7D2.

Roger

 

[tradesmith45]

Now there is a way to at least estimate the gains to get to electrons for your database, but you will need to extract the raw data with dcraw or rawdigger with no modification of the raw data. If you are interested I can help you.

The a6300/X-T2 Sony sensor uses their new thinner copper design that is supposed to expose more silicon to photons. My test images are showing about 0.2-0.3 EV greater brightness for a gray card than images from older sensors w/ same exposure. So to create a reasonable comparison means among other things estimating gain.

What procedure were you thinking of in your comment to Davey?

Thanks much!

 

[rnclark]

Now there is a way to at least estimate the gains to get to electrons for your database, but you will need to extract the raw data with dcraw or rawdigger with no modification of the raw data. If you are interested I can help you.

The a6300/X-T2 Sony sensor uses their new thinner copper design that is supposed to expose more silicon to photons. My test images are showing about 0.2-0.3 EV greater brightness for a gray card than images from older sensors w/ same exposure. So to create a reasonable comparison means among other things estimating gain.

What procedure were you thinking of in your comment to Davey?

Gain is proportional to pixel area. So given a pixel size, one can estimate gain. With modern sensors (the last few years), sensors have a maximum capacity of about 1800 to 2100 electrons per square micron (ISO 100). Older sensors are less. So one can use the age of the camera to estimate gain below current models.

Given gain, one can scale image data to a photon level and put them on a scale that can be compared. And this should be done on the raw data before Bayer demosaicking.

It is a little more complicated with cameras that clip the low end. Then some light on the sensor is needed to lift the signal so few pixels are clipped. For example, a Nikon D800 clips over 80% of the pixels at some ISOs.

The results should be accurate within about +/- 10%.

Roger

 

[tradesmith45]

Now there is a way to at least estimate the gains to get to electrons for your database, but you will need to extract the raw data with dcraw or rawdigger with no modification of the raw data. If you are interested I can help you.

The a6300/X-T2 Sony sensor uses their new thinner copper design that is supposed to expose more silicon to photons. My test images are showing about 0.2-0.3 EV greater brightness for a gray card than images from older sensors w/ same exposure. So to create a reasonable comparison means among other things estimating gain.

What procedure were you thinking of in your comment to Davey?

Gain is proportional to pixel area. So given a pixel size, one can estimate gain. With modern sensors (the last few years), sensors have a maximum capacity of about 1800 to 2100 electrons per square micron (ISO 100). Older sensors are less. So one can use the age of the camera to estimate gain below current models.

Given gain, one can scale image data to a photon level and put them on a scale that can be compared. And this should be done on the raw data before Bayer demosaicking.

It is a little more complicated with cameras that clip the low end. Then some light on the sensor is needed to lift the signal so few pixels are clipped. For example, a Nikon D800 clips over 80% of the pixels at some ISOs.

The results should be accurate within about +/- 10%.

Roger

Thanks for the reply. I'm trying to figure out if the RAW developer I own, DCRAW based Iridient, will give me the data needed prior to de-mosaicing. Using DCRAW on a Mac seems to have a steep learning curve.

I'm digging into these newest sensor cameras a bit more because I'm starting to suspect the new designs may have brought certain performance improvements but at a cost of greater thermal noise.

I'm going to start w/ the easy things - your standard stretched 10 min. dark @ ISO 1600. Cliff has derived gain for most of these cameras except Fuji which DXO does not test due to the unique X-trans arrangement.

 

[bclaff]

Now there is a way to at least estimate the gains to get to electrons for your database, but you will need to extract the raw data with dcraw or rawdigger with no modification of the raw data. If you are interested I can help you.

The a6300/X-T2 Sony sensor uses their new thinner copper design that is supposed to expose more silicon to photons. My test images are showing about 0.2-0.3 EV greater brightness for a gray card than images from older sensors w/ same exposure. So to create a reasonable comparison means among other things estimating gain.

If you are looking for e-/DN then you can use the following Unity ISO values:

ILCE-6300 251.6
Fujifilm X-T2 214.4

To get your gain simply divide Unity ISO by the ISO setting.

In both cases these are not valid at ISO 6400 and above.

Also for the X-T2 use 200 rather than 100 for ISO 100 since it is not truly ISO 200.

If you need figures for other cameras, let me know.
I don't expose this on my site as nicely as I should

 

[tradesmith45]

Now there is a way to at least estimate the gains to get to electrons for your database, but you will need to extract the raw data with dcraw or rawdigger with no modification of the raw data. If you are interested I can help you.

The a6300/X-T2 Sony sensor uses their new thinner copper design that is supposed to expose more silicon to photons. My test images are showing about 0.2-0.3 EV greater brightness for a gray card than images from older sensors w/ same exposure. So to create a reasonable comparison means among other things estimating gain.

If you are looking for e-/DN then you can use the following Unity ISO values:

ILCE-6300 251.6
Fujifilm X-T2 214.4

To get your gain simply divide Unity ISO by the ISO setting.

In both cases these are not valid at ISO 6400 and above.

Also for the X-T2 use 200 rather than 100 for ISO 100 since it is not truly ISO 200.

If you need figures for other cameras, let me know.
I don't expose this on my site as nicely as I should

--
Bill ( Your trusted source for independent sensor data at http://www.photonstophotos.net )

Ok, thanks Bill this could save me some time. Seems to correspond to what I'm finding. Unity ISO for the ILC-6300 is 15% higher than the X-T2.

To make sure I'm correctly understanding how to scale these, here's another dark frame comparison. W/ the X-T2 boosted by +4 EV, the boost for the a6300 dark should be +3.2 EV to simulate matched gain. This should actually be done by making the 6300 dark frame at a lower ISO. (BTW both these are cropped from the center where PDAF pixels are located and appear to be the most noisy areas.)

If I have this right, still looks like more thermal noise from the a6300. But if you look closely, there is another difference between these 2. In between the larger bright pixels of the a6300, there some small white noise that's not seen from the X-T2. Wondering if that means the blacks are clipped in the X-T2?