XTi/XT controlled low lighting...new dcraw results

RicksAstro

Senior Member
Messages
3,879
Solutions
2
Reaction score
1,163
Location
Waikoloa, HI, US
On others advice, I used dcraw to convert the raw files I took last night of extremely low light imaging. What I posted last night http://forums.dpreview.com/forums/read.asp?forum=1031&message=19887473 , with ridiculously large uncropped images) was using the Canon Utilities to convert the RAW to 16 bit TIF.

What I learned was the Canon utilities do manipulate the raw image extensively... a disappointment. The results from dcraw were SIGNIFICANTLY noisier, but a bit sharper. I'd rather have control of any manipulation, but I guess that's why there are other RAW converters for those more serious.

I had dcraw convert to raw B&W bayer pattern and used my imaging SW (MaximDL) to convert to color using absolutely no color adjustment at all (no black offsets, no color scaling) and used their high quality debayer algorithm. What I got was as as raw of images as you can get with no noise reduction, no sharpening and no color balancing...just what the sensor put out.

All shots are 2 minute exposures through the EF70-200L f/4 lens at f32 and ISO800. No noise reduction on either camera.

The first was under brighter room conditions (desk lamp and 2 monitors on). These both came out with the histogram trailing off in the high 20k's, with the XTi's black histogram edge being a little lower and white histogram edge being a little higher (the XT was more compressed towards the middle). In MaximDL, I set the whitepoint to 32768 and did a gamma stretch of 0.6 to 8 bits. That's it, no other processing whatsoever, and both images treated identically.

Here is a wide wiew crop of both images, with the XTI's scaled down to match the XT's:



Here is a full scale crop of there images, left at their native resolution:



The second series was taken in much lower lighting conditions (only 1 monitor on, and with a dark screen background) Same exact camera setup/exposure as the series above. The histograms showed a similar pattern...the XTi used a wider range of the histogram (lower black, higher white) . On closer inspection, the blue channel of the XTi was lower than the XT and the red channel was higher. This accounted for much of the wider histogram.

I processed them exactly the same as above except I used a whitepoint of 6900 for the 0.6 gamma stretch.

Here is a wide wiew crop of both images, with the XTI's scaled down to match the XT's:



And here is the full resolution crop:



In that last image, you can see the histogram differences, with the blacks grayer in the XT's image and the colors brighter in the XTi. If processed differently to optimize for the histogram for both, these would look more similar. But I didn't want to process these images any differently at all...they went through exactly the same process.

Do these apply to normal imaging? I think so in general. I can say pretty well that, under the circumstances I set up, the sensor on the XTi is not less sensitive overall than the XT. The blue seems less sensitive, but the red more. The green was a wash. This was at f/32 and microlenses have a way of shifting light gathering ability at different light cone angles, so the results could be a bit different with a fast lens.

But for now, I'm convinced that the XTi is not a step down from the XT. It seems similarly noisy (maybe a bit better) and is similarly sensitive (worse blue, better red), so the S/N should be similar to the XT...a significant acheivement with the extra Mpix...

Rick
 
What's the interest in 2 min exposures at f32?

Not being critical just wondering...
 
so I must now raise funds to get the XTi. LOL.

-------------------------------------------
See the colors of my world in:
thw.smugmug.com
 
Despite the pictures have not being fully processed by dcraw (used only to retrieve the bayer pattern) I admit that MaximDL is equally reliable for an unbiased comparison.

The overall different sensibility perceived in most comparisons with 350D seems now to lay in the in-camera jpeg processing (or raw converters using the image parameters stored in the raw) coupled with the metering applied by 400D (unless there were huge sample variations). I must apologize for having judged so quickly the 400D as less sensitive in some of my previous posts and tests adjusting the jpegs exposure. And I suspect that several reviews may still show 400D as less sensitive... humans can't see in raw mode and we depend on the conversions and the tuning parameters.

The slight difference in sensitivity for relative colors is normal for a new sensor; raw converters will account for the different tones seen in these crops when tuned (for example, dcraw has a specific color matrix for each camera, kindly provided by Adobe, but not yet available for 400D).

Noise is really good in both tests... nobody should be alarmed by these tests showing the raw noise (IMO the best way to compare cameras and sensors). The bigger P&S sensor (2/3") would have become overexposed by the noise in less than a minute at ISO 400...
 
I definitely feel better about 400D now :)
--
Urs, Casus.
Canon EOS 300D,
18-55 kit, Tamron 28-75/2.8,
EF 50/1.8, Sigma 70-300,
Sigma EF 500 DG Super
 
What I learned was the Canon utilities do manipulate the raw image
extensively... a disappointment. The results from dcraw were
SIGNIFICANTLY noisier, but a bit sharper. I'd rather have control
of any manipulation, but I guess that's why there are other RAW
converters for those more serious.
I had dcraw convert to raw B&W bayer pattern and used my imaging SW
(MaximDL) to convert to color using absolutely no color adjustment
at all (no black offsets, no color scaling) and used their high
quality debayer algorithm.
MaximDL can demosaic, you say ... does it not also load RAWs, or is it not just updated for the new camera? Does the latest DCRAW have explicit support, or does it just understand the RAW on its own?

I don't know if it supports the 400D yet, but IRIS

http://www.astrosurf.com/buil loads RAWs from the other Canon DSLRs, and will interpolate into color in 3 different ways, and has a function to split the 4 subimages in the RAW into 4 separate files while it is still in the greyscale state. It is quirky in many ways, but vey useful in others. It handles the RAW data quite literally. I have never found DCRAW to be very useful for exact RAW extraction; It has no way to scale the data so that it will load into PS without getting quantized. IRIS has no plugin capability, though, so MaximDL sounds interesting, because it has a plugin SDK as far as I can tell from quickly looking at a google of "MaximDL". That might be a good platform for me to write de-banding plugins. PS is not a very flexible environment, mathematically. Its function is to be a computer-based darkroom, not a mathshop. There's really no reason why it couldn't be both.
What I got was as as raw of images as
you can get with no noise reduction, no sharpening and no color
balancing...just what the sensor put out.
IMO, all the most important and basic noise measurements should be done on the RAW data. That tells you about the camera . Doing it with ACR or Canon conversions tells you about the combined effect of camera and software. Software is ephemeral and arbitrary.

--
John

 
very good test but i can see more details in 350d image , i gess the reason is becouse its more in focus.

all best
kristian
 
2 minutes was mainly to easily see the thermal characteristics. Since thermal noise accumulates linearly, it should be an indication for comparison purposes for much shorter exposures.

The f32 was chosen to allow as little light as possible in, particularly for the dim test. Purely arbitrary. Since I'm just trying to see the chip characteristics, it really didn't matter (except for possible microlens effects).

Rick
 
Maxim's a great program for manipulating an image at the most base levels. It handles bias, dark and flat calibration nicely as well.

It does handle RAWs natively, but it's not updated for the 400D yet. I always capture in native bayer mode for Astro work. Using a serial cable to the remote socket, I can control the XT directly from Maxim, taking any number of extremely long exposures at whatever ISO I choose, all remotely via the PC. I can treat the Rebel exactly as I do any of my dedicated cooled astro CCD cameras.

Maxim has a COM interface and is very easy to write scripts for...I've done many for both camera control as well as image processing.

I've given IRIS a try numerous times, but always leave frustrated with the UI. I'm sure if I gave it a better chance, it's like anything...once you get used to it, it's more intuitive. My. Buil has been a huge contributor to the astro community.

Rick
 
With 2 minute exposure and hand focusing, you can't really use these to judge sharpness at all...I could have bumped my desk or someone could have walked by. This was strictly for noise evaluation.

Rick
 
Rick, hi. Have you tried doing a 10 or 15min dark frame to get a handle on the dark current and amp noise? That would be an interesting comparison with the 350xt as well.

Duncan
With 2 minute exposure and hand focusing, you can't really use
these to judge sharpness at all...I could have bumped my desk or
someone could have walked by. This was strictly for noise
evaluation.

Rick
 
I'm not a big fan on just taking dark frames to analyze noise. It does give some data, but it's missing the S (signal) part of the S/N equation, which is what I'm worried about anyway.

I did do a 5 minute dark to see about amp/electronics glow...the XT had a little in the lower right corner (much, much better than the 10D or 300D) ever had. The XTi had no detectable amp glow at all, which is a very good thing!

Rick
 
I'm not a big fan on just taking dark frames to analyze noise. It
does give some data, but it's missing the S (signal) part of the
S/N equation, which is what I'm worried about anyway.

I did do a 5 minute dark to see about amp/electronics glow...the XT
had a little in the lower right corner (much, much better than the
10D or 300D) ever had. The XTi had no detectable amp glow at all,
which is a very good thing!

Rick
Well, even if the XTi has less signal, it might achieve a better SNR if there is less dark current and amp glow. The 10D/300D have much higher signal than the 20D but stubstantially worse noise, for an overall worse SNR.

Duncan
 
Hi Rick.

I was trying to do similar test with my camera but with poor results. How dark actually was there when you took those shots. I shot in the windowless bathroom leaving the door open just slightly so very little light would come in.

171 sec @ f7.1 ISO800, and no long exposure noise reduction, but the images look like normally noisy ISO 800 shots. I think there might some NR system on my D50 that I can't turn off since there is hardly any of those bright blue and red dots. Would this NR also cause problems if trying to shoot stars?

Full image (sorry, it's a bit dark)



100% Crop



-T

--

 
There is no point in shooting at F32 as the lens resolution and contrast will have gone to pot due to diffraction. Hence both sensors will outresolve the image by miles. F11 would probably be an optimal aperture or maybe F8.
 
The D50, like all Nikons, runs a median filter on long exposures to remove hot pixels. To defeat this, turn NR on, take your shot, but then turn the camera off after the shutter closes and "job NR" begins. By default the camera will then flush the image in the buffer onto the SD card, without running the median filter algorithm. This method of NR is known as "mode 3".

Duncan
 
I didn't try to quantify the light, so it's impossible to compare results apples to apples. The f/32 really cut the light getting to the sensor.

Are you shooting in RAW mode? How are you converting? The noise depends more on how much light is available rather than ISO setting. Like I've said, I use the ISO setting to adjust the dynamic range depending on conditions. If there are no blown highlights in ISO800 (as in your image), it's often better to shoot at ISO1600 to minimize quantization noise at the low end (seen sometimes as posterization of flat, dark areas). If ISO1600 is blowing out highlights, drop it to ISO800 or ISO400...the results will be similar if both are stretched based on their histogram.

My bright images had a fairly normal daytime looking histogram, so they aren't very noisy. My dim images were very, very dark unstretched. Setting a low value for a whitepoint and doing a gamma stretch on the RAW data (presumably still 12 bit A/D on the XTi) made them look more normal, but still very, very noisy on both as expected. I mean, visually I could just barely see the color of the little nascar car and with the lens at f/32, it's remarkable what these cameras captured really.

For shooting stars, your do NOT want any noise reduction. First, it makes calibration (dark subtraction) much, much less effective, and second it wipes out much of the detail in the basement of the histogram, which is where most of the data I want is. That's why I do exposure and calibration in B&W Bayer RAW.

Rick
 

Keyboard shortcuts

Back
Top