Reading mode: Light Dark

7D vertical banding - firmware error

Go to last post
John Sheehy said:
Fred Dominic said:
In any event, might I ask if you could better explain how it was that you came to the conclusion that this is post read and after the ADC?
Click to expand...
Because the very deepest modulations of the periodic banding only exists when the camera writes the RAW. Take shot 1 at 1 second, wait a little so the camera doesn't warm up, take shot 2 at 1 second, then, enable long exposure noise reduction for
Click to expand...
Ok, so I can't say that I understood what you meant after reading the above, but I did a bit of background reading on LENR, and essentially it sounds like the camera itself is doing blackframe subtraction (sorry, I'm still quite a camera newbie).

So let me then try to rephrase what you are saying (let me know if I've messed it up).

On the one hand, we've got an image x and a blackframe b, and the camera itself somehow generates b (when LENR is turned on) so the
camera does

x - b

and then does some further processing (lets call this further processing P[]), so
the image we get is:

y = P[x - b]

and you are seeing vertical banding in this y.

On the other hand, lets say we've got an image x, and we generate our own black
frame b. When we do our own blackframe subtraction, we get:

y' = P[x] - P

and you're saying that y' does not exhibit significant banding, but y does. Therefore, the banding must be introduced not by the sensor but by the further processing P[]. When we construct y', we are subtracting off any artifact caused
by P which is why y' does not show significant banding.

Is that essentially what you're saying?
 
BTW, does the RAW file information also contain the average sensor temperature at the time of exposure? If so, it would be possible to keep a database of blackframes on the internet for a 7D (a number of black-frame samples for each exposure-time and temperature combination), and then software like lightroom could use those to do blackframe subtraction at PP time, no?
 
Fred Dominic said:
On the other hand, lets say we've got an image x, and we generate our own black
frame b. When we do our own blackframe subtraction, we get:

y' = P[x] - P

and you're saying that y' does not exhibit significant banding, but y does. Therefore, the banding must be introduced not by the sensor but by the further processing P[]. When we construct y', we are subtracting off any artifact caused
by P which is why y' does not show significant banding.
Click to expand...
Fred Dominic said:
Is that essentially what you're saying?
Click to expand...
Yes. There is some very low contrast subliminal periodic banding in y', but y has it much deeper (deep enough so that there are almost no pixels greater than the mean blackpoint in some lines, for a corduroy effect) and at a higher frequency, and it is more visible in y in higher tonal ranges in actual images.

--
John

 
Fred Dominic said:
BTW, does the RAW file information also contain the average sensor temperature at the time of exposure? If so, it would be possible to keep a database of blackframes on the internet for a 7D (a number of black-frame samples for each exposure-time and temperature combination), and then software like lightroom could use those to do blackframe subtraction at PP time, no?
Click to expand...
I was assuming uniqueness for each camera, but I don't know that. I could subtract my own BF from an Imaging-Resource RAW, though; it could be that one specific correction was left in the firmware. Stranger things have happened. I don't think temperature has anything to do with this banding. I just tried to avoiding heating up the sensor so that the non-banding noise was as similar as possible.

--
John

 
J Watkins said:
For what it's worth I had vertical banding in borad daylight at ISO100 with a properly exposed and even .5 to 1 stop over video I shot at a local park.
It appeared in fairly calm water with a couple ducks floating across a pond.
From there it only got worse and ISO1600 was horrible.

The camera had already made one trip to canon and it wasn't fixed, focus was better but...
I ended up swapping it for a newer one that is much better.

I still have some minor banding in shadow areas that is worse in LR2.6 that it is in DPP for some reason.

I haven't tried forcing any banding in video with this body but I'm with the masses hoping there is a 'fix' via firmware soon!

Canon are you listening????? I didn't hop on the 5DMKII for the issues that it suffered out of the gate like the specular highlight black spots and it's banding.

I figured (hoped) the 7D had enough R&D that Canon had learned from their mistakes but I'm thinking that's not the case.

I guess I got spoiled but a flawless 40D and hoped the 7's would be as good or better.
Click to expand...
This is typical problem on 5D2 and 7D. I bet it's the same on 1D4 with digic4
 
John Sheehy wrote:
;[anip]
John Sheehy said:
Sloppy, sloppy Canon.
Click to expand...
I'm not surprised by anything anymore. Toyota, Canon, Congress: they all have the same bad attitude (sort of like mine).
John Sheehy said:
Edit: Actually, there is a very slight periodicity in my own black frame subtractions, but it is much lower contrast than the camera's

--
John

Click to expand...
--
Fred
 
landmarks4711 said:
J Watkins said:
I figured (hoped) the 7D had enough R&D that Canon had learned from their mistakes but I'm thinking that's not the case.

I guess I got spoiled but a flawless 40D and hoped the 7's would be as good or better.
Click to expand...
This is typical problem on 5D2 and 7D. I bet it's the same on 1D4 with digic4
Click to expand...
It is. The 1D4 has 5D2-like burlap patterns in the shadows at base ISO.

If this is a bandwidth issue, then at the very least, they should make a slower readout mode on their "better" cameras for those who don't need the 8 or 10 fps. The Rebel 500D has a lower framerate than the 50D, and it has no visible pattern noise at all at ISO 100, but the 50D does, with the same MP count!

--
John

 
John Sheehy said:
If this is a bandwidth issue, then at the very least, they should make a slower readout mode on their "better" cameras for those who don't need the 8 or 10 fps. The Rebel 500D has a lower framerate than the 50D, and it has no visible pattern noise at all at ISO 100, but the 50D does, with the same MP count!

--
John

Click to expand...
Still more likely the higher number of pipelines is the culprit here. Not the speed per se, but the parallel processing is what's creating the problem.

Now if the 550 doesn't show the problem...
 
agorabasta said:
Still more likely the higher number of pipelines is the culprit here. Not the speed per se, but the parallel processing is what's creating the problem.

Now if the 550 doesn't show the problem...
Click to expand...
The higher number of pipelines might account for some banding, but not what I am talking about. Carefully re-read what I wrote in this thread. The pipelines can not be responsible for this particular problem. They may be what the camera is trying to correct, but the camera corrects the problem incorrectly . The artifact of which I speak is the same (repeats) in all frames, but is still present in a camera subtraction, but not a subtraction from individual files. It, therefore, must be done by firmware. I am not talking about subtle subliminal banding. I am talking about distinct light and dark lines, that cut right through the random noise.

Yesterday I median-stacked 16 7D ISO 100 blackframes and subtracted them from one of them, and the result lost all deep high-frequency banding, completely, and most banding, in general. The same happened when I subtracted the median frame for a blackframe from October. IOW, most of the pattern noise in the 7D is completely fixed , and could have been calibrated out properly in the factory, or could even be done by firmware, if so programmed. Whatever attempts Canon is making at this, they are failing, miserably.

Without the fixed pattern noise, the 7D has one of the highest image-level DRs of Canon DSLRs at base ISO. Without the subtraction, I have seen the vertical stripes in the shadows of objects on the wall in well-exposed ISO 100 shots. The pixel std dev after fixed noise subtraction is as low as the 1D cameras (it's not much higher to begin with, but pattern noise, pound-for-pound, is much more visible than random noise).

I will give some comparative examples soon.

--
John

 
John Sheehy said:
agorabasta said:
Still more likely the higher number of pipelines is the culprit here. Not the speed per se, but the parallel processing is what's creating the problem.

Now if the 550 doesn't show the problem...
Click to expand...
The higher number of pipelines might account for some banding, but not what I am talking about. Carefully re-read what I wrote in this thread. The pipelines can not be responsible for this particular problem. They may be what the camera is trying to correct, but the camera corrects the problem incorrectly . The artifact of which I speak is the same (repeats) in all frames, but is still present in a camera subtraction, but not a subtraction from individual files. It, therefore, must be done by firmware. I am not talking about subtle subliminal banding. I am talking about distinct light and dark lines, that cut right through the random noise.

Yesterday I median-stacked 16 7D ISO 100 blackframes and subtracted them from one of them, and the result lost all deep high-frequency banding, completely, and most banding, in general. The same happened when I subtracted the median frame for a blackframe from October. IOW, most of the pattern noise in the 7D is completely fixed , and could have been calibrated out properly in the factory, or could even be done by firmware, if so programmed. Whatever attempts Canon is making at this, they are failing, miserably.

Without the fixed pattern noise, the 7D has one of the highest image-level DRs of Canon DSLRs at base ISO. Without the subtraction, I have seen the vertical stripes in the shadows of objects on the wall in well-exposed ISO 100 shots. The pixel std dev after fixed noise subtraction is as low as the 1D cameras (it's not much higher to begin with, but pattern noise, pound-for-pound, is much more visible than random noise).

I will give some comparative examples soon.
Click to expand...
That would be greatly appreciated.

Few other questions:

1) do you have access to > 1 7D and if so, is there a large amount of variability from camera to camera? If not, then perhaps there should be a web page with sets of blackframes that we can all use for subtraction. If you don't have access to > 1 7D, perhaps we all can contribute to testing out the variability across 7D samples.

2) Anyone know if there any easy way, say, for lightroom to do BF subtraction automatically?
John Sheehy said:
--
John

Click to expand...
 
John Sheehy said:
agorabasta said:
Still more likely the higher number of pipelines is the culprit here. Not the speed per se, but the parallel processing is what's creating the problem.

Now if the 550 doesn't show the problem...
Click to expand...
The higher number of pipelines might account for some banding, but not what I am talking about.
Click to expand...
Same to you :)

If the pipelines are 'too' independent, then it may be physically impossible to correct the issue in firmware. There may be no piece of hardware at an appropriate stage inside the cam to do the correction. Though I hope I'm wrong...

After all, the 7D really is the best cam Canon ever produced. In fact, it's the first Canon cam that produces an image that my over-sensitive colour perception can enjoy, rather than simply stand it...
 
agorabasta said:
John Sheehy said:
agorabasta said:
Still more likely the higher number of pipelines is the culprit here. Not the speed per se, but the parallel processing is what's creating the problem.

Now if the 550 doesn't show the problem...
Click to expand...
The higher number of pipelines might account for some banding, but not what I am talking about.
Click to expand...
Same to you :)

If the pipelines are 'too' independent, then it may be physically impossible to correct the issue in firmware. There may be no piece of hardware at an appropriate stage inside the cam to do the correction. Though I hope I'm wrong...
Click to expand...
Did you read what I wrote? The blackpoint offsets for vertical lines are the same in every frame. About 95% of the vertical banding noise energy is the same in every frame, from the day I bought the camera in October, until yesterday. That means that an array of less than 6000 3 or 4 bit numbers is all that is necessary to correct every image at that ISO, and maybe at others too (some elements of vertical banding may be gain-dependent, though).

--
John

 
John Sheehy said:
agorabasta said:
John Sheehy said:
agorabasta said:
Still more likely the higher number of pipelines is the culprit here. Not the speed per se, but the parallel processing is what's creating the problem.

Now if the 550 doesn't show the problem...
Click to expand...
The higher number of pipelines might account for some banding, but not what I am talking about.
Click to expand...
Same to you :)

If the pipelines are 'too' independent, then it may be physically impossible to correct the issue in firmware. There may be no piece of hardware at an appropriate stage inside the cam to do the correction. Though I hope I'm wrong...
Click to expand...
Did you read what I wrote? The blackpoint offsets for vertical lines are the same in every frame. About 95% of the vertical banding noise energy is the same in every frame, from the day I bought the camera in October, until yesterday. That means that an array of less than 6000 3 or 4 bit numbers is all that is necessary to correct every image at that ISO, and maybe at others too (some elements of vertical banding may be gain-dependent, though).

--
Click to expand...
Sure I do understand what you say. I also understand that, no matter how simple and easy to do it may seem, it may require a small additional piece of hardware that may be simply not present there in the cam!
 
agorabasta said:
Sure I do understand what you say. I also understand that, no matter how simple and easy to do it may seem, it may require a small additional piece of hardware that may be simply not present there in the cam!
Click to expand...
The camera's processor can read and write data, and add and subtract.

--
John

 
John Sheehy said:
agorabasta said:
Sure I do understand what you say. I also understand that, no matter how simple and easy to do it may seem, it may require a small additional piece of hardware that may be simply not present there in the cam!
Click to expand...
The camera's processor can read and write data, and add and subtract.
Click to expand...
The problem is there's more than one processor. With each doing it to its part of data, there may be no stage to correct the combined output.
As simple as that.
 
agorabasta said:
John Sheehy said:
agorabasta said:
Sure I do understand what you say. I also understand that, no matter how simple and easy to do it may seem, it may require a small additional piece of hardware that may be simply not present there in the cam!
Click to expand...
The camera's processor can read and write data, and add and subtract.
Click to expand...
The problem is there's more than one processor. With each doing it to its part of data, there may be no stage to correct the combined output.
As simple as that.
Click to expand...
No matter how you subdivide the responsibilities, the fact is, the problem is always the same, and correctable.

IT DOES NOT HAVE TO BE DONE DURING THE READOUT , but maybe it could.

--
John

 
John Sheehy said:
agorabasta said:
John Sheehy said:
agorabasta said:
Sure I do understand what you say. I also understand that, no matter how simple and easy to do it may seem, it may require a small additional piece of hardware that may be simply not present there in the cam!
Click to expand...
The camera's processor can read and write data, and add and subtract.
Click to expand...
The problem is there's more than one processor. With each doing it to its part of data, there may be no stage to correct the combined output.
As simple as that.
Click to expand...
No matter how you subdivide the responsibilities, the fact is, the problem is always the same, and correctable.

IT DOES NOT HAVE TO BE DONE DURING THE READOUT , but maybe it could.
Click to expand...
(no need shouting - I can do it even better)

Like I said, I hope it can be corrected with some FW trick. At the same time I completely understand that it may not be possible due to hardwiring of internal processing.

And I really don't give a damn about a problem that is mostly inexistent with proper choice of demosaicing - one can always use some latest dcraw version with 'VNG for four colors'; and I'd recommend the latest Ufraw with its default AHD - much better than it used to be.

Still your point that the issue is 'correctable' does not mean it's FW-correctable for in-cam processing. Chances are - it's not, as it's never got corrected for any recent multiproc cam from Canon.
 
agorabasta said:
And I really don't give a damn about a problem that is mostly inexistent with proper choice of demosaicing - one can always use some latest dcraw version with 'VNG for four colors'; and I'd recommend the latest Ufraw with its default AHD - much better than it used to be.
Click to expand...
Just as I suspected. You're in the wrong town. I wasn't discussing that problem at all.
agorabasta said:
Still your point that the issue is 'correctable' does not mean it's FW-correctable for in-cam processing. Chances are - it's not, as it's never got corrected for any recent multiproc cam from Canon.
Click to expand...
That would be evidence of nothing.

--
John

 
John Sheehy said:
agorabasta said:
And I really don't give a damn about a problem that is mostly inexistent with proper choice of demosaicing - one can always use some latest dcraw version with 'VNG for four colors'; and I'd recommend the latest Ufraw with its default AHD - much better than it used to be.
Click to expand...
Just as I suspected. You're in the wrong town. I wasn't discussing that problem at all.
John Sheehy said:
Still your point that the issue is 'correctable' does not mean it's FW-correctable for in-cam processing. Chances are - it's not, as it's never got corrected for any recent multiproc cam from Canon.
Click to expand...
That would be evidence of nothing.
Click to expand...
There is no need to argue. I think that John has developed an experiment that provides what I think is extremely strong evidence that the problem is indeed something that could easily be corrected by a bit of software in the FW. Why they didn't do this is anyone's guess, and anything we come up with can at best only be speculation.

I think this is a very valuable discovery. Now what would be equally valuable would be to figure out a way to easily deal with it. I.e., a "petition" to canon is unlikely to force them to come up with a FW patch where the problem will vanish. So, the best thing I think to do would be to come up with a way for 7D owners to be able to work the subtraction into their workflow (i.e., it is unreasonable to load every raw image into photoshop to do bf subtraction all the time, it just takes too much time when you've got thousands of images). If there was a simple step in lightroom that would be much easier (perhaps we could ask adobe to offer a subtraction operation in 3.0 or something). Whatever, the most useful thing now is to figure out how to come up with a time-saving way to deal with the problem.

Now John, what about those examples you mentioned above you'd provide? :-)
John Sheehy said:
--
John

Click to expand...
 
You must log in or register to reply here.

Keyboard shortcuts

f
Forum
m
My threads
Mobile site
About
Editorial content
Cameras & Lenses
Community
All content, design, and layout are Copyright © 1998–2025 Digital Photography Review All Rights Reserved.
Reproduction in whole or part in any form or medium without specific written permission is prohibited.
When you use DPReview links to buy products, the site may earn a commission.
©GPS Media - Guides, Products, Services.
Back
Top