Raw image divided into a magenta band and a green band

Started Jun 29, 2013 | Discussions
Charles2 Veteran Member • Posts: 5,136
Raw image divided into a magenta band and a green band

The DP2M made a decent JPG. Here is a screen shot of the unedited X3F in SPP 5.5.1:

I presume there is no fix - but can we do anything to avoid this mess?

Sigma DP2
If you believe there are incorrect tags, please send us this post using our feedback form.
MOD Kendall Helmstetter Gelner Forum Pro • Posts: 20,063
Re: Raw image divided into a magenta band and a green band

Charles2 wrote:

The DP2M made a decent JPG. Here is a screen shot of the unedited X3F in SPP 5.5.1:

....

I've seen this in one image before also.  So far, as you say, there does not seem to be a way to recover it...

I think that it happened either because I was near the end of the battery, or I am also suspect of it being and SD card error.

Have you tried opening the image in Irident RawDeveloper, which supports the DP cameras?  I didn't think of that until just now myself...

If there's some way you could send me a link to the file, I'd also like to take a look and see if I can see any similarities between it and the one I had.

 Kendall Helmstetter Gelner's gear list:Kendall Helmstetter Gelner's gear list
Sigma 70-200mm F2.8 EX DG OS HSM Sigma 8-16mm F4.5-5.6 DC HSM Sigma 24-70mm F2.8 EX DG HSM Sigma 50-500mm F4.5-6.3 DG OS HSM Sigma 85mm F1.4 EX DG HSM +4 more
OP Charles2 Veteran Member • Posts: 5,136
Re: Raw image divided into a magenta band and a green band

You can download the X3F at

http://www4.zippyshare.com/v/34262031/file.html

Took a subsequent shot; no battery or card problem.

You do not know in the field that you have a useless raw file. If Sigma Corp. cannot fix the problem, can firmware test for it and show a warning "Raw file corrupted" on the LCD?

MOD Kendall Helmstetter Gelner Forum Pro • Posts: 20,063
Re: Raw image divided into a magenta band and a green band

Charles2 wrote:

You can download the X3F at

http://www4.zippyshare.com/v/34262031/file.html

Took a subsequent shot; no battery or card problem.

It's not like it's a regular thing - I just think it could be a reason why it happened.  So to prevent re-occurance one strategy would be to change out batteries before they are fully dead.

I've not seen it happen again though, despite no efforts to do that... but I did replace my SD card since then.

You do not know in the field that you have a useless raw file. If Sigma Corp. cannot fix the problem, can firmware test for it and show a warning "Raw file corrupted" on the LCD?

That would be nice, as you say you can't know on the spot there's an issue.

But I'm stil holding out hope there's some way to fix the file.

 Kendall Helmstetter Gelner's gear list:Kendall Helmstetter Gelner's gear list
Sigma 70-200mm F2.8 EX DG OS HSM Sigma 8-16mm F4.5-5.6 DC HSM Sigma 24-70mm F2.8 EX DG HSM Sigma 50-500mm F4.5-6.3 DG OS HSM Sigma 85mm F1.4 EX DG HSM +4 more
Laurence Matson
Laurence Matson Forum Pro • Posts: 11,959
Re: Raw image divided into a magenta band and a green band

Charles2 wrote:

The DP2M made a decent JPG. Here is a screen shot of the unedited X3F in SPP 5.5.1:

I presume there is no fix - but can we do anything to avoid this mess?

Having looked at it, I think the only two possibilities are a low battery or a bad card. Each of those problems can lead to a "spiking" effect, where a glitch triggers an incorrect response temporarily. If the battery were low, it could cause a sudden drop in power, which could cause this. A bad card - defect in a sector - could do the same thing.

The embedded jpeg, on the other hand, seems fine.

I would forward this file to Sigma Japan. They look at this stuff carefully, and can work on solutions going forward. For the time being, the jpeg can be processed in SPP to get a good result for further work in you image editing program.

-- hide signature --

Laurence
laurence at appledore-farm dot com
"I thought: I read something in a book, I dream, I imagine, and it comes true. And it is exactly like this in life.
"You can dream, and it comes true, as long as you can get out of the certitudes. As long as you can get a pioneering spirit, as long as you can explore, as long as you can think off the grid. So much time we spend in our education, in our lives is spent learning certitudes, learning habits, trying to fight against the unknown, to avoid the doubts and question marks. As soon as you start to love the unknown, to love the doubts, to love the question marks, life becomes an absolutely fabulous adventure."
Bertrand Piccard, a Swiss person
http://www.pbase.com/lmatson
http://www.pbase.com/sigmadslr
http://www.howardmyerslaw.com

OP Charles2 Veteran Member • Posts: 5,136
Re: Raw image divided into a magenta band and a green band
1

Laurence Matson wrote:

Having looked at it, I think the only two possibilities are a low battery or a bad card. ...

the battery indicator showed no problem - one bar as I recall. Now we need to second guess the indicator? The card has shown no problem.

For the time being, the jpeg can be processed in SPP to get a good result for further work in you image editing program.

Not an important shot. Instead of using the JPG, I will start a new craze called DuoTint, another area where Sigma Foveon cameras dominate. Aficionados will debate where the line dividing the magenta overlay from the green overlay should go, and which color belongs on top for each particular shot.

MOD Kendall Helmstetter Gelner Forum Pro • Posts: 20,063
Re: Raw image divided into a magenta band and a green band

Charles2 wrote:

Laurence Matson wrote:

Having looked at it, I think the only two possibilities are a low battery or a bad card. ...

the battery indicator showed no problem - one bar as I recall. Now we need to second guess the indicator? The card has shown no problem.

The thing about flash cards is that they might show no problems generally but have some flaws. If you encounter any kind of error like this is usually better to get a new one and treat that as a spare.

You could also see if there are some applications that might test the SD card for write errors and map out bad sectors.

It seems like modern day SD cards, even high-end ones are much more badly made than they used to be.  I had a Duracell card have the casing totally fall apart which made me have to tweeze out bits of the case from the camera.  I've also had *multiple* Sandisk cards die on me in suspicious ways, from writes generally failing suddenly to the whole card going utterly unreadable overnight.  That's why I'm more suspicious of a card issue than a battery issue.

As for the battery, with one bar left the battery ranges from somewhat full to pretty much empty.

In general I wouldn't worry about it and treat it as a fluke, like I said I saw it happen once - but that was over the span of many thousands of images and I've not seen it happen again. If you do see it happen again then I'd say it would be time to figure out more of a process to prevent the issue.

Not an important shot. Instead of using the JPG, I will start a new craze called DuoTint, another area where Sigma Foveon cameras dominate. Aficionados will debate where the line dividing the magenta overlay from the green overlay should go, and which color belongs on top for each particular shot.

Well it did obey the rule of thirds so you have that going for you.

 Kendall Helmstetter Gelner's gear list:Kendall Helmstetter Gelner's gear list
Sigma 70-200mm F2.8 EX DG OS HSM Sigma 8-16mm F4.5-5.6 DC HSM Sigma 24-70mm F2.8 EX DG HSM Sigma 50-500mm F4.5-6.3 DG OS HSM Sigma 85mm F1.4 EX DG HSM +4 more
Johan Borg Senior Member • Posts: 2,555
Re: Raw image divided into a magenta band and a green band

Maybe not totally scientific, but I had more or less the same happen to me a few times with a (rather expensive) SD card, changed to a new card and never experienced it again...

lawadd New Member • Posts: 9
Re: Raw image divided into a magenta band and a green band

Had the same problem with several files on the SD15.  Tested the SD cards on Bayer sensor cameras.  Not an issue with SD.  Might be bad sectors in the HDD on the computer.

MOD Kendall Helmstetter Gelner Forum Pro • Posts: 20,063
Re: Raw image divided into a magenta band and a green band

lawadd wrote:

Had the same problem with several files on the SD15. Tested the SD cards on Bayer sensor cameras. Not an issue with SD. Might be bad sectors in the HDD on the computer.

The problem is that SD cards wig out in strange ways, and sometimes just do not work well on a particular device.

I had once SD card that would work in the camera just fine - but not on the computer at all!!  That should be impossible, there's no way it should happen.  But happen it did.

So any oddity now (like broken files) I just get a new SD card.  It makes life a lot simpler and usually I don't have problems again (until the SD card randomly dies).

 Kendall Helmstetter Gelner's gear list:Kendall Helmstetter Gelner's gear list
Sigma 70-200mm F2.8 EX DG OS HSM Sigma 8-16mm F4.5-5.6 DC HSM Sigma 24-70mm F2.8 EX DG HSM Sigma 50-500mm F4.5-6.3 DG OS HSM Sigma 85mm F1.4 EX DG HSM +4 more
victorgv Senior Member • Posts: 1,554
Re: Raw image divided into a magenta band and a green band

Kendall Helmstetter Gelner wrote:

lawadd wrote:

Had the same problem with several files on the SD15. Tested the SD cards on Bayer sensor cameras. Not an issue with SD. Might be bad sectors in the HDD on the computer.

The problem is that SD cards wig out in strange ways, and sometimes just do not work well on a particular device.

I had once SD card that would work in the camera just fine - but not on the computer at all!! That should be impossible, there's no way it should happen. But happen it did.

So any oddity now (like broken files) I just get a new SD card. It makes life a lot simpler and usually I don't have problems again (until the SD card randomly dies).

Happens quite offen with SD cards .

 victorgv's gear list:victorgv's gear list
Sigma DP1 Sigma DP2 Merrill Sigma dp2 Quattro
Davethepom
Davethepom Regular Member • Posts: 390
Coincidence?

This happened to me for the first time last weekend. 5 of 78 shots all exhibited this exact issue. I noted thast the jpeg's were unaffected, but the RAW image (.X3F) was a big issue. I ended up converting them to monochrome and just using the "Blue" channel, all fixed

Maybe it was the "supermoon" having cause & effect?

What I didn't do. and will try tonight, is using Iridient to process instead of SPP.

Rgds Pom

OP Charles2 Veteran Member • Posts: 5,136
Re: Coincidence?

Davethepom wrote:

This happened to me for the first time last weekend. 5 of 78 shots all exhibited this exact issue. I noted that the jpeg's were unaffected, but the RAW image (.X3F) was a big issue....

Five of 78!

The fact that JPG is fine suggests that the problem is not battery charge nor SD card. (I ran Windows chkdsk on my card and my hard drive; no problem.) The constant pattern of failure - magenta and green divided by a horizontal line - strongly suggests a trouble point in the firmware processing that writes the strictly raw portion of the X3F file.

The questions for Sigma Corp.: what is this error, and is it initiated by a characteristic of the data of a shot or by something in the camera environment, such as the user pressing a certain button at a precise moment while the preceding shot is still being processed?

Were those five shots together in sequence or scattered over the 78?

Davethepom
Davethepom Regular Member • Posts: 390
Re: Coincidence?

No, they were "scattered"; though they were towards the end of the shoot rather than the start (maybe battery or disk write issues).

Interestingly, the only one that I have left as a native .X3F (which still exhibits the issue) will not open with Iridient Developer. It simply crashes the application.

SPP (5.5.2)

Iridient Developer (2.1.1)

Unfortunately, I'm using a mac, and hence disk warrior wont do a consistency check on a none mac volume. "disk utility" verified the CF card, no problems.

Rgds Pom

Roland Karlsson Forum Pro • Posts: 27,318
Re: I doubt its the SD card

If it was faulty SD cards or maybe Sigma hardware being incompatible with several SD cards, then you would get lots of strange problems. Files should be totally unreadable. And there should be lots of JPEG problem, as JPEG is very sensitive to small faults in the file.

Now its only those starnge bandings in RAW, which hints at the RAW file creating code has a bug. Its probably the delta encoding that is faulty.

One question - those files with colored sections, are they smaller than usual? I.e. could parts of one color be missing in the file?

NOTE: that even if parts of one color is missing - the index data table is written at the end of the files. So - if the card is faulty and the file is truncated, then you should get no image at all.

So, if one color is truncated - then this has to be under software control, somehow. Otherwise, there would be no index data at the end.

 Roland Karlsson's gear list:Roland Karlsson's gear list
Sigma DP3 Merrill Sigma dp2 Quattro Sony RX100 III Pentax K-3 Pentax K-1 +14 more
victorgv Senior Member • Posts: 1,554
Re: Coincidence?

Davethepom wrote:

No, they were "scattered"; though they were towards the end of the shoot rather than the start (maybe battery or disk write issues).

Interestingly, the only one that I have left as a native .X3F (which still exhibits the issue) will not open with Iridient Developer. It simply crashes the application.

SPP (5.5.2)

Iridient Developer (2.1.1)

Unfortunately, I'm using a mac, and hence disk warrior wont do a consistency check on a none mac volume. "disk utility" verified the CF card, no problems.

Rgds Pom

If it is more towards the end i would strongly suspect SD card not that it is bad per se nut in ombination with camera it might work not to well.

 victorgv's gear list:victorgv's gear list
Sigma DP1 Sigma DP2 Merrill Sigma dp2 Quattro
Nancy and Pete Spader Senior Member • Posts: 2,050
I suspect the Battery

I suspect it is the battery. As Kendall pointed out, with only one bar you may be very close to empty but it is hard to tell. I have never had this particular problem but when I do have file problems it is usually because my battery is very close to empty. In my case I usually just lose the file(s) and then the camera stops working. When we put in aa charged battery things are back to normal.
 What is odd is sometimes I will lose 2 or three files, not just one.

The solution I use is that as soon as I see the last bar I put in a new charged battery as soon as possible just to avoid problems. I always start a shoot with a fully charged battery as well. The way we use our cameras we do get very good battery performance so this does not require changing batteries a lot, just keeping track of battery discharge.

Nancy sometimes loses shots if she tries to take too many shots at once with the SD14. We notice the SD1M will not let you do that, so it is no longer a problem.

I would also suggest sending this file to Sigma, as Kendall mentioned. They do take these things seriously and perhaps they can do something about it. Just making the bar system of reporting battery charge more reliable would be a help. I notice that the SD1M manual says that either two or one bar means the battery is low and will need to be changed soon. This means anything but all three bars puts you into the warning zone. It would make more sense if they could set it up so that there is a difference between 2 bars and 1 bar. Given the size of the files with the Merrill cameras this is even more important now.

Best of luck avoiding the problem in the future.

Pete

OP Charles2 Veteran Member • Posts: 5,136
Re: I doubt its the SD card

Roland Karlsson wrote:

One question - those files with colored sections, are they smaller than usual? I.e. could parts of one color be missing in the file?

No, the file size is in typical DP2M range, 45MB.

I didn't find an email address for Sigma Japan, but I did email Sigma USA support with the link to the X3F file posted above. I hope that was not an unreasonable step for a U.S. resident to take.

ArvoJ Senior Member • Posts: 1,367
Re: I doubt its the SD card

Roland Karlsson wrote:

If it was faulty SD cards or maybe Sigma hardware being incompatible with several SD cards, then you would get lots of strange problems. Files should be totally unreadable. And there should be lots of JPEG problem, as JPEG is very sensitive to small faults in the file.

Now its only those starnge bandings in RAW, which hints at the RAW file creating code has a bug. Its probably the delta encoding that is faulty.

One question - those files with colored sections, are they smaller than usual? I.e. could parts of one color be missing in the file?

NOTE: that even if parts of one color is missing - the index data table is written at the end of the files. So - if the card is faulty and the file is truncated, then you should get no image at all.

So, if one color is truncated - then this has to be under software control, somehow. Otherwise, there would be no index data at the end.

It most likely is card writing failure after all. RAW data writing should be sequential operation; if single bit goes wrong, everything after that is corrupted. In jpg single bit usually doesn't corrupt entire image, although may create visible corrupted square or something similar. X3F (newer) format has unfortunately very low fault tolerance.

Another possibility is some (buffer) RAM problem - writing-reading huge amount of data may corrupt some bits. This should affect more images IMO, but maybe these affected ones are at specific buffer locations or so.

Of course other hardwre or algorithmical problems are possible either. Only what you can do, is to try some other brand card - if problems continue, then probably card is not related; if problems won't occur anymore (or there are alot of more problems), then it is most likely card incompatbility.

-- hide signature --

Arvo
Sigma/Foveon information collection and little galleries:
http://www.stv.ee/~donq/sigma.htm
http://www.stv.ee/~donq/images.htm

 ArvoJ's gear list:ArvoJ's gear list
Sigma SD14 Sigma 10-20mm F4-5.6 EX DC HSM Sigma 105mm F2.8 EX DG Macro +1 more
MOD Kendall Helmstetter Gelner Forum Pro • Posts: 20,063
Chkdsk or something else?

Charles2 wrote:

Davethepom wrote:

This happened to me for the first time last weekend. 5 of 78 shots all exhibited this exact issue. I noted that the jpeg's were unaffected, but the RAW image (.X3F) was a big issue....

Five of 78!

The fact that JPG is fine suggests that the problem is not battery charge nor SD card. (I ran Windows chkdsk on my card and my hard drive; no problem.)

Chkdsk only checks the filesystem, not the physical state of the media, unless you run chkdsk /r (which I'm not sure you can do on a Windows system outside of recovery mode).

What you really want is something like "scandisk" I think (not sure, have not run Windows for a while) - basically something that actually tries to write and read to every sector on the card.

The constant pattern of failure - magenta and green divided by a horizontal line - strongly suggests a trouble point in the firmware processing that writes the strictly raw portion of the X3F file.

But it also hints at a way that SPP may handle arbitrary errors in the RAW data block of a file.

In my images the effect was the same but the location of the stripe was different.

The questions for Sigma Corp.: what is this error, and is it initiated by a characteristic of the data of a shot or by something in the camera environment, such as the user pressing a certain button at a precise moment while the preceding shot is still being processed?

Were those five shots together in sequence or scattered over the 78?

I personally think it's a combination of the camera interacting with an SD card that in some way introduces those errors with very low probability.

 Kendall Helmstetter Gelner's gear list:Kendall Helmstetter Gelner's gear list
Sigma 70-200mm F2.8 EX DG OS HSM Sigma 8-16mm F4.5-5.6 DC HSM Sigma 24-70mm F2.8 EX DG HSM Sigma 50-500mm F4.5-6.3 DG OS HSM Sigma 85mm F1.4 EX DG HSM +4 more
Keyboard shortcuts:
FForum MMy threads