SDQH - Kalpanika X3F Extract purple image issues with DNG in LR & Other Software

Been using a DP2M for a while, using a flow with the x3f_extract tool ( X3F TOOLS VERSION = 0.57-windows-x86_64 )to bring the files into LR which has been working great.

Inside Darktable with message " 'Sigma SD Quattro H' color matrix not found"

Windows DNG preview

Upgraded to a SDQH this week, and the one issue that I can't find a good answer for is the images are purple.

Compressed, Uncompressed. With Wrapper without, tiff, DNG, with Adobe DNG converter, ect.

It looks like there is no camera color profile being included with the DNG (Or unreadable) As I can apply older .DNG profiles that improve it a little bit.

Has anyone else seen this? Is there a combination of flags for x3fextract that need to be used to get color information imbedded into the DNG properly?

Do I need to build a color profile from a Color Checker reference card? Is this a bug in the Kalpanika tool?

Thanks to whomever has a solution for this!

Reads very much like a WB issue, can you upload the DNG? Without the EXIF and DCP file, I can only guess as to what the issue is. Cheers.

P.S. why not shoot SOOC DNGs? For the chroma edge artefacts, running a 3x3 Median filter in CIELAB on the colour channels will help.

EEvan wrote:

Been using a DP2M for a while, using a flow with the x3f_extract tool ( X3F TOOLS VERSION = 0.57-windows-x86_64 )to bring the files into LR which has been working great.

Upgraded to a SDQH this week, and the one issue that I can't find a good answer for is the images are purple.

Compressed, Uncompressed. With Wrapper without, tiff, DNG, with Adobe DNG converter, ect.

It looks like there is no camera color profile being included with the DNG (Or unreadable) As I can apply older .DNG profiles that improve it a little bit.

Has anyone else seen this? Is there a combination of flags for x3fextract that need to be used to get color information imbedded into the DNG properly?

Do I need to build a color profile from a Color Checker reference card? Is this a bug in the Kalpanika tool?

Thanks to whomever has a solution for this!

Hi . I am not sure what you are trying to do. I have some question.

• Do the DNG file that the camera produce work fine?
• Do the OOC JPEG work fine?
• Is the first, very dark and grayish image, OK?

You can always go here and post a new issue: https://github.com/Kalpanika/x3f/issues

NOTE - the windows DNG preview will probably never work. It has probably not support for Linear RAW version of DNG, that we use. Almost no one, except Adobe has that.

Can you post a link to the DNG? There are a million and one ways to do it so, take your pick e.g. Dropbox

EEvan wrote:

The files are here: https://www.dropbox.com/sh/lwicgm3x41s2mhz/AAApcf0rx5KS0a2WWQAbjDpBa?dl=0

Occam's razor demands that I ask whether or not you used the "-unprocessed" tag?

Because your DNG reads:

<CalibrationIlluminant1>0</CalibrationIlluminant1>
<CalibrationIlluminant2>0</CalibrationIlluminant2>
<ColorMatrix1 Rows="0" Cols="0"/>
<ColorMatrix2 Rows="0" Cols="0"/>
<ForwardMatrix1 Rows="0" Cols="0"/>
<ForwardMatrix2 Rows="0" Cols="0"/>
<ReductionMatrix1 Rows="0" Cols="0"/>
<ReductionMatrix2 Rows="0" Cols="0"/>
<EmbedPolicy>0</EmbedPolicy>
<ProfileLookTableEncoding>0</ProfileLookTableEncoding>
<BaselineExposureOffset>0.000000</BaselineExposureOffset>
<DefaultBlackRender>0</DefaultBlackRender>

i.e. it contains zero information for colour conversion

Not sure what to suggest now... has the sd Q H received a firmware update recently?

I have tried to convert that file on my Mac, using x3f-extract 0.57.

Same result, the DNG file can be opened but is corrupted color-wise. The original X3F opens just fine in SPP however.

In verbose mode, x3f_extract outputs lots of messages which sound like warnings to me:

dbg: Matrix entry for LUT_RR_x1/Channel is out of order (index/1 != order/0)
dbg: Matrix entry for LUT_RR_x1/InputVal is out of order (index/0 != order/1)
dbg: Matrix entry for LUT_RR_x2/Channel is out of order (index/1 != order/0)
dbg: Matrix entry for LUT_RR_x2/InputVal is out of order (index/0 != order/1)
dbg: Matrix entry for LUT_RR_x4/Channel is out of order (index/1 != order/0)
dbg: Matrix entry for LUT_RR_x4/InputVal is out of order (index/0 != order/1)
dbg: Matrix entry for ColumnOffsets_HI_TOP/Channel is out of order (index/1 != order/0)
dbg: Matrix entry for ColumnOffsets_HI_TOP/InputVal is out of order (index/0 != order/1)
dbg: Matrix entry for ColumnOffsets_HI_BOTTOM/Channel is out of order (index/1 != order/0)

... (and so on)

EEvan wrote:

Been using a DP2M for a while, using a flow with the x3f_extract tool ( X3F TOOLS VERSION = 0.57-windows-x86_64 )to bring the files into LR which has been working great.

Inside Darktable with message " 'Sigma SD Quattro H' color matrix not found"

Windows DNG preview

Upgraded to a SDQH this week, and the one issue that I can't find a good answer for is the images are purple.

Compressed, Uncompressed. With Wrapper without, tiff, DNG, with Adobe DNG converter, ect.

It looks like there is no camera color profile being included with the DNG (Or unreadable) As I can apply older .DNG profiles that improve it a little bit.

Has anyone else seen this? Is there a combination of flags for x3fextract that need to be used to get color information imbedded into the DNG properly?

Do I need to build a color profile from a Color Checker reference card? Is this a bug in the Kalpanika tool?

Thanks to whomever has a solution for this!

my first DNG file out of a Quattro was quite weird as well - I suspect white balance issues.

Open the DNG in Raw Therapee. It has a wider latitude when dealing with color than Adobe products - and perhaps Darktable (I have little experience with Dark Table).

Other advise - always set the white balance manually.

Interesting to note the differences between the dp0 and sd Q H: 2048 vs 256 Black Point and no mention of Analog Gain for the sd Q H, rather Digital which is accepted as being detrimental to Dynamic Range

When I run it through my own dp0 tuned converter, it renders much the same as Kalpanika but I don't account for DigitalISOGain (B,G,R = 4,4,2).

N.B. the EXIF tag "Exposure Compensation : +255" points to it being a RAW White Balancing issue. I would guess that this would require new code in order for X3F_Extract to support it, it's not a bug.

The developer of Iridient Developer told me that he gave up on supporting X3Fs because SIGMA kept significantly moving the goal posts with each sensor generation... looks like they're doing it with firmware updates now too!

tagscuderia wrote:

Interesting to note the differences between the dp0 and sd Q H: 2048 vs 256 Black Point and no mention of Analog Gain for the sd Q H, rather Digital which is accepted as being detrimental to Dynamic Range

When I run it through my own dp0 tuned converter, it renders much the same as Kalpanika but I don't account for DigitalISOGain (B,G,R = 4,4,2).

N.B. the EXIF tag "Exposure Compensation : +255" points to it being a RAW White Balancing issue. I would guess that this would require new code in order for X3F_Extract to support it, it's not a bug.

The developer of Iridient Developer told me that he gave up on supporting X3Fs because SIGMA kept significantly moving the goal posts with each sensor generation... looks like they're doing it with firmware updates now too!

Yes, they are moving them all the time.

geni1105 wrote:

I have tried to convert that file on my Mac, using x3f-extract 0.57.

Same result, the DNG file can be opened but is corrupted color-wise. The original X3F opens just fine in SPP however.

In verbose mode, x3f_extract outputs lots of messages which sound like warnings to me:

dbg: Matrix entry for LUT_RR_x1/Channel is out of order (index/1 != order/0)
dbg: Matrix entry for LUT_RR_x1/InputVal is out of order (index/0 != order/1)
dbg: Matrix entry for LUT_RR_x2/Channel is out of order (index/1 != order/0)
dbg: Matrix entry for LUT_RR_x2/InputVal is out of order (index/0 != order/1)
dbg: Matrix entry for LUT_RR_x4/Channel is out of order (index/1 != order/0)
dbg: Matrix entry for LUT_RR_x4/InputVal is out of order (index/0 != order/1)
dbg: Matrix entry for ColumnOffsets_HI_TOP/Channel is out of order (index/1 != order/0)
dbg: Matrix entry for ColumnOffsets_HI_TOP/InputVal is out of order (index/0 != order/1)
dbg: Matrix entry for ColumnOffsets_HI_BOTTOM/Channel is out of order (index/1 != order/0)

... (and so on)

This is normal. We do not know what order those entries shall be in. But, so far, it is only entries we do not use.

tagscuderia wrote:

...

When I run it through my own dp0 tuned converter, it renders much the same as Kalpanika but I don't account for DigitalISOGain (B,G,R = 4,4,2).

N.B. the EXIF tag "Exposure Compensation : +255" points to it being a RAW White Balancing issue. I would guess that this would require new code in order for X3F_Extract to support it, it's not a bug.

Analog vs Digital, the good news is that the steps are the same and the maths is simple so it's easy to apply the WB correction — realise that that's little benefit if you want to use Kalpanika... I'm doing it by hand

Edit: doesn't appear to require clipping the RAW channels, so even easier than the dp Quattro conversion!

EEvan wrote:

Roland Karlsson wrote:

tagscuderia wrote:

Interesting to note the differences between the dp0 and sd Q H: 2048 vs 256 Black Point and no mention of Analog Gain for the sd Q H, rather Digital which is accepted as being detrimental to Dynamic Range

When I run it through my own dp0 tuned converter, it renders much the same as Kalpanika but I don't account for DigitalISOGain (B,G,R = 4,4,2).

N.B. the EXIF tag "Exposure Compensation : +255" points to it being a RAW White Balancing issue. I would guess that this would require new code in order for X3F_Extract to support it, it's not a bug.

The developer of Iridient Developer told me that he gave up on supporting X3Fs because SIGMA kept significantly moving the goal posts with each sensor generation... looks like they're doing it with firmware updates now too!

Yes, they are moving them all the time.

Interesting, so if it's a WB issue, is there anything that can easily be done on my side to resolve it?

Even if it's not a BUG, should I post this to the GitHub as an issue? Do you need any additional examples or images that would be helpful?

Thanks

You can post it as an issue to github. It is nice to have it documented there. Write as much as possible.

I shall be frank. We are a bit fed up with the problems of keeping our software compatible with the every whim of Sigma. We do put some efforts in it and we really do want it to work. But, sometimes it feels impossible.

EEvan wrote:

tagscuderia wrote:

tagscuderia wrote:

...

When I run it through my own dp0 tuned converter, it renders much the same as Kalpanika but I don't account for DigitalISOGain (B,G,R = 4,4,2).

N.B. the EXIF tag "Exposure Compensation : +255" points to it being a RAW White Balancing issue. I would guess that this would require new code in order for X3F_Extract to support it, it's not a bug.

Analog vs Digital, the good news is that the steps are the same and the maths is simple so it's easy to apply the WB correction — realise that that's little benefit if you want to use Kalpanika... I'm doing it by hand

Edit: doesn't appear to require clipping the RAW channels, so even easier than the dp Quattro conversion!

How are you doing it by hand?

I'm using ImageJ for the maths, colorMatrix for matrices and DCPs to apply LUTs and tone curves; all the info required is available in the EXIF and metadata. I have zero knowledge of coding but RAW development is surprisingly easy!

I wouldn't recommend it though, whilst SPP's Quattro conversion has steadily gotten worse in my eyes... it's incredible what they've achieved with the data — Quattro is a b*tch 

tagscuderia wrote:

EEvan wrote:

tagscuderia wrote:

tagscuderia wrote:

...

When I run it through my own dp0 tuned converter, it renders much the same as Kalpanika but I don't account for DigitalISOGain (B,G,R = 4,4,2).

N.B. the EXIF tag "Exposure Compensation : +255" points to it being a RAW White Balancing issue. I would guess that this would require new code in order for X3F_Extract to support it, it's not a bug.

Analog vs Digital, the good news is that the steps are the same and the maths is simple so it's easy to apply the WB correction — realise that that's little benefit if you want to use Kalpanika... I'm doing it by hand

Edit: doesn't appear to require clipping the RAW channels, so even easier than the dp Quattro conversion!

How are you doing it by hand?

I'm using ImageJ for the maths, colorMatrix for matrices and DCPs to apply LUTs and tone curves; all the info required is available in the EXIF and metadata. I have zero knowledge of coding but RAW development is surprisingly easy!

I wouldn't recommend it though, whilst SPP's Quattro conversion has steadily gotten worse in my eyes... it's incredible what they've achieved with the data — Quattro is a b*tch

I'm new to Quattro. What version of SPP would you recommend?

At this stage I don't think that there's much option or point in not using 6.5.4 (Win) / 6.5.5 (Mac). The sd Q H isn't compatible with the oldersoftware, nor is my dp0 after firmware updates

6.6 is a major release, maybe it will come with IQ improvements? Live in hope.