The X3F Extractor Utility: Official Thread

Started Jul 10, 2008 | Discussions
Thermionic Forum Member • Posts: 63
The X3F Extractor Utility: Official Thread

Since the thread where much of the discussion on this utility has taken place has unfortunately met its 150-post-limit "demise", I felt the need to centralize the discussion on this important and extremely useful tool.

Pavel Sokolov's "X3F Sigma Raw File Unpacker" is a very useful tool for extracting the various data sections from Sigma X3F RAW files created by the SD series SLR cameras. It is available for download directly from Pavel's site at http://www.photofo.com

I have rehashed and combined my posts below for convenience, and hopefully to spark some larger discussion on the utility:

-- hide signature --

Tiffles has made a believer out of me!

On his advice, I downloaded Pavel Sokolov's X3F extractor tool and pulled the raw image out of an X3F file that I've been having trouble with in SPP. After using the tool, I am blown away by what appear to be significant flaws in the color gamut handling of the SPP software.

Some have speculated that the overactive red-channel behavior that occasionally occurs in very brightly lit scenes (especially certain flowers in direct sunlight) may be due to IR reflectivity on the subject that overcomes the simple IR-cut filter in the camera and overexposes the red layer in the Foveon sensor. That is certainly a matter for further discussion, but should be

Samples will follow...

Here is a crop from the original file, that is ridiculously and (in SPP) unrecoverably "overexposed" in the red channel. The flower is purely red and was lit by direct afternoon sunlight. Exposure was spot-metered directly on the brightest surface of the flower petal area. There is no orange whatsoever in the original scene... the odd additional orange coloration seems to be due to SPP's handling of color information that is outside the sRGB and AdobeRGB color spaces.

This second image is a crop from the same source image, extracted from RAW to 16-bit TIFF using the X3f Extractor tool. After extracting, I opened the file in Photoshop and used the Auto Curves function to brighten the image, and then adjusted the saturation and hue center to correct the color. It looks MUCH closer to the original scene:

Obviously it's a quick and dirty snappy photo to begin with, but these results are MUCH better than what is possible with SPP given the same source X3F. It's important to note that it is possible to capture the same shot without the red overexposure, but the overall exposure must be dialed down so far that the blue and green channels are significantly underexposed, which is not an ideal solution. The detail I was looking for IS being captured in the spot-metered shot, it's just not apparent due to the color space limitations of SPP in its current form.

Based on this, I feel comfortable saying that SPP is very likely the fundamental source of the problem... the RAW data simply doesn't have the kind of red-channel overexposure that I'm experiencing in images processed using the normal RAW tools. The camera works wonderfully, and thanks to Sigma for a fantastic design... the software though, not so much.

Obviously based on my images above, it is apparent that the camera is capable of capturing the visual information, and in fact does so very well... it's just that the range of color that the camera captures faithfully is well outside the limits of SPP's color space conversion capabilities.

I'll be contacting Sigma with the results of this experiment, and I'll try to see if there are plans for a new SPP that may fix some of these problems. I'll keep you all posted with regard to their response.

-------------------------------------------

More to come....

--
~ J.Porter ~

OP Thermionic Forum Member • Posts: 63
Re: The X3F Extractor Utility: Official Thread
-- hide signature --

~ BRIEF OVERVIEW OF THE X3F EXTRACTOR TOOL ~

The X3F Extractor can pull out any of the data sections of the X3f format separately.

The data sections are:

PROP Properties - Simple photo conditions data, similar to EXIF

CAMF Cam data - Extended information about the camera configuration, including information on the camera's calibration parameters.
IMA2 Small preview - 221x147 resolution bitmap preview for on-camera view
IMA2 RAW image - the full RAW image data ( saves as 16bit TIFF only )
IMA2 JPG preview - full res. 180dpi JPEG preview image (with EXIF)

There is a great deal of camera configuration information in the CAMF section, it is 38K in the raw file and exports as about 9K of text. The content of this file is the best evidence that I have seen that the SD14's behavior is able to be actively calibrated by Sigma on a per-camera basis. My camera is a "SD14F13_Rev3" and my calibration is a "SD14Cal Rel_17".

There is also a section in the CAMF file for offset correction of a series of adjustments, mine looks like this:

Tag name: AutoAdjustParams
OffsetExposure = 0.0
OffsetContrast = 0.0
OffsetShadow = 0.0
OffsetHighlight = 0.0
OffsetSaturation = 0.0
OffsetSharpness = 0.0
OffsetX3FillLight = 0.0
OffsetColorAdjustR = 0.0
OffsetColorAdjustG = 0.0
OffsetColorAdjustB = 0.0

You can see that mine are set to 0... I'm not sure if this is reflective of the settings from the camera menus or of correction settings that Sigma can program in to "calibrate" (correct) a camera's behavior.

There are also some tags in various places for very interestingly-named features like "Flare Factor", "RedSharpen", "MaxOutputLevel", "RawSaturationLevel", "CalTemperature", "SensorTemperature", "DBG_MaxRGB", and others. Very interesting.

------------------------------------------------
~ INTERFACE IN-DEPTH ~

The utility gives you the ability to output the separate data sections individually as needed. The text sections can be saved in any format, I use .TXT for simplicity.

The small image preview saves as a .BMP raster bitmap image, the RAW data saves as a 16bit TIFF, and the JPEG output saves as a full JPEG file that is (I would guess) identical to what the camera would capture in JPEG mode.
The main dialog looks like this:

When saving the 16bit TIFF from the RAW data section, the tool gives two additional options, which are:
"Apply Gamma to the RAW image"
"Apply Matrix to the RAW image"

Applying Gamma seems to have a fixed gamma effect, but Applying Matrix causes a dialog to pop up that gives some very significant adjustments. These have a dramatic effect on the TIFF output image.
The Apply Matrix dialog looks like this:

The "Default" function changes the values to exactly what you see in the image.

The "RT" function literally rotates the values around the outer cells in the matrix, in a clockwise fashion. The center value, at the "Raw Green" "Out Green" intersection, stays the same. I have absolutely no idea what this function is intended to accomplish.

The other two options ("Flip Vertical"? "Flip Horizontal"> ) are greyed out (disabled) for me.

-------------------------------------------

I hope this helps... anyone with questions or suggestions or more information, please participate!

Thanks,
--
~ J.Porter ~

SandyF Forum Pro • Posts: 14,941
link: JPEG extractor tools

here is a link to the two JPEG extractor tools many of us have been using for some time

http://forums.dpreview.com/forums/read.asp?forum=1027&message=27870885&q=extractor&qf=m

Best regards, Sandy
sfleischmann@gilder.com
http://www.pbase.com/sandyfleischman
http://www.flickr.com/photos/sandyfleischmann

 SandyF's gear list:SandyF's gear list
Sigma DP2 Sigma DP1 Sigma DP1 Merrill Sigma DP2 Merrill Sigma DP3 Merrill +7 more
Ed_S
Ed_S Veteran Member • Posts: 8,078
Re: The X3F Extractor Utility: Official Thread

J Porter,

Thanks for this compilation of your links and comments elsewhere. Also to Sandy for bringing the "JPEG Extractor" links across - different but also immensely useful, especially to those who may want quick electronic "proofs" or prefer the in-camera JPEG engine in some situations.

Question - forgive me if I've missed it - when I downloaded Pavel's tool the other day and made a couple of attempts on a similarly "overexposed" photo of red flowers, I found the image came out quite dark. Is there a "recommended" set of starting settings? When I tried to lighten it up after conversion to TIFF I couldn't quickly get a pleasing starting point.

Kind regards,
--
Ed_S
http://www.pbase.com/ecsquires

Hugh_T Regular Member • Posts: 227
Questions..

An interesting post!

I have a couple of questions which I hope you will clarify:

1. You state that this is an official thread - so I take it you have the necessary permission from Pavel to represent this product.

2. Your first sample is so incredibly bad that I find it hard to believe it is a good example of 'bad reds' - perhaps as a new Sigma user you should work more with the Sigma camera and SPP to get the best possible processing of your image. I have had similar issues with reds and have discussed this with Sigma since Feb this year and to be honest my very worst example is nowhere near as bad as your example.

3. Judging from your blog and your posts here it appears that you are on some sort of a mission to expose the Sigma bad reds - I think before you progress this too much you take the good advice of Gary Mercer and others in a previous thread on how to tackle photographing red subjects.

-- hide signature --

Regards

Hugh_T

alexzn
alexzn Senior Member • Posts: 1,752
Has anyone had made a LR preset that works for extracted 16-bit TIFF files

I would love to have my workflow be just batch-dumping the X3F into the 16-bit TIFFs that contain all the information and then having LR take over from that point on. Unfortunately SPP produces TIFFs that clearly do not have all the information captured in the RAW file.

What Sigma chose to do with Foveon X3F and SPP is a very bad idea- if you consciously restrict users to your proprietary RAW software, you better make sure that it is very very good, and a small company cannot possibly make tools that compete with moster efforts such as Photoshop, Lightroom or even Aperture.

SPP is a pain in the rear end underfeatured, underperforming and designed with very little attention to usability. I now switched to the Windows version, which has barely passable speed, the Mac version was a true unmitigated disaster.

Sigma/Foveon- please open your RAW format to outside developers, it will increase your user base, you would not have to pay money to people who write cr&ppy software for you, and your customers will produce better images. What's not to like?

Alex

Ed_S
Ed_S Veteran Member • Posts: 8,078
Re: Has anyone had made a LR preset that works for extracted 16-bit TIFF files

Alex,

I have no axe to grind with you but am honestly curious about parts of your - shall we say - "emphatic" post.

AlexZN wrote:

I would love to have my workflow be just batch-dumping the X3F into
the 16-bit TIFFs that contain all the information and then having LR
take over from that point on. Unfortunately SPP produces TIFFs that
clearly do not have all the information captured in the RAW file.

Can you be more specific? SPP (whether JPEG or TIFF option) seems to produce a converted image which is in most cases close to useable with no extensive user intervention required. What is the "missing information" that renders SPP converted files unusable for your purposes?

What Sigma chose to do with Foveon X3F and SPP is a very bad idea- if
you consciously restrict users to your proprietary RAW software, you
better make sure that it is very very good, and a small company
cannot possibly make tools that compete with moster efforts such as
Photoshop, Lightroom or even Aperture.

SPP is a pain in the rear end underfeatured, underperforming and
designed with very little attention to usability. I now switched to
the Windows version, which has barely passable speed, the Mac version
was a true unmitigated disaster.

I'm just not sure this is a universally held assessment or necessarily a "consensus" among Sigma users. IMHO (and seemingly among many users) the Windows version works for what it's apparently intended to do, be an X3F RAW converter. I'm open to the POV that its algorithms could benefit from improvement but I don't think most (including Sigma) intended it as a replacement for or competition with the full featured editing suites you mention. And I was under the impression that most MAC users (I'm not among the elite) were relatively satisfied with that version - guess I missread.

Sigma/Foveon- please open your RAW format to outside developers, it
will increase your user base, you would not have to pay money to
people who write cr&ppy software for you, and your customers will
produce better images. What's not to like?

Not sure what would constitute an "open" format by your definition. Clearly Adobe has access to the format so its RAW converter can be used to process the X3Fs into PS products. If you mean make it available unlicensed or "free" to the industry - well, I guess that's a business decision I'm not informed enough to debate. But even if X3F were available ubiquitously, would it be attractive to the various interests you mention for them to devote the development cost and support to a relatively niche application? Or is that in fact why they aren't doing it?

Just musings.

Kindest regards,
--
Ed_S
http://www.pbase.com/ecsquires

alexzn
alexzn Senior Member • Posts: 1,752
Re: Has anyone had made a LR preset that works for extracted 16-bit TIFF files

Ed_S wrote:

Alex,

I have no axe to grind with you but am honestly curious about parts
of your - shall we say - "emphatic" post.

emphatic? Was not intended to be that emphatic...

AlexZN wrote:

I would love to have my workflow be just batch-dumping the X3F into
the 16-bit TIFFs that contain all the information and then having LR
take over from that point on. Unfortunately SPP produces TIFFs that
clearly do not have all the information captured in the RAW file.

Can you be more specific? SPP (whether JPEG or TIFF option) seems to
produce a converted image which is in most cases close to useable
with no extensive user intervention required. What is the "missing
information" that renders SPP converted files unusable for your
purposes?

I can recover more highlight abd shadow information in SPP than I can when I batch-convert the X3Fs into 16-bit TIFFs and try to recover them in Lightroom. I am sure that Lightrom deals with 16bit files very well.

What Sigma chose to do with Foveon X3F and SPP is a very bad idea- if
you consciously restrict users to your proprietary RAW software, you
better make sure that it is very very good, and a small company
cannot possibly make tools that compete with moster efforts such as
Photoshop, Lightroom or even Aperture.

SPP is a pain in the rear end underfeatured, underperforming and
designed with very little attention to usability. I now switched to
the Windows version, which has barely passable speed, the Mac version
was a true unmitigated disaster.

I'm just not sure this is a universally held assessment or
necessarily a "consensus" among Sigma users. IMHO (and seemingly
among many users) the Windows version works for what it's apparently
intended to do, be an X3F RAW converter. I'm open to the POV that its
algorithms could benefit from improvement but I don't think most
(including Sigma) intended it as a replacement for or competition
with the full featured editing suites you mention. And I was under
the impression that most MAC users (I'm not among the elite) were
relatively satisfied with that version - guess I missread.

You misread. Mac version is known to be horrendously slow and buggy, and it crashes very often; search the forums for the user feedback. I have yet to see a person who likes the current SPP on a Mac. I gave up on it and now I run the Windows version through VMware.

Now, your comments about the "elite" puzzles me. Macs are nowdays price-competitive for what you get and on top of that you have a very well designed and well-behaved operating system that is better than Vista in almost every respect. I know many people who switched to the Mac from Windows, but very few who did the reverse. I have a mac at home and Win at work, so I am well-versed in both platforms. For me Mac is better machine hands-down. In any case, I don;t want to start the platform discussion, but it is true that Macs continue to be very popular in the image-processing universe, so having a good Mac version is probably a good idea.

Sigma/Foveon- please open your RAW format to outside developers, it
will increase your user base, you would not have to pay money to
people who write cr&ppy software for you, and your customers will
produce better images. What's not to like?

Not sure what would constitute an "open" format by your definition.
Clearly Adobe has access to the format so its RAW converter can be
used to process the X3Fs into PS products. If you mean make it
available unlicensed or "free" to the industry - well, I guess that's
a business decision I'm not informed enough to debate. But even if
X3F were available ubiquitously, would it be attractive to the
various interests you mention for them to devote the development cost
and support to a relatively niche application? Or is that in fact
why they aren't doing it?

The technical description in the link from the original post in this thread mentions (I am loosely quoting) that X3F is the only format that encrypts the data; Nikon also encrypts but only the white balance data. I remember reading somewhere that 3rd parties have to reverse-engineer the Foveon formats (I am not sure if that's completely true). But regardless of technical details, I actually don't see a problem in making the X3F format available free to anyone. No one else will make Foveon chips, that IP-protected. No one will make cameras with Foveon chips- that's governed by OEM and licensing agreement. The only thing that could happen is that more applications would start to speak Foveon, which would ease the acceptance of the technology by the markeptplace. I would say that it is a very good business decision, because it will remove obstacles for people for entering the Sigma/Foveon platform. Besides, Sigma does not have the resources to develop a competing product (exhibit A- SPP which has the functionality and user interface of a 10 year old RAW converter, it does not even crop the photos).

Here is an example of a different approach that takes care of a "niche product" problem. Ricoh decided not to produce their own RAW format and instead just used Adobe DNG, which instantaneously gave people who bought their cameras full compatibility with PS, LR, and Elements. Chances are that a person who cares about the RAW format uses one of those programs anyway. No one at the Ricoh forum has ever complained about RAW processing software (or at least hardly ever does), it is just not an issue.

The point that I keep making is that by refusing to leverage what the industry has done so far, Sigma is not helping the Foveon sensor get its due respects. hope that makes my position clear.

Alex

Just musings.

Kindest regards,
--
Ed_S
http://www.pbase.com/ecsquires

OP Thermionic Forum Member • Posts: 63
Re: Questions..

Hugh_T wrote:

1. You state that this is an official thread - so I take it you have
the necessary permission from Pavel to represent this product.

Hugh,

The "official thread" title is meant to be absolutely tongue-in-cheek, because the utility itself is so very "unofficial". Based on the way that Pavel himself has represented the utility in his posts, and has offered it openly for simple download, I believe that it is a significant stretch of language to refer to it as a "product". It is a user-developed, non-commercial utility that Pavel has made available to the larger Sigma owner community, and as such I feel it completely appropriate to discuss it in a central fashion in this forum. If the cheeky title is inappropriate, I am sure that Pavel himself will chime in, or a moderator will edit the title text appropriately.

I hope that Pavel will participate in this discussion, his hard work is greatly appreciated and he will certainly be able to provide insights into the color-related questions based on his intimate familiarity with the X3F file structure.

2. Your first sample is so incredibly bad that I find it hard to
believe it is a good example of 'bad reds' - perhaps as a new Sigma
user you should work more with the Sigma camera and SPP to get the
best possible processing of your image. I have had similar issues
with reds and have discussed this with Sigma since Feb this year and
to be honest my very worst example is nowhere near as bad as your
example.

To put it bluntly (but respectfully), I don't feel that I need a primer on how to "shoot reds", because it is patently obvious from my second image posted above that the problem is not with the camera, and is not with my exposure settings. The problem seems to be with the existing color space conversion algorithms used by the Sigma Photo Pro software.

I very intentionally used these two images as an example in order to show an absolute worst case scenario. Both of the images posted are from exactly the same X3F file. The sensor is working perfectly well to capture the scene in this case. I certainly did not post these images to invite a photographic critique! Rather, it was intended to spark some discussion about this tool, about the color space problems that SPP exhibits, and perhaps elicit some new information from other people that may want to chime in and elevate the conversation.

3. Judging from your blog and your posts here it appears that you are
on some sort of a mission to expose the Sigma bad reds - I think
before you progress this too much you take the good advice of Gary
Mercer and others in a previous thread on how to tackle photographing
red subjects.

I'm certainly not on any sort of mission. The blog entry that you're referring to was the complete text of the email that I sent to Sigma support at their request, detailing the color behavior that I was experiencing. I used the blog format to host the content to the web in a way that is convenient for my own use. There is no "mission" or anything of the sort at work, and nothing that I have posted is intended to be in any way negative. If you choose to interpret my statements that way, then it may be reflective of your own negative preconceptions, rather than my intent or the actual content of my statements.

Sigma's product works fantastically, and I believe if you'll read my above statements you'll see that I stated that. What isn't ideal is the behavior of the bundled Photo Pro software, which I believe is from a third party software developer anyway, and certainly is not reflective of Sigma's product engineering. I would like to see if we can figure out the root causes of the color space conversion problems, and perhaps assist Sigma Japan in solving them if we are in any way able to do so.

Please, let's keep the discussion on the topic at hand and try to avoid steering this thread into the land of negativity.

Thank you,
--
~ J.Porter ~

SandyF Forum Pro • Posts: 14,941
SPPx vs ACR

ACR4.1 and higher, thus any Photoshop software that supports ACR4.1 and higher, work perfectly fine with Sigma SD14 X3F RAW files. Honestly for 99% of my photos, I really see no difference in conversion in PS Elements 5/ACR4.4.1 or SPP2.4 or SPP2.5. In some trickier photos, one may have the advantage over the other (PSx vs SPPx). It's more a matter of workflow than any real differences in the RAW processor conversion.

Support should come for the DP1 X3F RAW files in the near future... is my assumption from various threads.

If you don't like SPP (I do actually) using some other converter is an easy option. Now for SD9/10/14s; coming for DP1s.
Best regards, Sandy
sfleischmann@gilder.com
http://www.pbase.com/sandyfleischman
http://www.flickr.com/photos/sandyfleischmann

 SandyF's gear list:SandyF's gear list
Sigma DP2 Sigma DP1 Sigma DP1 Merrill Sigma DP2 Merrill Sigma DP3 Merrill +7 more
OP Thermionic Forum Member • Posts: 63
Re: The X3F Extractor Utility: Official Thread

Ed_S wrote:

Question - forgive me if I've missed it - when I downloaded Pavel's
tool the other day and made a couple of attempts on a similarly
"overexposed" photo of red flowers, I found the image came out quite
dark. Is there a "recommended" set of starting settings? When I tried
to lighten it up after conversion to TIFF I couldn't quickly get a
pleasing starting point.

Ed,

You're absolutely right! They are very dark. As Tiffles suggested at one point in another thread, a good place to start with Photoshop is to use the "Auto" feature in the Curves dialog to brighten the overall exposure and channel values to a good starting point. This is only one of several ways to achieve a similar general exposure adjustment in Photoshop. Then an appropriate hue-center adjustment and saturation adjustment can be made to taste.

I believe that some of this process can be avoided or improved on by the use of adjustments to the "Matrix" feature in Pavel's X3F tool.

Thanks,
--
~ J.Porter ~

SandyF Forum Pro • Posts: 14,941
Re: Questions..

Thermionic wrote:
...

What isn't ideal is
the behavior of the bundled Photo Pro software, which I believe is
from a third party software developer anyway,

SPP is based on Foveon's work

and certainly is not
reflective of Sigma's product engineering.

I suggest you read JLK's posts on the SPP background

Best regards, Sandy
sfleischmann@gilder.com
http://www.pbase.com/sandyfleischman
http://www.flickr.com/photos/sandyfleischmann

added: for example http://forums.dpreview.com/forums/read.asp?forum=1027&message=28436134

 SandyF's gear list:SandyF's gear list
Sigma DP2 Sigma DP1 Sigma DP1 Merrill Sigma DP2 Merrill Sigma DP3 Merrill +7 more
Ed_S
Ed_S Veteran Member • Posts: 8,078
Re: Has anyone had made a LR preset that works for extracted 16-bit TIFF files

Alex,

Ok - fair enough - we see some things differently by degree.

Incidentally, the Mac "elite" was just in jest. I've spent much of my adult technological life in the shadow of Mac users who had the "real" computers. I guess we have read the Mac vs. PC impressions of SPP differently. I recall the days of earlier Windows releases when I think the PC users were in collective envy of how "well" the Mac release seemed to be working.

Kind regards,
--
Ed_S
http://www.pbase.com/ecsquires

Ed_S
Ed_S Veteran Member • Posts: 8,078
Re: The X3F Extractor Utility: Official Thread

J. Porter,

Whew - thought I was having yet another melt down of the synapses in the gray matter. That's helpful. I guess a bit of time spent experimenting might be in order. Obviously not a fully automated process but that's ok. Some of the best things in life aren't easy to do. If anyone happens to come up with a good set of defaults it would be nice to know.

Kind regards,
--
Ed_S
http://www.pbase.com/ecsquires

Pavel Sokolov
Pavel Sokolov Senior Member • Posts: 1,231
I do not object. Permission is gained :) [nt]
-- hide signature --
DaSigmaGuy Forum Pro • Posts: 12,303
Re: The X3F Extractor Utility: Official Thread

Thermionic wrote:

Since the thread where much of the discussion on this utility has
taken place has unfortunately met its 150-post-limit "demise", I felt
the need to centralize the discussion on this important and extremely
useful tool.

Pavel Sokolov's "X3F Sigma Raw File Unpacker" is a very useful tool
for extracting the various data sections from Sigma X3F RAW files
created by the SD series SLR cameras. It is available for download
directly from Pavel's site at http://www.photofo.com

I have rehashed and combined my posts below for convenience, and
hopefully to spark some larger discussion on the utility:

-- hide signature --

Tiffles has made a believer out of me!

On his advice, I downloaded Pavel Sokolov's X3F extractor tool and
pulled the raw image out of an X3F file that I've been having trouble
with in SPP. After using the tool, I am blown away by what appear to
be significant flaws in the color gamut handling of the SPP software.

Some have speculated that the overactive red-channel behavior that
occasionally occurs in very brightly lit scenes (especially certain
flowers in direct sunlight) may be due to IR reflectivity on the
subject that overcomes the simple IR-cut filter in the camera and
overexposes the red layer in the Foveon sensor. That is certainly a
matter for further discussion, but should be

Samples will follow...

Here is a crop from the original file, that is ridiculously and (in
SPP) unrecoverably "overexposed" in the red channel. The flower is
purely red and was lit by direct afternoon sunlight. Exposure was
spot-metered directly on the brightest surface of the flower petal
area. There is no orange whatsoever in the original scene... the odd
additional orange coloration seems to be due to SPP's handling of
color information that is outside the sRGB and AdobeRGB color spaces.

This second image is a crop from the same source image, extracted
from RAW to 16-bit TIFF using the X3f Extractor tool. After
extracting, I opened the file in Photoshop and used the Auto Curves
function to brighten the image, and then adjusted the saturation and
hue center to correct the color. It looks MUCH closer to the
original scene:

Obviously it's a quick and dirty snappy photo to begin with, but
these results are MUCH better than what is possible with SPP given
the same source X3F. It's important to note that it is possible to
capture the same shot without the red overexposure, but the overall
exposure must be dialed down so far that the blue and green channels
are significantly underexposed, which is not an ideal solution. The
detail I was looking for IS being captured in the spot-metered shot,
it's just not apparent due to the color space limitations of SPP in
its current form.

Based on this, I feel comfortable saying that SPP is very likely the
fundamental source of the problem... the RAW data simply doesn't have
the kind of red-channel overexposure that I'm experiencing in images
processed using the normal RAW tools. The camera works wonderfully,
and thanks to Sigma for a fantastic design... the software though,
not so much.

Obviously based on my images above, it is apparent that the camera
is capable of capturing the visual information, and in fact does so
very well... it's just that the range of color that the camera
captures faithfully is well outside the limits of SPP's color space
conversion capabilities.

Interesting, the after result looks very good...would it be possible for you to send me that RAW file so I could run some experiments myself?
--
DSG
--

-- hide signature --
 DaSigmaGuy's gear list:DaSigmaGuy's gear list
Fujifilm FinePix F660EXR Sigma SD10 Sigma SD14 Samsung Galaxy S7 +25 more
alexzn
alexzn Senior Member • Posts: 1,752
Re: SPPx vs ACR

I am aware that ACR reads XD files, but I am patiently waiting for the ACR to understand DP1 files... Then I will be happily using LR and my level of satisfaction with DP1 will increase tremendously (I still like the camera even now).

JUdged by the time it takes Adobe to come up with the plugin, Sigma is either not very cooperative, or Adobe just didn't get to it. Either way, that's not good for the DP1 users.

One more thought about RAW converters: these days people are likely to have cameras from different manufacturers (I certainly do), so having one RAW converter such as LR that reads everything makes a lot of sense. That way I don't have to learn two UIs, two sets of quirks, etc.

Alex

SandyF Forum Pro • Posts: 14,941
Re: SPPx vs ACR

are you following the posts about Adobe support? for example http://forums.dpreview.com/forums/read.asp?forum=1027&message=28488959

I agree it's nice to have my PSElements (I don't have expensive PS, just the

An alternative (if you don't like SPP) is just import TIFFs or JPEGs from the DP1 RAW, saved in the different format but not 'processed,' then work them in PSx to your normal workflow.

Best regards, Sandy
sfleischmann@gilder.com
http://www.pbase.com/sandyfleischman
http://www.flickr.com/photos/sandyfleischmann

 SandyF's gear list:SandyF's gear list
Sigma DP2 Sigma DP1 Sigma DP1 Merrill Sigma DP2 Merrill Sigma DP3 Merrill +7 more
Gary Dean Mercer Clark
Gary Dean Mercer Clark Veteran Member • Posts: 5,552
Re: The X3F Extractor Utility: Official Thread

J.

You should share your findings with Sigma USA and Sigma, Japan.

Thermionic wrote:

Since the thread where much of the discussion on this utility has
taken place has unfortunately met its 150-post-limit "demise", I felt
the need to centralize the discussion on this important and extremely
useful tool.

Pavel Sokolov's "X3F Sigma Raw File Unpacker" is a very useful tool
for extracting the various data sections from Sigma X3F RAW files
created by the SD series SLR cameras. It is available for download
directly from Pavel's site at http://www.photofo.com

I have rehashed and combined my posts below for convenience, and
hopefully to spark some larger discussion on the utility:

-- hide signature --

Tiffles has made a believer out of me!

On his advice, I downloaded Pavel Sokolov's X3F extractor tool and
pulled the raw image out of an X3F file that I've been having trouble
with in SPP. After using the tool, I am blown away by what appear to
be significant flaws in the color gamut handling of the SPP software.

Some have speculated that the overactive red-channel behavior that
occasionally occurs in very brightly lit scenes (especially certain
flowers in direct sunlight) may be due to IR reflectivity on the
subject that overcomes the simple IR-cut filter in the camera and
overexposes the red layer in the Foveon sensor. That is certainly a
matter for further discussion, but should be

Samples will follow...

Here is a crop from the original file, that is ridiculously and (in
SPP) unrecoverably "overexposed" in the red channel. The flower is
purely red and was lit by direct afternoon sunlight. Exposure was
spot-metered directly on the brightest surface of the flower petal
area. There is no orange whatsoever in the original scene... the odd
additional orange coloration seems to be due to SPP's handling of
color information that is outside the sRGB and AdobeRGB color spaces.

This second image is a crop from the same source image, extracted
from RAW to 16-bit TIFF using the X3f Extractor tool. After
extracting, I opened the file in Photoshop and used the Auto Curves
function to brighten the image, and then adjusted the saturation and
hue center to correct the color. It looks MUCH closer to the
original scene:

Obviously it's a quick and dirty snappy photo to begin with, but
these results are MUCH better than what is possible with SPP given
the same source X3F. It's important to note that it is possible to
capture the same shot without the red overexposure, but the overall
exposure must be dialed down so far that the blue and green channels
are significantly underexposed, which is not an ideal solution. The
detail I was looking for IS being captured in the spot-metered shot,
it's just not apparent due to the color space limitations of SPP in
its current form.

Based on this, I feel comfortable saying that SPP is very likely the
fundamental source of the problem... the RAW data simply doesn't have
the kind of red-channel overexposure that I'm experiencing in images
processed using the normal RAW tools. The camera works wonderfully,
and thanks to Sigma for a fantastic design... the software though,
not so much.

Obviously based on my images above, it is apparent that the camera
is capable of capturing the visual information, and in fact does so
very well... it's just that the range of color that the camera
captures faithfully is well outside the limits of SPP's color space
conversion capabilities.

I'll be contacting Sigma with the results of this experiment, and
I'll try to see if there are plans for a new SPP that may fix some of
these problems. I'll keep you all posted with regard to their
response.

-------------------------------------------

More to come....

--
~ J.Porter ~

-- hide signature --
 Gary Dean Mercer Clark's gear list:Gary Dean Mercer Clark's gear list
Canon EOS 5D Mark II Canon EOS 7D Sony a77 II Canon EF 50mm f/1.4 USM Canon EF 85mm f/1.8 USM +17 more
Tiffles Veteran Member • Posts: 4,972
Re: Has anyone had made a LR preset that works for extracted 16-bit TIFF files

I hope they pay Irident for making a really nice raw converter based on RD, with different and editable and bypassable matrixes.

O.
--
http://www.flickr.com/photos/ollivr/
http://www.flickriver.com/photos/ollivr/popular-interesting/
http://seen.by.spiegel.de/ollivr-1

 Tiffles's gear list:Tiffles's gear list
Sigma DP1 Sigma DP1 Merrill Kodak DCS Pro 14n Nikon D700 Sigma SD10 +6 more
Keyboard shortcuts:
FForum MMy threads