Joseph S. Wisniewski wrote:
[snip]
Not at all. It's a lot more than a bit of metadata. It's a question
of fundamental algorithms.
[snip]
Camera after camera, it's not just data, but special algorithms
required to deal with it.
I've snip that valuable insight into some of the problems of raw conversion.
In summary, raw conversion needs lots of algorithms, many (perhaps most) specific to subsets of camera models. Each of those algorithms needs its parameters, typically specific to the camera model. The task of a raw converter is to supply and run those algorithms. The task of a common raw format (CRF), if one exists, is to contain the required parameters, both to enable the raw converter to select the required algorithms, and to enable it to run the selected algorithms with the right camera parameters.
A CRF (common raw format) can't itself compensate for lack of the required algorithms. If a raw converter doesn't supply algorithms to cater for weak or zero AA filters, it doesn't matter what AA filter parameters are in the CRF. Ditto for algorithms to cater for offset sensor elements as in Fujifilm cameras. (And so on, including Foveon's special requirements). So the existence of a CRF can't (which an exception I'll come back to) enable a raw converter to handle subsets of cameras for which it doesn't supply the algorithms. That is one complaint about DNG: "why doesn't it cause this raw converter that doesn't handle original RAFs to handle them when they have been converted to DNG?" The answer is: "because using raw deliberately postpones a lot of work to the raw converter, and if the raw converter of choice can't do that work - tough!"
The exception is a non-default option in Adobe's DNG conversion code to convert to Linear DNG. This typically (not for Foveon for which different principles apply) means the demosaicing is done before the DNG file is written, so there are 3 or more channels. (Mostly 3, but the F828 has 4). This raises the question "doesn't that mean that whatever software handles the Linear DNG file can't possibly give better results than Adobe software?" Curiously, the answer is "no". I'll come back to that later.
Then there's the issue of color.
[snip]
What sort of color information does DNG contain? Did Adobe look at
how other raw converters do color, and make DNG capable of
satisfying every raw converter's needs? Did they ask other raw
converter writers (you know, like you do if you're really creating
an open standard, instead of just making a lot of noise about doing
it)? No. They simply gave DNG the kind of spectral coefficients
used by Adobe Camera Raw.
So, DNG isn't good for the consumer, it's good for Adobe. It
insures that no other raw decoder can ever outperform ACR. Is it
any wonder that so few are buying into it?
The people who convert to DNG, then use non-Adobe software because they believe the latter gives better results, would disagree with you! Some people believe that Silkypix can process their raw DNGs better than Adobe software can, and/or can handle their Linear DNGs from Foveon sensors better than Adobe can handle X3Fs. In other words, the specific camera details, or the demosaicing, or the initial conversion in the case of X3Fs, turns out to be only part of what makes photographers favour one or another raw converter, even for image quality rather than workflow.
So can DNG cater for all those extra details that some (not all) raw converters need? And can raw converters still work if they are not present?
There are parameters in the DNG specification that I have never seen used. For example AsShotPreProfileMatrix& AsShotPreProfileMatrix. AsShotPreProfileMatrix: "This tag contains an ICC profile that, in conjunction with the AsShotPreProfileMatrix tag, provides the camera manufacturer with a way to specify a default color rendering from camera color space coordinates (linear reference values) into the ICC profile connection space". Similarly, there is another pair, of which CurrentICCProfile says: "The CurrentICCProfile and CurrentPreProfileMatrix tags have the same purpose and usage as the AsShotICCProfile and AsShotPreProfileMatrix tag pair, except they are for use by raw file editors rather than camera manufacturers".
I don't know how far these would go to satisfy the raw converters that want more than the basic matrices that map between the camera color space coordinates and CIE XYZ coordinates, used by Adobe software.
Silkypix, (perhaps like some other raw converters?), uses the colour data from the DNG if it doesn't have any data of its own for that camera model. If it does have that data, it offers the user a choice of which to use. Personally, I don't like the default colour rendering of ACR, so I have used a calibration script using my GretagMacbeth ColorChecker to modify the ACR defaults. Other raw converters may provide their own camera calibration capability, and hence override the DNG/ACR defaults. Raw converters are not contrained to use only the basic DNG CIE XYZ transformation matrices.
In response to a recent question from me, Thomas Knoll responded:
"Yes, a new version of the DNG spec is planned, probably during the first half of 2007. Currently the only new tag Adobe is using is the ColorimetricReference tag, which is already included in the DNG SDK. It it is hint to the raw converter so it can select the correct default conversion parameters. (If the data is already in the ICC profile PCS, then the converter probably needs different default values, so it can skip the scene referred to output referred mapping normally required for raw data).
"We are currently talking to various developers who have some requested changes in the DNG specification".
http://adobe.groupbrowser.com/post907356-4.html 
