What is the embedded profile in a DNG actually used for?

[Thomas001le]

I've been looking into DXO PureRaw and noticed that the DNG files generated by DXO PureRAW sometimes have a different camera profile embedded (i.e. Color Matrix, Forward Matrix, Camera Calibration matrix, .. you name it) than the Adobe DNG Converter would attach to a DNG file generated for the same camera. This has the annoying effect that white balance values (temperature/tint) in lightroom are different and not directly tansferrable.

If white balance is actual set to the same spot white balance, there doesn't seem to be a difference in color rendering anymore even though literal white balance values are different.

I played around with this a bit and copying color profile from Adobe DNG Converter DNGs to DXO PureRAW generated DNGs actually works. The white balance values now match exactly and color rendering is the same. This could indicate that the linear camera values actually use the same color space for DXO PureRAW and the Adobe DNG Converter.

I looked over the DNG specifications a bit and looked at the tags in both DNG files. The AsShot white balance is embedded in camera color space and not as XY coordinates. To get the XY coordinates, and thus the color temperature, you need to apply the camera profile. Thus it makes sense that the color temperatures are different even for the same camea neutral white point if the camera profiles are different.

Now, Adobe Lightroom comes with its own set of camera profiles in DCP files (I also have a few imported ones). To my understanding DCP profiles basically contain the same camera profile matrices that would are in a DNG file.

Now I got lost a bit and have questions

1. Does Lightroom actually use the embedded camera profile in a DNG file if it finds a DCP profile for the camera?

2. Is the embedded DNG profile still used to do the convertion from the camera white point (in camera color space) to XY/CCT?

3. Shouldn't the "AsShot" white point change when a different camera profile is selected since the white point in camera color space is now translated to XY differently?

 

[Jack Hogan]

1. Does Lightroom actually use the embedded camera profile in a DNG file if it finds a DCP profile for the camera?

Depending on settings, it can. Or you can select it manually

2. Is the embedded DNG profile still used to do the convertion from the camera white point (in camera color space) to XY/CCT?

This is your issue. The conversion to CCT and back to control white balance is done using so-called CM reverse matrices. If your profile uses the same ones that Adobe uses*, you should get the same CCT results. If not the CCT reading will be different though no worse. Thinking aloud this should not necessarily result in incorrect tones.

3. Shouldn't the "AsShot" white point change when a different camera profile is selected since the white point in camera color space is now translated to XY differently?

No because the as shot CameraNeutral is simply the WB multipliers that the camera estimated at the time of capture. Those, in combination with CMs, are what is used to estimate the CCT. If the CMs are different, so will the CCT.

DcamProf (Lumariver) estimates its own 'better' CMs based on a known target - but gives you the option of using Adobe CMs when creating a custom dng profile so that this issue will not present itself.

Jack

* They are the ones part of the dcp profile when Adobe DNG converter is used.

https://www.strollswithmydog.com/white-point-cct-tint/

 

[Dave_J_E]

I've been looking into DXO PureRaw and noticed that the DNG files generated by DXO PureRAW sometimes have a different camera profile embedded (i.e. Color Matrix, Forward Matrix, Camera Calibration matrix, .. you name it) than the Adobe DNG Converter would attach to a DNG file generated for the same camera. This has the annoying effect that white balance values (temperature/tint) in lightroom are different and not directly tansferrable.

If white balance is actual set to the same spot white balance, there doesn't seem to be a difference in color rendering anymore even though literal white balance values are different.

I played around with this a bit and copying color profile from Adobe DNG Converter DNGs to DXO PureRAW generated DNGs actually works. The white balance values now match exactly and color rendering is the same. This could indicate that the linear camera values actually use the same color space for DXO PureRAW and the Adobe DNG Converter.

I looked over the DNG specifications a bit and looked at the tags in both DNG files. The AsShot white balance is embedded in camera color space and not as XY coordinates. To get the XY coordinates, and thus the color temperature, you need to apply the camera profile. Thus it makes sense that the color temperatures are different even for the same camea neutral white point if the camera profiles are different.

Now, Adobe Lightroom comes with its own set of camera profiles in DCP files (I also have a few imported ones). To my understanding DCP profiles basically contain the same camera profile matrices that would are in a DNG file.

Now I got lost a bit and have questions

1. Does Lightroom actually use the embedded camera profile in a DNG file if it finds a DCP profile for the camera?

From what I've seen it loads the profile embedded in the DNG file by default but other profiles can be chosen as for any raw file. At least this is what happens with the dng files from my iPhone.

Also note that the Adobe DNG Converter embeds the Adobe Standard profile (for the camera identified in the raw file) in the dng file.

2. Is the embedded DNG profile still used to do the convertion from the camera white point (in camera color space) to XY/CCT?

As Jack mentions, the CM's contained in the color profile are used by Adobe to convert to CCT and Tint. So in LR/ACR the CM's used will be the ones in the Color Profile selected, whether it be the one in the DNG or a DCP.

Note that Adobe uses dual illuminant profiles, I don't know what DXO does. This could lead to differences in the CCT calculation. Also the method used for tint calculation are not standardised (in relation to scale factor) so different software vendors may use different approaches.

It's quite common to get variations in calculated CCT and Tint across different software, depending on methods used.

3. Shouldn't the "AsShot" white point change when a different camera profile is selected since the white point in camera color space is now translated to XY differently?

What Jack said.

Dave

 

[brucebanner1122]

The embedded profile in a DNG (Digital Negative) file is primarily used for color interpretation and rendering consistency across different software and devices. This profile contains color calibration data that ensures the raw image is displayed accurately, preserving the intended color balance, tone curves, and white balance adjustments. Camera manufacturers or software may embed a custom profile tailored to the specific sensor's characteristics, helping achieve more faithful color reproduction when the file is processed. Additionally, some editing programs use this embedded profile as a reference when applying default or custom color corrections. While many raw editors allow users to override this profile with their own, it serves as an essential starting point for accurate image processing.

 

[ggbutcher]

I've been looking into DXO PureRaw and noticed that the DNG files generated by DXO PureRAW sometimes have a different camera profile embedded (i.e. Color Matrix, Forward Matrix, Camera Calibration matrix, .. you name it) than the Adobe DNG Converter would attach to a DNG file generated for the same camera. This has the annoying effect that white balance values (temperature/tint) in lightroom are different and not directly tansferrable.

If white balance is actual set to the same spot white balance, there doesn't seem to be a difference in color rendering anymore even though literal white balance values are different.

I played around with this a bit and copying color profile from Adobe DNG Converter DNGs to DXO PureRAW generated DNGs actually works. The white balance values now match exactly and color rendering is the same. This could indicate that the linear camera values actually use the same color space for DXO PureRAW and the Adobe DNG Converter.

I looked over the DNG specifications a bit and looked at the tags in both DNG files. The AsShot white balance is embedded in camera color space and not as XY coordinates. To get the XY coordinates, and thus the color temperature, you need to apply the camera profile. Thus it makes sense that the color temperatures are different even for the same camea neutral white point if the camera profiles are different.

Now, Adobe Lightroom comes with its own set of camera profiles in DCP files (I also have a few imported ones). To my understanding DCP profiles basically contain the same camera profile matrices that would are in a DNG file.

Now I got lost a bit and have questions

1. Does Lightroom actually use the embedded camera profile in a DNG file if it finds a DCP profile for the camera?

2. Is the embedded DNG profile still used to do the convertion from the camera white point (in camera color space) to XY/CCT?

3. Shouldn't the "AsShot" white point change when a different camera profile is selected since the white point in camera color space is now translated to XY differently?

Not an answer to your question, but I've found this description of the Adobe workflow and DCP/DNGs to be useful:

https://rawtherapee.com/mirror/dcamprof/camera-profiling.html#dng_profiles

Just one section of a much larger treatise on camera profiling, also quite useful. Written by the author of Lumariver.

 

[Thomas001le]

1. Does Lightroom actually use the embedded camera profile in a DNG file if it finds a DCP profile for the camera?

Depending on settings, it can. Or you can select it manually

2. Is the embedded DNG profile still used to do the convertion from the camera white point (in camera color space) to XY/CCT?

This is your issue. The conversion to CCT and back to control white balance is done using so-called CM reverse matrices. If your profile uses the same ones that Adobe uses*, you should get the same CCT results. If not the CCT reading will be different though no worse. Thinking aloud this should not necessarily result in incorrect tones.

3. Shouldn't the "AsShot" white point change when a different camera profile is selected since the white point in camera color space is now translated to XY differently?

No because the as shot CameraNeutral is simply the WB multipliers that the camera estimated at the time of capture. Those, in combination with CMs, are what is used to estimate the CCT. If the CMs are different, so will the CCT.

I think I meant something else. I import a image into Lightroom and lightroom estimates the CCT to be let's say 6000K based on the embeded white point in camera color space and based on the embedded CM matrices.

Now, I select a different dcp profile when developing the image. This profile has a different set of CM matrices. Suddenly 6000K does not map to the as shot camera neutral white point anymore. The image might get a color tint. If the CMs from the new dcp profile would have been used the camera neutral white point would translate to let's say 6200K, but lightroom does not update that.

That means it also behaves differently whether a profile was embedded or selected afterwards here?

Does that make sense or am I missing something here?

DcamProf (Lumariver) estimates its own 'better' CMs based on a known target - but gives you the option of using Adobe CMs when creating a custom dng profile so that this issue will not present itself.

Jack

* They are the ones part of the dcp profile when Adobe DNG converter is used.

https://www.strollswithmydog.com/white-point-cct-tint/

 

[Dave_J_E]

I think I meant something else. I import a image into Lightroom and lightroom estimates the CCT to be let's say 6000K based on the embeded white point in camera color space and based on the embedded CM matrices.

Now, I select a different dcp profile when developing the image. This profile has a different set of CM matrices. Suddenly 6000K does not map to the as shot camera neutral white point anymore. The image might get a color tint. If the CMs from the new dcp profile would have been used the camera neutral white point would translate to let's say 6200K, but lightroom does not update that.

That means it also behaves differently whether a profile was embedded or selected afterwards here?

Does that make sense or am I missing something here?

Are you sure the DCP profile has different CM's to the embedded profile?

I opened up an iPhone dng (which has an Apple ProRes profile embedded) in LR or ACR and got an As Shot WB reading of 4500degK and Tint of +19. If I select an Adobe Standard DCP profile instead, I get readings of 4900degK and Tint +22. These two profiles have different CM's (one lot from Apple, one lot from Adobe). The Apple one works better color wise by the way.

Dave

 

[Jack Hogan]

I think I meant something else. I import a image into Lightroom and lightroom estimates the CCT to be let's say 6000K based on the embeded white point in camera color space and based on the embedded CM matrices.

Now, I select a different dcp profile when developing the image. This profile has a different set of CM matrices. Suddenly 6000K does not map to the as shot camera neutral white point anymore. The image might get a color tint. If the CMs from the new dcp profile would have been used the camera neutral white point would translate to let's say 6200K, but lightroom does not update that.

That means it also behaves differently whether a profile was embedded or selected afterwards here?

I don't know enough about the inner workings of LR to to be more specific. LR estimates of CCT can be quite a bit off but that's usually not critical for good color around 6000K, see for instance

https://www.strollswithmydog.com/white-balance-is-off/

One can guess that if the difference in estimated CCT between the two profiles is large enough it will show in the interpolated (C and F) matrices so tones could become a little warmer or cooler, but again in typical situations they should be in the ballpark.

For best results take a reading of the illuminant CCT/tint at the scene and make your own 'tight' profile (single illuminant or e.g. 5000K and 6500K) with your own CMs. Then don't touch those dials.

Jack

 

[Thomas001le]

Thanks a lot for all the helpful comments. I did some round of experiments and also looked at the embedded profiles in all the DNGs more carefully. This is all a bit technical and refers to the DNG spec.

I also implemented the method white balance estimation myself based on the DNG specs. My implementation is close to what lightroom prints (LR seems to round to the nearest multiple of 50K), so I hope it's at least a good indicator if something is completely wrong.

Some results for images shot with the Sony A7IV.

• If I convert a ARW file to a DNG file using the Adobe DNG converter, the embeded ColorMatrix (CM) matches the one from Adobe Standard. Let's call this DNG the "Adobe DNG". The "Profile Calibration Signature" is set to "com.adobe".
• The Adobe DNG also contains some Camera Calibration (CC) matrices and the "Camera Calibration Signature" is set to "com.adobe"
• All 3rd party color profiles I own (Color fidelity and Cobalt) also have "Profile Calibration Signature" set to "com.adobe", also their CM match the one from Adobe Standard.
• To my understanding this means that if I apply one of those 3rd party profile the CC matrices from the Adobe DNG will still be applied. Yet, the embedded color profile in the DNG is replaced with the one from the DCP profile.
• It makes sense that they call have the same Color temperature for the same white point since this only depends on CC/CM matrices.

Now, where it gets interesting is the DNG generated by DXO PureRAW. Let's call it the "DXO DNG":

• The CM of the DXO DNG is **different** from the Adobe Standard CM.
• The CC of the DXO DNG is also different
• With the embeded CC and CM I could reproduce the color temperature shown in LR.
• There is not "Camera Calibration Signature" nor "Profile Calibration Signature" set for the DNG
• CM and CC being different cause a different color temperature to be computed for the same white point
• Adobe DNG and DXO DNG are visually identical

It's still not clear to me what happens when applying a DCP profile to the DXO DNG.

• According to the DNG spec the CC matrices of the DXO DNG should not be used since the Camera Calibration Signature of the DXO DNG (missing!) and the Profile Calibration Signature ("com.adobe") don't match.
• The CM matrices of the DXO DNG are replaced by the CM of the DCP profile

So this means the embeded profile of the DXO DNG is irrelevant for color conversion EXCEPT that it is used for color temperature conversion?

I also don't understand why DXO would chose to still split their color profile into CM and CC matrices if they don't match their CM matrices to any reference profile.

Anyway, that's all. It was a fun ride through the DNG specs