Custom image and DNG

[miles500]

Inadvertently I must have pressed a few wrong buttons and ended up with some of my photos having bleach bypass applied to them. Initially they looked awful when viewed on my monitor but after a few seconds the embedded JPEG gave way to the RAW DNG and all looked well - until I tried to print and then I got a dreadful green cast. After some hours of checking the printer out and using other programs, I came to the conclusion that the bleach bypass info was retained in the RAW file even though it looked fine on the monitor. To test this out I reprocessed the DNG in camera to convert to the normal bright setting and saved to JPEG and this time I could print again without the cast, although it took a lot of tweaking to get anything like the result I wanted.

Can you make the DNG available for download?

Then I can poke around inside it to see if I can spot what is going on.

Hi Barry,

This is a dropbox link to the original Pentax DNG file.

I think that the DNG converter has solved the problem, but it would be useful to know whether there is any information which governs printing and which relates to the custom image and which sticks to the file despite changing back to natural either in camera or by using the Pentax utility.

I'm a bit slow here. I've left some of my tools on my old PC, so I've had to re-download the DNG SDK to have a look inside the DNG. (The SDK includes a command-line program called dng_validate.exe which I've found very useful over many years for looking inside DNG and other files without getting to the hex/binary level).

I've compared your DNG with a DNG of my own from a K-3. (At the same firmware version 1.11). Yours doesn't look like a DNG straight from the camera. It has quite a bit of extra XMP metadata, as though it has been through one or both of Lightroom or ACR. (ACR 7.4?)

I'm not clear what the history of the DNG is. What does "original Pentax DNG file" mean?

 

[brew1brew]

Miles, I downloaded the updated one again and processed it just like I did on the first version you placed on dropbox, I got the same result and then I added SilkyPix's HDR function at 25 (the lowest setting). you can see all 3 versions here

revised JPG from your latest dropbox upload converted with SilkyPix 6 pro

WB - cloudy

contrast - a little strong

color - V2

next Same settings however I added Silkypix HDR setting to 25

--
Les
anthisphoto.smugmug.com

 

[Barry Pearson]

Inadvertently I must have pressed a few wrong buttons and ended up with some of my photos having bleach bypass applied to them. Initially they looked awful when viewed on my monitor but after a few seconds the embedded JPEG gave way to the RAW DNG and all looked well - until I tried to print and then I got a dreadful green cast. After some hours of checking the printer out and using other programs, I came to the conclusion that the bleach bypass info was retained in the RAW file even though it looked fine on the monitor. To test this out I reprocessed the DNG in camera to convert to the normal bright setting and saved to JPEG and this time I could print again without the cast, although it took a lot of tweaking to get anything like the result I wanted.

Can you make the DNG available for download?

Then I can poke around inside it to see if I can spot what is going on.

Hi Barry,

This is a dropbox link to the original Pentax DNG file.

I think that the DNG converter has solved the problem, but it would be useful to know whether there is any information which governs printing and which relates to the custom image and which sticks to the file despite changing back to natural either in camera or by using the Pentax utility.

I'm a bit slow here. I've left some of my tools on my old PC, so I've had to re-download the DNG SDK to have a look inside the DNG. (The SDK includes a command-line program called dng_validate.exe which I've found very useful over many years for looking inside DNG and other files without getting to the hex/binary level).

I've compared your DNG with a DNG of my own from a K-3. (At the same firmware version 1.11). Yours doesn't look like a DNG straight from the camera. It has quite a bit of extra XMP metadata, as though it has been through one or both of Lightroom or ACR. (ACR 7.4?)

I'm not clear what the history of the DNG is. What does "original Pentax DNG file" mean?

Yes I put it through Adobe DNG converter just prior to posting via Dropbox. The original DNG file was processed in ACDSee and I think that after the printing problems, I put it through ACR to see whether it would solve the problem- I am not sure which version but it is the one which came with PSE 11. I am a bit duddered by all the things that i tried and to avoid any doubt, I have now gone back to the file on the SD card and have replaced the file with this one which is entirely uncontaminated by any RAW converter.

https://www.dropbox.com/s/n15gpvfzdoh8wpm/IMGP9211.DNG?dl=0

Thanks. What I can say is: at the internal tag level it is just like my own out-of-camera DNG, and I can't spot any problem with it. And in Lightroom it looks the same as the first one you put into Dropbox, which suggests that all that extra XMP metadata isn't making a difference.

I can't spot any green cast, but I haven't attempted to print them However, I did softproof them, and that didn't show a green cast either.

I don't think I have anything of value to tell you!

 

[miles500]

Miles, I processed it with Silkypix 6 pro. used

WB - cloudy

contrast - a little strong

color - V2

you can download it from my smugmug if you want to try to print it. but I do see a slight green ting when viewing the original PNG and not with the JPG from Silkypix.

--
Les
anthisphoto.smugmug.com

Many thanks Les, I have downloaded your version but the acid test will be to see whether it prints correctly

--
Miles500

 

[miles500]

Inadvertently I must have pressed a few wrong buttons and ended up with some of my photos having bleach bypass applied to them. Initially they looked awful when viewed on my monitor but after a few seconds the embedded JPEG gave way to the RAW DNG and all looked well - until I tried to print and then I got a dreadful green cast. After some hours of checking the printer out and using other programs, I came to the conclusion that the bleach bypass info was retained in the RAW file even though it looked fine on the monitor. To test this out I reprocessed the DNG in camera to convert to the normal bright setting and saved to JPEG and this time I could print again without the cast, although it took a lot of tweaking to get anything like the result I wanted.

Can you make the DNG available for download?

Then I can poke around inside it to see if I can spot what is going on.

Hi Barry,

This is a dropbox link to the original Pentax DNG file.

I think that the DNG converter has solved the problem, but it would be useful to know whether there is any information which governs printing and which relates to the custom image and which sticks to the file despite changing back to natural either in camera or by using the Pentax utility.

I'm a bit slow here. I've left some of my tools on my old PC, so I've had to re-download the DNG SDK to have a look inside the DNG. (The SDK includes a command-line program called dng_validate.exe which I've found very useful over many years for looking inside DNG and other files without getting to the hex/binary level).

I've compared your DNG with a DNG of my own from a K-3. (At the same firmware version 1.11). Yours doesn't look like a DNG straight from the camera. It has quite a bit of extra XMP metadata, as though it has been through one or both of Lightroom or ACR. (ACR 7.4?)

I'm not clear what the history of the DNG is. What does "original Pentax DNG file" mean?

Yes I put it through Adobe DNG converter just prior to posting via Dropbox. The original DNG file was processed in ACDSee and I think that after the printing problems, I put it through ACR to see whether it would solve the problem- I am not sure which version but it is the one which came with PSE 11. I am a bit duddered by all the things that i tried and to avoid any doubt, I have now gone back to the file on the SD card and have replaced the file with this one which is entirely uncontaminated by any RAW converter.

https://www.dropbox.com/s/n15gpvfzdoh8wpm/IMGP9211.DNG?dl=0

Thanks. What I can say is: at the internal tag level it is just like my own out-of-camera DNG, and I can't spot any problem with it. And in Lightroom it looks the same as the first one you put into Dropbox, which suggests that all that extra XMP metadata isn't making a difference.

I can't spot any green cast, but I haven't attempted to print them However, I did softproof them, and that didn't show a green cast either.

I don't think I have anything of value to tell you!

 

[Gerry Winterbourne]

That appears to have been through some adobe product as it profiled as process version 2012 which the k3 doesn't do.

K-3 is fully supported by Adobe PV2012.

 

[Barry Pearson]

That appears to have been through some adobe product as it profiled as process version 2012 which the k3 doesn't do.

K-3 is fully supported by Adobe PV2012.

K-3 is supported, apparently fully, by process versions 2003, 2010, and 2012.

(I opened Miles' DNG in Lighroom and switched process versions. The results looked pretty much the same. This should work with ACR too).

K-3 uses DNG version 1.2.0.0. That was launched in May 2008. Every version of ACR, Lighroom, or DNG Converter, since then should support its DNGs. Hence versions of Photoshop CS3 with ACR 4.5, Lightroom 2.2, and DNG Converter 4.5. Or later.

Process versions are to do with the raw converter algorithms and the editing adjustments available. Hence rendering. They are not directly to do with supported cameras or versions of DNG.

 

[Gerry Winterbourne]

That appears to have been through some adobe product as it profiled as process version 2012 which the k3 doesn't do.

K-3 is fully supported by Adobe PV2012.

K-3 is supported, apparently fully, by process versions 2003, 2010, and 2012.

I was wrong, strictly speaking, by being brief. So, in this sentence, are you, Barry - the longer, correct answer is what you've given below (especially the sentence I've underlined).

(I opened Miles' DNG in Lightroom and switched process versions. The results looked pretty much the same. This should work with ACR too).

For anyone not familiar with these options, here's a screen shot in ACR of Miles's shot on the Camera Calibration tab. In the other tabs I set all the sliders to zero and the tone curve to linear.

You can choose one of the three Process Versions and (if the original shot was in DNG) either the Embedded camera (colour) profile or Adobe Standard. Using PEF you can use only Adobe Standard. I set the colour picker on two spots and you can read their RGB values at top left. Changing between the six options (3PV x 2 profiles), none of the RGB values are identical but neither is there a lot of variation. I was going to post screen shots of them all but the differences are so small it wasn't worth the effort.

The main changes in the newer PVs are in the reduction of haloes and other such things, and not primarily in the colours.

K-3 uses DNG version 1.2.0.0. That was launched in May 2008. Every version of ACR, Lighroom, or DNG Converter, since then should support its DNGs. Hence versions of Photoshop CS3 with ACR 4.5, Lightroom 2.2, and DNG Converter 4.5. Or later.

Process versions are to do with the raw converter algorithms and the editing adjustments available. Hence rendering. They are not directly to do with supported cameras or versions of DNG.

Other people have said that Mile's shot opens in PV2010; that is, presumably, because they are using CS5, which doesn't read PV2012.



--
---
Gerry
___________________________________________
First camera 1953, first Pentax 1985, first DSLR 2006

Galleries by Gerry Winterbourne

www.pbase.com

[email protected]

 

[Barry Pearson]

Other people have said that Mile's shot opens in PV2010; that is, presumably, because they are using CS5, which doesn't read PV2012.

When I opened Miles' first DNG in Lightroom 5.7, (equivalent to ACR 8.7, as used by Photoshop CC and perhaps CS6), it opened in Process Version 2010.

That is because I imported it using neutral presets that I had created a couple of years ago for special cases, rather than my normal K-3 presets. It is complicated!

 

[JNR]

Miles, I processed it with Silkypix 6 pro. used

WB - cloudy

contrast - a little strong

color - V2

you can download it from my smugmug if you want to try to print it. but I do see a slight green ting when viewing the original PNG and not with the JPG from Silkypix.

--
Les
anthisphoto.smugmug.com

Many thanks Les, I have downloaded your version but the acid test will be to see whether it prints correctly

--
Miles500

I strongly suspect it will print with green tinge because Silkypix will pick up the hidden instruction in the same way that ACR/LR and C1 did.

What seems to be missing here is the understanding of how DNGs vary greatly from what Adobe developed originally. Pentax and other companies essentially code a TIF instruction and put the DNG extension on it. The DNG converter is the only sure, quick way to fully strip the instruction and obtain a true raw DNG as Adobe envisioned it.

The process code (2003, 2010, 2012) has nothing to do with it.

--
JNR
www.jamesrobins.com

 

[Gerry Winterbourne]

Other people have said that Mile's shot opens in PV2010; that is, presumably, because they are using CS5, which doesn't read PV2012.

When I opened Miles' first DNG in Lightroom 5.7, (equivalent to ACR 8.7, as used by Photoshop CC and perhaps CS6), it opened in Process Version 2010.

That is because I imported it using neutral presets that I had created a couple of years ago for special cases, rather than my normal K-3 presets. It is complicated!

This is clearly a difference between ACR and LR - I didn't need (or want) to import it; I just opened it so I got my K-3 Camera Raw Defaults.

 

[miles500]

My first trial print after using Adobe DNG Converter was promising, but when I tried again with the same file today, it was the same old story of a green sky. I went back to the beginning and converted the original file again. I have used different printing profiles and different programs to print with, but to no avail. My image looks fine on screen and and soft proofed in ACDSee. Again I have selected the sky and removed all green colour, but it still comes back when I print, whatever settings I use and regardless of the image on the monitor. After so many hours and so many prints, I have come to the conclusion that if a DNG file is set to custom image, it is virtually impossible to get a print from it with a normal setting.

All is not lost as I can use the affected files as Projected Images or as monochrome ones, but it is very disappointing as I thought that this waterfront scene against a stormy background would have made a very nice print. I just wish that Pentax would put their custom image settings safely into the menus somewhere as it is pretty easy to catch the four way button as I have demonstrated!

 

[miles500]

Miles, I downloaded the updated one again and processed it just like I did on the first version you placed on dropbox, I got the same result and then I added SilkyPix's HDR function at 25 (the lowest setting). you can see all 3 versions here

revised JPG from your latest dropbox upload converted with SilkyPix 6 pro

WB - cloudy

contrast - a little strong

color - V2

next Same settings however I added Silkypix HDR setting to 25

--
Les
anthisphoto.smugmug.com

Les,



Thanks for your trouble. Just to say that your versions print out just like the others but thanks for helping anyway!



Miles

--
Miles500

 

[Barry Pearson]

What seems to be missing here is the understanding of how DNGs vary greatly from what Adobe developed originally. Pentax and other companies essentially code a TIF instruction and put the DNG extension on it. The DNG converter is the only sure, quick way to fully strip the instruction and obtain a true raw DNG as Adobe envisioned it.

What does that mean?

The DNG from a Pentax camera conforms fully to the Adobe DNG specification! I've been using out-of-camera DNGs from Pentax cameras since those cameras used lossless compression. I never have to use the DNG Converter on my Pentax out-of-camera DNGs!

Here is how I test this. I have downloaded the DNG SDK from the Adobe site. In the "targets" folder structure (for example in win/release) is a command-line program called dng_validate.exe. As its name implies, it gives a text-breakdown of the tags of a DNG, and indicates errors.

A DNG from a K-3 (or several earlier Pentax cameras) is a perfectly conformant DNG file. When comparing the text-breakdown of a DNG from the camera with the text-breakdown of a DNG that has gone through the DNG Converter after coming from the camera, the main difference is lots of XMP metadata added by the DNG Converter. That metadata is optional, and not required by the DNG specification.

For anyone wanting to examine DNGs in this way, here is a useful DOS command-line:

dng_validate -v abc.dng > def.txt

where abc.dng is the name of the (supposed) DNG file, and def.txt is the name of the text file to be created to show the analysis. (I'm assuming that the default DOS directory has been established as the folder with the DNG file in).

You might see text such as:

Validating "Miles2.DNG"...

Uses little-endian byte order
Magic number = 42
IFD 0: Offset = 8, Entries = 39
NewSubFileType: Preview Image
ImageWidth: 160
ImageLength: 120
BitsPerSample: 8 8 8
Compression: Uncompressed
PhotometricInterpretation: RGB
Make: "RICOH IMAGING COMPANY, LTD."
Model: "PENTAX K-3"

[vast amount more ....!]

Validation complete

 

[Barry Pearson]

My first trial print after using Adobe DNG Converter was promising, but when I tried again with the same file today, it was the same old story of a green sky. I went back to the beginning and converted the original file again. I have used different printing profiles and different programs to print with, but to no avail. My image looks fine on screen and and soft proofed in ACDSee. Again I have selected the sky and removed all green colour, but it still comes back when I print, whatever settings I use and regardless of the image on the monitor. After so many hours and so many prints, I have come to the conclusion that if a DNG file is set to custom image, it is virtually impossible to get a print from it with a normal setting.

All is not lost as I can use the affected files as Projected Images or as monochrome ones, but it is very disappointing as I thought that this waterfront scene against a stormy background would have made a very nice print. I just wish that Pentax would put their custom image settings safely into the menus somewhere as it is pretty easy to catch the four way button as I have demonstrated!

What program are you using to print?

I can't find any reason why your DNGs would give problems. Your problems appear to be caused by the printing-program, not the camera.

 

[awaldram]

Other people have said that Mile's shot opens in PV2010; that is, presumably, because they are using CS5, which doesn't read PV2012.

When I opened Miles' first DNG in Lightroom 5.7, (equivalent to ACR 8.7, as used by Photoshop CC and perhaps CS6), it opened in Process Version 2010.

That is because I imported it using neutral presets that I had created a couple of years ago for special cases, rather than my normal K-3 presets. It is complicated!

This is clearly a difference between ACR and LR - I didn't need (or want) to import it; I just opened it so I got my K-3 Camera Raw Defaults.

--
---
Gerry
___________________________________________
First camera 1953, first Pentax 1985, first DSLR 2006
http://www.pbase.com/gerrywinterbourne
[email protected]

Having seen a pv variance in adobe products reported by miles DMG vs k3 I. Used rawrgeapee which reports the pv used

K3 reports as 2002

Miles DNG reports as 2012

Afain 2012 was the first to support multi area exposure.

--
My PPG
http://www.pentaxphotogallery.com/artists/andrewwaldram
My Photo Stream
http://www.flickr.com/photos/awaldram/
1x.com
http://1x.com/artist/awaldram/wall

 

[Barry Pearson]

In case anyone gets confused about the relationship between DNG, TIFF, TIFF/EP, and other de facto or de jure standards, here is a breakdown:

DNG's relationship to standards

It is seen that DNG is based on ISO 12234-2, TIFF/EP. (TIFF/EP is the only ISO standard raw file format. The DNG specification says:

TIFF Compatible

DNG is an extension of the TIFF 6.0 format, and is compatible with the TIFF-EP standard. It is possible (but not required) for a DNG file to simultaneously comply with both the Digital Negative specification and the TIFF-EP standard.

Future:

ISO are currently working, far too slowly, towards updating TIFF/EP using the DNG specification).

ISO TC42 WG18 originally based TIFF/EP on TIFF. (TIFF is owned by Adobe).

 

[Gerry Winterbourne]

My first trial print after using Adobe DNG Converter was promising, but when I tried again with the same file today, it was the same old story of a green sky. I went back to the beginning and converted the original file again. I have used different printing profiles and different programs to print with, but to no avail. My image looks fine on screen and and soft proofed in ACDSee. Again I have selected the sky and removed all green colour, but it still comes back when I print, whatever settings I use and regardless of the image on the monitor. After so many hours and so many prints, I have come to the conclusion that if a DNG file is set to custom image, it is virtually impossible to get a print from it with a normal setting.

As Barry suggests, I think this is caused by the way your printer is getting the data from the file.

A raw file includes several things: the basic luminance data for each pixel plus an array of tags. These tags include masses of metadata about camera and lens; and what I'll call picture data (details of the in-camera settings (either defaults or what you chose), colour profile and more).

When the raw file is edited the basic luminance data are always unchanged. The lens/camera metadata is usually read in part but rarely the whole of it. The picture data maybe read by the editing program or it may not: the SilkyPix software provided by Pentax does read the picture tabs, so if you've set bleach bypass the conversion will show it unless you tell the program not to. Adobe programs ignore the tags with picture data, except that if you shoot GNG you can choose to read the colour profile tag for the embedded Pentax profile.

All the edits that you make to a raw file are stored in tabs alongside the basic luminance data, so as long as your software is compatible it can show what effect your edits have had. But often edits made by one program can't be read by a different program. (Incidentally, this is one of the things that worries some people about Adobe CC - if the subscription lapses you can't read your raw edits and have to go back to the original file).

What I suspect is that your viewing software is compatible with your raw editing program and is reading what you and other people have done, but your printer software isn't compatible and is effectively going back and reading the original tags.

Have you tried saving what you see on screen as JPG and looking at the result on screen; and then printing that JPG? If you have and it still looks different in the print what I've said must be wrong ...

 

[miles500]

Many thanks for all the replies and especially to Gerry for his explanations. I have printed from ACDSee, PSE, Canon Easy Print pro, Capture one 7 and the images in question always turn out greenish. All these programs of course use the same printer driver as I have only one photo printer, so the answer might lie there. I guess life is however too short to get too hung up about this so I have to call it a day before I waste any more ink and paper!