Adobe Digital Negative (DNG) instead of RAW

Adobe introduces a 'standard' for RAW files, DNG. For now they
offer a converter to go from RAW to DNG, but they hope that
manufacturers are going to write directly to DNG format, so you
wouldn't need support anymore for all those different RAW formats.
It supports lossless compression too..
See: http://www.adobe.com/products/dng/main.html

--
Menno
http://www.xs4all.nl/~mvanb/albums
Seems an excellent idea.

I wonder if the file sizes are any smaller than Pentax RAW.

I will download it and see what happens after conversion. If so, let's hope the manufacturers take it up.

Cheers
--
Richard Day - 'Carpe Diem!'
Gloucester UK
 
Adobe introduces a 'standard' for RAW files, DNG. For now they
offer a converter to go from RAW to DNG, but they hope that
manufacturers are going to write directly to DNG format, so you
wouldn't need support anymore for all those different RAW formats.
It supports lossless compression too..
See: http://www.adobe.com/products/dng/main.html

--
Menno
http://www.xs4all.nl/~mvanb/albums
Seems an excellent idea.

I wonder if the file sizes are any smaller than Pentax RAW.

I will download it and see what happens after conversion. If so,
let's hope the manufacturers take it up.
It is a great idea, assuming that

(a) Adobe make the standard available to anyone who wants to use it at little or no cost
(b) Camera makers adopt it.

Worst case is that only photoshop reads the files, and no camera writes them.

Best case is everyone within six months it's as widely supported as JPEG, (OK as PNG then) and Pentax ship a firmware upgrade to support it.

We know they don't compress files, and a "Pentax-own-brand" compressed-raw would mean they had to write new software which I don't see happening. A vendor-independant compressed-raw has a better chance (a menu option could let you choose DNG or PEF)

Pentax's software is the worst thing about the whole *ist-D package (and by a big margin). Talk to Nikon and Canon owners and they think their software sucks too. Software companies should not make cameras and optics companies shouldn't write software.
 
Adobe introduces a 'standard' for RAW files, DNG. For now they
offer a converter to go from RAW to DNG, but they hope that
manufacturers are going to write directly to DNG format, so you
wouldn't need support anymore for all those different RAW formats.
It supports lossless compression too..
See: http://www.adobe.com/products/dng/main.html

--
Menno
http://www.xs4all.nl/~mvanb/albums
Seems an excellent idea.

I wonder if the file sizes are any smaller than Pentax RAW.

I will download it and see what happens after conversion. If so,
let's hope the manufacturers take it up.
It is a great idea, assuming that
(a) Adobe make the standard available to anyone who wants to use it
at little or no cost
(b) Camera makers adopt it.

Worst case is that only photoshop reads the files, and no camera
writes them.
Best case is everyone within six months it's as widely supported as
JPEG, (OK as PNG then) and Pentax ship a firmware upgrade to
support it.
We know they don't compress files, and a "Pentax-own-brand"
compressed-raw would mean they had to write new software which I
don't see happening. A vendor-independant compressed-raw has a
better chance (a menu option could let you choose DNG or PEF)

Pentax's software is the worst thing about the whole *ist-D package
(and by a big margin). Talk to Nikon and Canon owners and they
think their software sucks too. Software companies should not make
cameras and optics companies shouldn't write software.
I've downloaded the converter and converted a Pentax RAW file to DNG.

PEF file size = 13.5MB DNG file size = 6.14MB

Opened and examined in PS CS against the original PEF file - I cannot see any discernable differences. Colour, jaggies, noise etc., etc.. all seem identical.

Looks good to me.

Come on Pentax - adopt it and give us a firmware upgrade, double the capacity on my 1GB cards. YES PLEASE!

--
Richard Day - 'Carpe Diem!'
Gloucester UK
 
PEF file size = 13.5MB DNG file size = 6.14MB
Come on Pentax - adopt it and give us a firmware upgrade, double
the capacity on my 1GB cards. YES PLEASE!
Yes I also hope this is possible with a firmware upgrade, that would be great (Pentax has a good reason to use this standard, AFAIK it has the largest RAW-size of all the comparable DSLR's).

Menno
 
Richard,

Sounds great. Can you tell us if the DNG converter keeps all of the metadata intact?

Cheers!
 
I just converted 444MB of PEF files into 180MB of DNGs!!!
It took few seconds on a centrino laptop...

Metadata seems to be intact (from Photoshop file properties).

Adobe just cut my storage requirements by more than 2X for free!

Thanks Adobe!

I hope Pentax users can enjoy DNG with a firmware update altough I think it could be difficult (Pentax *ist D firmware memory limitation / processing power limitation). Let's hope for the best...

Vincenzo
 
Richard,

Sounds great. Can you tell us if the DNG converter keeps all of
the metadata intact?

Cheers!
Everything is intact, apart from the reportage on lens make and model, which is only available in the Pentax Browser anyway (and usually only if a Pentax lens is used) and as that cannot read DNG format I don't know if this information is kept..

Incedentally you can set the DNG parameters to store the data in uncompressed format as well as it's default lossless compression. In uncompressed format the files are 11.9MB.

Rgds
--
Richard Day - 'Carpe Diem!'
Gloucester UK
 
I am praying for a firmware update for .dng, I just converted 3.6 gig of raw files into 1.6 gig of .dng, now thats space saving without loss of quality!!!
I just converted 444MB of PEF files into 180MB of DNGs!!!
It took few seconds on a centrino laptop...

Metadata seems to be intact (from Photoshop file properties).

Adobe just cut my storage requirements by more than 2X for free!

Thanks Adobe!

I hope Pentax users can enjoy DNG with a firmware update altough I
think it could be difficult (Pentax *ist D firmware memory
limitation / processing power limitation). Let's hope for the
best...

Vincenzo
--
http://www.torphoto.net
 
PEF file size = 13.5MB DNG file size = 6.14MB
Come on Pentax - adopt it and give us a firmware upgrade, double
the capacity on my 1GB cards. YES PLEASE!
Yes I also hope this is possible with a firmware upgrade, that
would be great (Pentax has a good reason to use this standard,
AFAIK it has the largest RAW-size of all the comparable DSLR's).

Menno
Just did a bulk conversion. 1760 raw files converted in 59 minutes in 3GHz P4 HT. Reduced from 22.4GB down to 9.02GB. Average of 6.1MB each. (Biggest was 7.4MB, Smallest only 3.3 MB!)
 
The guys/gals at Pentax who write firmware are presumably no longer working on the istD - probably not even the istDS - they'll most likelky already be working on whatever is the pipeline after that - unless there is a need to patch the D or DS due to a true defect. This is a 'feature' issue not a 'defect' issue.

IMO Pentax aren't going to release any new features for an 'old' camera. Any improvements will go into the next model to maximise revenue on new sales (and maybe some upgrade purchases).

On the other hand, hopefully the Pentax Browser/Raw 2.0 software that is coming along with the DS will be a significant improvement - I saw some screenshots in a link from someone's posting and it looked a much richer interface than the current POS - and fingers crossed that us istD owners will also get the benefit there.
I just converted 444MB of PEF files into 180MB of DNGs!!!
It took few seconds on a centrino laptop...

Metadata seems to be intact (from Photoshop file properties).

Adobe just cut my storage requirements by more than 2X for free!

Thanks Adobe!

I hope Pentax users can enjoy DNG with a firmware update altough I
think it could be difficult (Pentax *ist D firmware memory
limitation / processing power limitation). Let's hope for the
best...

Vincenzo
--
http://www.torphoto.net
 
Richard,

Sounds great. Can you tell us if the DNG converter keeps all of
the metadata intact?

Cheers!
Unforunately, the creation and modification dates change to the
time of the conversion, rather than the date the image was shot.
(At least in the Mac version.)
I have to modyfy that. The date of cretaion and modification file itself is changed, but the original date is preserved intact in the "Camera Data (EXIF)".

Nevertheless, it would have been nice to preserve the creation date of the FILE itself, as that would have made it extremely easy to sort images by date, meaning the actual date the image was shot.
 
PEF file size = 13.5MB DNG file size = 6.14MB
Come on Pentax - adopt it and give us a firmware upgrade, double
the capacity on my 1GB cards. YES PLEASE!
Yes I also hope this is possible with a firmware upgrade, that
would be great (Pentax has a good reason to use this standard,
AFAIK it has the largest RAW-size of all the comparable DSLR's).
It is certainly possible because one file writing format is as good as another.

There is a world of difference between the previous version of this request

"Dear Pentax, please make PEF losslessly compressed and re-write the browser and photo lab software to go with it" .
and the new one

"Dear Pentax, please give a us a choice of PEF or DNG to support software from 3rd parties."

The following would be valid reasons NOT to.

(a) Adobe screw up (1); they charge huge licensing fees for using it, and no other software writers adopt it.

(b) Adobe screw up (2); they don't provide developer support to the camera makers to help include a DNG encoder so the development effort makes it too costly.
(c) Pentax limitation. There's no space in the firmware.

Now from the Adobe press release

Adobe have avoided screw-up (1) as I called it

"The Digital Negative Specification is being posted to the Adobe Web site free of any legal restrictions or royalties, enabling integration of the .DNG file format into digital cameras, printers, and software products."

As for screw-up (2), well they've partially avoided that too

"The Digital Negative Specification is based on the TIFF EP format, an accepted standard, and already the basis of many proprietary raw formats"

I think that is what is used in PEF. I've certainly read that each colour plane is actually a TIFF ... this implies relatively little work to do for Pentax

So the ball would be in Pentax's court. We know on the on hand (Optio 550) they don't introduce "fixes" when the camera is working as designed, and on the other they have improved support for M-series lenses, introduced remote assistant, and updated the Exif support in the *ist-D.
 
The guys/gals at Pentax who write firmware are presumably no longer
working on the istD - probably not even the istDS - they'll most
likelky already be working on whatever is the pipeline after that -
unless there is a need to patch the D or DS due to a true defect.
This is a 'feature' issue not a 'defect' issue.
That is not the pattern that the *ist-D followed, 1.10 firmware made a big step to support m-series lenses, and updated exif support, and 1.11 added support for remote assistant. These where feature adds, not fixes.

I'd be very surprised if most of the firmware isn't common to the *ist-D and DS, and much is carried forward to whatever comes next. Effectively they're three different builds from the same same code base. I think the product planning meeting goes like this.

Agenda item (x) Adobe DNG --- do we adopt it one (code name of next camera) ? Mumblings of 'it's easy' 'no reason why not' etc. Ok do we back port it the *ist-DS and *ist-D ?
 
Richard,

Sounds great. Can you tell us if the DNG converter keeps all of
the metadata intact?

Cheers!
Unforunately, the creation and modification dates change to the
time of the conversion, rather than the date the image was shot.
(At least in the Mac version.)
I have to modyfy that. The date of cretaion and modification file
itself is changed, but the original date is preserved intact in the
"Camera Data (EXIF)".
Nevertheless, it would have been nice to preserve the creation date
of the FILE itself, as that would have made it extremely easy to
sort images by date, meaning the actual date the image was shot.
It's the same in the Windows version.

I agree with you and have reported this back to Adobe.

Regards
--
Richard Day - 'Carpe Diem!'
Gloucester UK
 
Looks great!! And if a firmware upgrade would be able to allow the istD and istDS to write these files that would be even better.

...but putting the RAW data through a compression algorithm may also slow down the process of writing the files. Existing JPEG compression is probably hardwired into the processor, but this compression would have to be entirely done through the firmware (=software). Wouldn't that slow down things? And as these algorithms tend to be relatively complex, is there enough space in the firmware for this?

Would seem a shame though if it not possible... Why can't Adobe work with camera manufacturers, then the istDS could have had this DNG raw format... BTW, anyone get more info on the raw files in the istDS? Anyone passing at Photokina e.g.?

rgds, Wim
 
PEF file size = 13.5MB DNG file size = 6.14MB
Come on Pentax - adopt it and give us a firmware upgrade, double
the capacity on my 1GB cards. YES PLEASE!
Yes I also hope this is possible with a firmware upgrade, that
would be great (Pentax has a good reason to use this standard,
AFAIK it has the largest RAW-size of all the comparable DSLR's).
It is certainly possible because one file writing format is as good
as another.

There is a world of difference between the previous version of this
request
"Dear Pentax, please make PEF losslessly compressed and re-write
the browser and photo lab software to go with it" .
and the new one
"Dear Pentax, please give a us a choice of PEF or DNG to support
software from 3rd parties."

The following would be valid reasons NOT to.
(a) Adobe screw up (1); they charge huge licensing fees for using
it, and no other software writers adopt it.
(b) Adobe screw up (2); they don't provide developer support to the
camera makers to help include a DNG encoder so the development
effort makes it too costly.
(c) Pentax limitation. There's no space in the firmware.

Now from the Adobe press release

Adobe have avoided screw-up (1) as I called it
"The Digital Negative Specification is being posted to the Adobe
Web site free of any legal restrictions or royalties, enabling
integration of the .DNG file format into digital cameras, printers,
and software products."

As for screw-up (2), well they've partially avoided that too
"The Digital Negative Specification is based on the TIFF EP format,
an accepted standard, and already the basis of many proprietary raw
formats"

I think that is what is used in PEF. I've certainly read that each
colour plane is actually a TIFF ... this implies relatively little
work to do for Pentax

So the ball would be in Pentax's court. We know on the on hand
(Optio 550) they don't introduce "fixes" when the camera is working
as designed, and on the other they have improved support for
M-series lenses, introduced remote assistant, and updated the Exif
support in the *ist-D.
If Pentax have a shortage of space in the firmware, drop the TIFF option and replace with DNG, you can always make a TIFF from either a RAW or DNG file later. It is highly probable that the demand for TIFF files by agencies will change to DNG anyway, judging by the comments from Getty etc..

Useful informative links

http://www.forbes.com/businesswire/feeds/businesswire/2004/09/27/businesswire20040926005022r1.html

http://www.forbes.com/businesswire/feeds/businesswire/2004/09/27/businesswire20040926005020r1.html
http://www.luminous-landscape.com/reviews/software/dng.shtml

Cheers
--
Richard Day - 'Carpe Diem!'
Gloucester UK
 
Wim, it is unlikely to take much processing power to do this compression, as JPEG loss-less compression is specified in the DNG standard, and this likely uses Huffman compression as the other option of arithmetic compression would be still subject to license fees, would indeed be slower, and would likely only reduce file sizes by another 10%. Huffman compression is the last stage used in JPEG encoding, so most of what is required should be already be built into the harware engine, since that engine likely already has enough bit depth in calculations to deal with the 12 bit per photosite data from the CCD (likely the engine works in 16 bits).

In short, Huffman compression, which is likely used, takes very little time, and is already partially built into the cameras that support JPEG images.

I would think that most of the time required to convert directly from PEF to DNG format is just reading and writing the fairly large file sizes.

Regards, GordonBGood
Looks great!! And if a firmware upgrade would be able to allow the
istD and istDS to write these files that would be even better.

...but putting the RAW data through a compression algorithm may
also slow down the process of writing the files. Existing JPEG
compression is probably hardwired into the processor, but this
compression would have to be entirely done through the firmware
(=software). Wouldn't that slow down things? And as these
algorithms tend to be relatively complex, is there enough space in
the firmware for this?

Would seem a shame though if it not possible... Why can't Adobe
work with camera manufacturers, then the istDS could have had this
DNG raw format... BTW, anyone get more info on the raw files in the
istDS? Anyone passing at Photokina e.g.?
 
Thanks Gordon! I didn't read the specs myself... Not that it would have helped... ;-)

In that case, Pentax should make an effort to adapt the firmware for the istDS at least. That way that would add an extra element to boost istDS sales... Time will tell I guess...

Wim
Wim, it is unlikely to take much processing power to do this
compression, as JPEG loss-less compression is specified in the DNG
standard, and this likely uses Huffman compression as the other
option of arithmetic compression would be still subject to license
fees, would indeed be slower, and would likely only reduce file
sizes by another 10%. Huffman compression is the last stage used
in JPEG encoding, so most of what is required should be already be
built into the harware engine, since that engine likely already has
enough bit depth in calculations to deal with the 12 bit per
photosite data from the CCD (likely the engine works in 16 bits).

In short, Huffman compression, which is likely used, takes very
little time, and is already partially built into the cameras that
support JPEG images.

I would think that most of the time required to convert directly
from PEF to DNG format is just reading and writing the fairly large
file sizes.

Regards, GordonBGood
 

Keyboard shortcuts

Back
Top