NEF or DNG?

Europa2010AD

Active member
Messages
50
Reaction score
0
Location
MD, US
For long term archival purposes... which format do you guys suggest I use? If you suggest I use DNG, do you simply convert the NEF file into DNG using CameraRAW, and then get rid of the NEFs?

Also, I noticed DNG doesn't use a separate file to save the image settings (e.g. NEF uses .xmp files for settings). Does that mean after I've made changes to DNG files, I won't be able to revert them back to their original states?
 
ATTENTION-ATTENTION
To my surprise I discoverd that the DNG format i not as un iversal as it sounds.
Ex.

Rawshooter Premium should read the DNG format, so I converted some NEF's into DNG.
Files originated from D70s : accepted
from D80 : no RAW files found
answer: only NEF files from supported cameras are accepted, and the D80 is not!

Wim
 
You need to get the new DNG converter from the adobe site. They now support the D80
 
i will just stick to nef. i mean there won't be one day to another no nef no more. so if sometime in the future adobe really drops the support of the nef files in photoshop i will have more than enough time to convert all my stuff then. right now i prefer having nef, less fiddeling. and i think chances are rather small that an upcoming photoshop version won't be able to open nef anymore even if nikon stops using it in their cameras
--
Mario

My Gallery
http://mariogrolimund.ma.funpic.de
 
You need to get the new DNG converter from the adobe site. They
now support the D80
It's not the converter, it's the raw processing program itself.

Most raw converters only support a DNG if they already support the raw file that the DNG was converted from. DNG is just a wrapper, the raw processing program needs to know how to handle the data inside that wrapper.

--
Normally, a signature this small can't open its own jumpgate.

Ciao! Joe

http://www.swissarmyfork.com
 
ATTENTION-ATTENTION
To my surprise I discoverd that the DNG format i not as un iversal
as it sounds.
Ex.
Rawshooter Premium should read the DNG format, so I converted some
NEF's into DNG.
Files originated from D70s : accepted
from D80 : no RAW files found
answer: only NEF files from supported cameras are accepted, and the
D80 is not!
Wim, how did you convert from NEF to DNG? Did you set it to embed the original RAW file into the DNG as well?

This may sound like an improbable question but was your version of DNG converter updated enough to be able to properly read a D80 NEF file? I can't recall if the DNG converter makes use the ACR program to read the NEF files. If so, did you have the latest ver of ACR that supports the D80 NEF?

larsbc
 
--nefs to the NX icon, you can not drag and drop DNG to the icon for opening.

There is a significant space savings with DNG, but after fooling with several hundred of them, and retaining the NEF files, I think I will just stop... Archival storage is not that expensive anymore.
Those that use other converters may feel differently.
MATTinNE_FL
 
As already stated before it is not the DNG converter to blame but Rawshooter premium. With other RAW converters everything is OK.

Wim
 
There is a significant space savings with DNG, but after fooling
with several hundred of them, and retaining the NEF files, I think
I will just stop... Archival storage is not that expensive anymore.
People generally don't convert to DNG for the purpose of saving space. The main reason is that most RAW formats, including NEF, are proprietary. The file format of those RAW files are often unique for each type and brand. There's no guarantee that you can open your precious RAW files in the future with newer software.

Adobe tries to solve this issue by getting people to use DNG. The DNG format is fully open to everyone, which makes it a lot easier for future software to incorperate DNG support.

If you really want to be sure you can convert your RAW files to DNG and include the original RAW file inside the DNG. This results in a very large file (RAW+DNG), but then you'll always have the original.
 
For long term archival purposes...
I would then suggest JPEG (fine/100%) instead.
which format do you guys suggest
I use? If you suggest I use DNG, do you simply convert the NEF file
into DNG using CameraRAW, and then get rid of the NEFs?
I use Capture NX (it gives, for ME, the best results with NEFs), so it's NEFs all the way.
Also, I noticed DNG doesn't use a separate file to save the image
settings (e.g. NEF uses .xmp files for settings).
That's one of the main reasons that I use NEFs. I can change whatever settings I (or camera) have done, even get the original RAW file.

--
Paulo Goulart
( http://goulart.zenfolio.com )
 
Most raw converters only support a DNG if they already support the
raw file that the DNG was converted from. DNG is just a wrapper,
the raw processing program needs to know how to handle the data
inside that wrapper.
I think you're wrong about that! The whole idea behind DNG is that you don't have those manufacturer specific RAW processing procedures anymore. When software supports DNG it should be able to read all DNG's, no matter what the original RAW file format was.

This also shows why it might still be useful to include the original RAW file inside the DNG file. There might be some sort of information that can be extracted from native RAW files, which aren't supported by DNG. This could mean loss of data. DNG tries to get around this, by letting manufacturers add custom meta data. But not all software that can read DNG can also process this custom meta data. This might be the problem you are referring to.
 
If you're worried that NEF might dissappear in the future, why not
keep everythin in NEF, and archive a NEF to DNG converter?
Because programs tend to be obsolete even faster than file formats are. Your NEF to DNG converter program may not work well on the next version of Windows or the Mac, and probably won't work at all on Linux.

Think of it this way. If you had archived documents back when you had a TRS-80, or an Apple ][ computer, and you had archived a conversion program along with those documens, would you still be able to run that conversion program?

No, if you are concerned about long-term archival, you really need to look at the format rather than a program that can read it.

Eric
--
http://www.lumenssolutions.com/photography/
 
Most raw converters only support a DNG if they already support the
raw file that the DNG was converted from. DNG is just a wrapper,
the raw processing program needs to know how to handle the data
inside that wrapper.
I think you're wrong about that!
I wish I were.
The whole idea behind DNG is that
you don't have those manufacturer specific RAW processing
procedures anymore.
No, that's actually just fluff. The "whole idea behind DNG" was to draw attention away from some negative press Adobe was getting concerning their own draconian intellectual property practices. They had recently come under fire for three issues:

1) Dropping the open (and free) licensing terms for the PhotoShop SDK (software development kit, the tool that lets people write PhotoShop plugins and utilities) and implementing a new system where you paid for access to the SDK, but Adobe might just pocket your money and still not approve you for access.

2) Their new activation scheme (if you've ever had your legitimate copy of PhotoShop spontaneously "deactivate" on you, you'd understand).

3) Arranging the arrest of Russian programmer Dmitry Sklyarov for attempting to decrypt the encrypted portion of Adobe's proprietary .pdf format (they made a DMCA case out of it, despite his screen reader for blind people being an obvious fair use case).

Camera manufacturers have been encrypting parts of raw files for years. Raw file decoder writers (including Adobe) quickly (but quietly) cracked the encryption. Nikon was unfortunate enough to adopt Canon style encryption of the D2X white balance parameters at a time when Adobe needed a diversion, so instead of doing business as usual and cracking it (Dave Coffin did it within 72 hours of someone sending him his first D2X raw file) they put man-years into a media blitz of press releases, software development, and pompous speeches about Adobe being the white knight who was going to save you from the evils of encryption.

In short, they created an enormous pile of DNG.
When software supports DNG it should be able to
read all DNG's, no matter what the original RAW file format was.
That's the dream, and it's a concept propagated by Adobe evangelists, but if you read Adobe press releases about DNG, it's not something that Adobe has ever promised.

(to be continued)

--
Normally, a signature this small can't open its own jumpgate.

Ciao! Joe

http://www.swissarmyfork.com
 
This also shows why it might still be useful to include the
original RAW file inside the DNG file. There might be some sort of
information that can be extracted from native RAW files, which
aren't supported by DNG. This could mean loss of data. DNG tries to
get around this, by letting manufacturers add custom meta data. But
not all software that can read DNG can also process this custom
meta data. This might be the problem you are referring to.
Not at all. It's a lot more than a bit of metadata. It's a question of fundamental algorithms.

For example, any raw decoder can handle a basic Bayer pattern raw file with fairly colormetric (responding to color like the human eye) filters and a strong anti-aliasing filter. That got you an acceptable image from the basic Nikon D100 or Canon 10D style camera.

But if you have a weaker AA filter, you now need some sort algorithm to reduce moire (red and blue color bands across detailed areas). If you want to decode a Leica DMR or M8 file, you need an even more aggressive and sophisticated moire reduction algorithm. The DNG file can't provide that algorithm, all it can provide is the metadata that says you need it. So, many raw decoders that can decode DNG from cameras that don't need this algorithm can't decode M8 raw files, even though the M8 outputs DNG as its native format.

There are more severe examples. Fuji S3 and S5 have HDR (high dynamic range) sensors: double Bayer matrix that interleaves red, green, and blue sensors of two different sensitivities. The raw processor has to figure out how to do HDR processing, to put the two differently exposed images back together into one image. DNG can tell you that the data is there, but until the raw processing program knows what to do with the data, DNG is no help.

The Fuji Super CCD sensors (used in the S1, S2, S3, and S5 DSLRs,as well as all their point and shoot and prosumer cameras that have raw output) also require another algorithm to rotate the Bayer matrix 45 degrees, because Fuji rotates it to get higher horizontal and vertical resolution scores.

The best raw processors do sharpening based on the shape that a camera's AA filter "spreads" a point of light into (this is called the PSF, the "point spread function"). To get the detail that Fuji fans have been talking about ever since the S1, you have to cope with Fuji's AA filter having a diamond shaped PSF (again 45 degree rotation), instead of the square PSF of Nikon cameras, or the round PSF of Canons). DNG only codes the "strength" of the AA filter, it says nothing about the PSF shape.

Foveon sensors (Sigma SD9, SD10, SD14, and DP1, Polaroid X530) need unique noise reduction (high correlated noise between channels), clipping compensation, red channel sharpening, and very unique color math because a conventional 3x3 matrix transform (more on those, later) of Foveon data produces strange results (if you want to get technical, the matrix has large off-diagonal coefficients that increase mathematical noise).

Nikon D2X and D200 need algorithms to cope with Nikon's use of independent hardware gain controls on the red, green, and blue channels.

Camera after camera, it's not just data, but special algorithms required to deal with it.

Then there's the issue of color.

All raw files contain very crude color information, just 11 numbers: a red and blue gain value used for white balance, and a 3x3 matrix that transforms sensor output to as close to "human eye" colors as is possible for a simple bit of math like that. This is why 3x3 matrix color tends to leave the reds, blues, and purples a bit unnatural looking. Fuji, Foveon, and Sony's RGBE sensor (only used in the Sony 828, to date) are especially ill suited for standard 3x3 matrix color.

Most raw processing software developers want better color then you can get from a 3x3 matrix operation, so each implements a different way of looking at color. Phase One Capture One is perhaps the most sophisticated, they use two space-to-space interpolators, basically ICC profiles of the sensor in warm and cool light, and interpolate between the interpolators to get good color for whatever lighting conditions you shot in. So, P1C1 can't do raw decoding unless Phase One has actually had a sample of the particular camera in their lab to measure the spectral sensitivity of the sensor and compute those profiles. Converting to DNG won't help. If they've never calibrated the software for a Nikon D3X or Canon 1Ds III, they can't decode the raw files properly.

What sort of color information does DNG contain? Did Adobe look at how other raw converters do color, and make DNG capable of satisfying every raw converter's needs? Did they ask other raw converter writers (you know, like you do if you're really creating an open standard, instead of just making a lot of noise about doing it)? No. They simply gave DNG the kind of spectral coefficients used by Adobe Camera Raw.

So, DNG isn't good for the consumer, it's good for Adobe. It insures that no other raw decoder can ever outperform ACR. Is it any wonder that so few are buying into it?

--
Normally, a signature this small can't open its own jumpgate.

Ciao! Joe

http://www.swissarmyfork.com
 
If you're worried that NEF might dissappear in the future, why not
keep everythin in NEF, and archive a NEF to DNG converter?
Because programs tend to be obsolete even faster than file formats
are. Your NEF to DNG converter program may not work well on the
next version of Windows or the Mac,
In it's defense, the DNG converter is very simple: it just does file IO, doesn't talk to ports, and doesn't have much of a UI. It can run on very old operating systems, and there's nothing that will violate any protections or use such sophisticated OS facilities that it won't run on future systems.
and probably won't work at all
on Linux.

Think of it this way. If you had archived documents back when you
had a TRS-80, or an Apple ][ computer, and you had archived a
conversion program along with those documens, would you still be
able to run that conversion program?
Of course you would.

http://discover-net.net/~dmkeil/
http://www.arrowweb.com/mkr/
http://www.vavasour.ca/jeff/trs80.html
http://www.wbwip.com/a2web/a2emul.html
http://www.classicgaming.com/vault/appleemu.shtml
http://apple2.intergalactic.de/
No, if you are concerned about long-term archival, you really need
to look at the format rather than a program that can read it.
Then it's lucky we have people like David Coffin. His open source dcraw program decodes the raw file formats for about 300 cameras, including many that are so obscure Adobe didn't build support for them into the DNG converter.

http://cybercom.net/~dcoffin/dcraw/
http://en.wikipedia.org/wiki/Dcraw

It's in C so generic you can compile it on just about anything. There are builds for 386 so generic that they can run on anything from 14 year old 386 machines with 64 megs of RAM and MS-DOS or Windows 95 right up to a quad core with Vista, and builds optimized for MMX, 64 bit Intel or AMD, etc. There are builds for the fruit flavored computer substitute all the way from OS 6 to 9, and every feline version of OS/X from panther to puddycat. Ports to embedded systems, hand held devices (I've got it on my PDA).

And it's so widely distributed that if Dave disappeared today, someone would check it into sourceforge tomorrow and turn it into a community project.

There are GUI (graphical user interface) front ends for dcraw, for those who don't like command lines. In Windows, you can use the open source RAWdrop. For Linux, there's three different GIMP plugins, and UFRaw. There's also GUIs for Mac.

So yes, right now, we have a path to support all cameras, past, present, and future, no matter how obscure, on computers ranging from near unusable antiques to today's best, right out into the future.

And we did it all, without a pile of DNG.

--
Normally, a signature this small can't open its own jumpgate.

Ciao! Joe

http://www.swissarmyfork.com
 
There is a significant space savings with DNG, but after fooling
with several hundred of them, and retaining the NEF files, I think
I will just stop... Archival storage is not that expensive anymore.
People generally don't convert to DNG for the purpose of saving
space.
Actually, they do. The strongest support of DNG is among the users of cameras that have no raw file compression and produce raw files much larger than the competition. Barry Pearson, dpReview's resident DNG evangelist, stresses this point.

http://www.barrypearson.co.uk/articles/dng/benefits.htm

DNG can cut the size of Fuji, Pentax, older Nikon or Canon (basically, 2003 and earlier) or Olympus files in half.
The main reason is that most RAW formats, including NEF, are
proprietary. The file format of those RAW files are often unique
for each type and brand. There's no guarantee that you can open
your precious RAW files in the future with newer software.
Actually, there is. There isn't a guarantee that you can open them with all software, but there is one that you can open it with a lot of software.
Adobe tries to solve this issue by getting people to use DNG. The
DNG format is fully open to everyone, which makes it a lot easier
for future software to incorperate DNG support.
Speaking as a software developer, it doesn't make things "a lot easier", nor even a little easier. It makes things a bit harder. It's something that raw software decoders have to support, to compensate for the people who convert files to DNG needlessly. It's not something that anyone wants to support.
If you really want to be sure you can convert your RAW files to DNG
and include the original RAW file inside the DNG. This results in a
very large file (RAW+DNG), but then you'll always have the original.
--
Normally, a signature this small can't open its own jumpgate.

Ciao! Joe

http://www.swissarmyfork.com
 
Wow, that was some great insight into the RAW situation... Thanx for your elaborate answer. I haven't started converting my NEF's to DNG, but now I don't think I ever will...
 

Keyboard shortcuts

Back
Top