NEF file size increase in PP (ViewNX2)

I shoot Raw (NEF + Jpeg Fine) with a D5000 and am new to PP in Raw.

Every time I make the slightest change to the NEF file in ViewNX2 (even just changing the WB, sharpening or a tiny exposure correction) its size increases typically by around 60%, on average from around 10mb to 16mb (an issue for me no matter how affordable additional harddrives are - already using 3 for one original and 2 backups). And once saved, which is required to keep the changes when switching to another file, I cannot get the original file size back by reverting and saving again.

Is this the normal behaviour of NEF files in ViewNX?

How about other raw converters?

Which good raw converter/PP software allows to record the modifications with a more reasonable increase in file size, ideally in NEF format?

Or do you have to save in TIFF in non-Nikon converters?

--

D5000 - Nikkor AF-S DX 18-105mm F/3.5-5.6G ED VR - Nikkor AF-S 50mm f/1.4 G - Nikkor AF-S 70-300mm VR - Canon PowerShot S3

Lightroom, Aperture and others, leave the original file alone and write their adjustments into a separate file. The second file is very small because it only contains instructions, not an image.

R Ortiz said:

Lightroom, Aperture and others, leave the original file alone and write their adjustments into a separate file. The second file is very small because it only contains instructions, not an image.

Click to expand...

Thanks. Are these files linked, i.e. when you transfer the processed file to another storage device or software both file come with it easily or do you have to make sure they come in pairs via manual selection of both?

I suppose none of those instruction files are compatible between programs and that th universal file exchange format for that would be TIF? When a TIF file is created from a retouched NEF, are all the instructions then included in it, including the information allowing to revert to the original?

How about Capture NX? Does it bloat the NEF file size by 60% like VNX when it updates the instructions right in the NEF file?

Sorry for all the questions. Still puzzled why VNX cannot just write the simple instructions into the NEF without increasing the file size that much.
--

D5000 - Nikkor AF-S DX 18-105mm F/3.5-5.6G ED VR - Nikkor AF-S 50mm f/1.4 G - Nikkor AF-S 70-300mm VR - Canon PowerShot S3

Polyglot92 said:

Thanks. Are these files linked, i.e. when you transfer the processed file to another storage device or software both file come with it easily or do you have to make sure they come in pairs via manual selection of both?

Click to expand...

They go together.

Polyglot92 said:

I suppose none of those instruction files are compatible between programs and that th universal file exchange format for that would be TIF? When a TIF file is created from a retouched NEF, are all the instructions then included in it, including the information allowing to revert to the original?

Click to expand...

After saving the file as tif, you won't be able to change the edits previuosly applied to the image.

Polyglot92 said:

How about Capture NX? Does it bloat the NEF file size by 60% like VNX when it updates the instructions right in the NEF file?

Click to expand...

Saving a NEF in NX2 increases the file size substantially. Sometimes even more than 60%.

Polyglot92 said:

Thanks. Are these files linked, i.e. when you transfer the processed file to another storage device or software both file come with it easily or do you have to make sure they come in pairs via manual selection of both?

Click to expand...

If you use Lightroom or Bridge to manage your images, these "sidecar" files are teken care of automatically - if you move, rename, copy or delete an image the XMP file follows suit invisibly. I expect the same is true for other programs that make sidecar files. When working entirely in a manual file-based way, you do have to take care of this yourself. In the case of Lightroom, these are only optional redundant copies of information held primarily in LR's own database, so it does not matter if they get lost.

Polyglot92 said:

I suppose none of those instruction files are compatible between programs

Click to expand...

Correct, with the exception that the Adobe programs are inter-compatible. This aside, each program has its own metadata format for the various settings involved in Raw conversion.

Polyglot92 said:

and that th universal file exchange format for that would be TIF? When a TIF file is created from a retouched NEF, are all the instructions then included in it, including the information allowing to revert to the original?

Click to expand...

In the case of a Raw file there was no "original" image - only a starting default processing of some kind, and source data for that. And btw defaults can change or be different in one program or another.

A TIFF is a converted file, it is no longer a Raw. So Raw conversion settings have become irrelevent in this case, and are no longer included. And the Raw data is not part of the TIFF anyway. However: other kinds of metadata such as keywords, copyright etc are still applicable and should usually be included - these are in a standard universally understood "language", as with JPG images. The reason for using TIFF rather than JPG is purely to avoid compression losses, and preserve a file that is still suitable for further processing. But it's still a converted file.

Polyglot92 said:

Sorry for all the questions. Still puzzled why VNX cannot just write the simple instructions into the NEF without increasing the file size that much.

Click to expand...

I can't help you with that software, sorry. Adobe "sidecar" processing metadata is just a few Kb, sometimes hundreds of Kb if lots of local adjustment masks are included. I would have expected NX data to be similar, since the function is somewhat similar. But perhaps a large preview bitmap is also being included.

regards, RP

RP

Thanks for your answers everyone.

So both ViewNX and CaptureNX blow up the NEF file size by at least 60% to write some simple instructions and apparently to add a (second?) useless preview image. For now I won't be spending 200$ on a proprietary software and then the same amount again on additional harddrives, or end up shooting less.

TIFF doesn't seem like a good option either since from what I have seen it blows up the file size to 35mb in 8bit and 70mb in 16bit, at least when created in ViewNX.

So I will be looking at Photoshop Elements first, I guess, and perhaps Lightroom or DxO.

--

D5000 - Nikkor AF-S DX 18-105mm F/3.5-5.6G ED VR - Nikkor AF-S 50mm f/1.4 G - Nikkor AF-S 70-300mm VR - Canon PowerShot S3

but it is by far the best all in one available.

I believe that the version of ACR included with Elements is a more limited version of the raw conversion software.

I haven't experienced the file bloat from Vnx, but everytime from Cnx, so it only gets occasional use now, since LR3 upped it's conversion quality.

Not sure why you are getting that much file bloat. I use VNX for all my D700 NEF conversions to JPEG and never see that kind of percentage increase even with tweeks to the settings. A typical Raw NEF before edits is @25 MB, after it is 15-20%. Although I don't shoot Raw + JPEG which may be the issue.

Also, Not sure why you are shooting Raw + JPEG, a JPEG file is embedded in the Raw data set for camera display and software display. It'll save you card and hard drive space and then you can convert only the files you want in post.

Polyglot92 said:

I shoot Raw (NEF + Jpeg Fine) with a D5000 and am new to PP in Raw.

Every time I make the slightest change to the NEF file in ViewNX2 (even just changing the WB, sharpening or a tiny exposure correction) its size increases typically by around 60%, on average from around 10mb to 16mb (an issue for me no matter how affordable additional harddrives are - already using 3 for one original and 2 backups). And once saved, which is required to keep the changes when switching to another file, I cannot get the original file size back by reverting and saving again.

Is this the normal behaviour of NEF files in ViewNX?

How about other raw converters?

Which good raw converter/PP software allows to record the modifications with a more reasonable increase in file size, ideally in NEF format?

Or do you have to save in TIFF in non-Nikon converters?

--

D5000 - Nikkor AF-S DX 18-105mm F/3.5-5.6G ED VR - Nikkor AF-S 50mm f/1.4 G - Nikkor AF-S 70-300mm VR - Canon PowerShot S3

Click to expand...

--
35 to 4 x 5 - NPS Member

Convert RAW to tiff in ViewNX2, process tiff in whatever program pleases you and save, create jpegs as required. Don't process the RAW.

JBP said:

Not sure why you are getting that much file bloat. I use VNX for all my D700 NEF conversions to JPEG and never see that kind of percentage increase even with tweeks to the settings. A typical Raw NEF before edits is @25 MB, after it is 15-20%. Although I don't shoot Raw + JPEG which may be the issue.

Click to expand...

Apparently it is due to the difference between D700 and D5000 NEF files or the absolute increase is roughly the same, translating into different % increases with you starting out with a larger file: 10 + 6 mb for me, 25 + 5 for you. I don't see why the additional Jpeg created in the camera would have anything to do with it.

JBP said:

Also, Not sure why you are shooting Raw + JPEG, a JPEG file is embedded in the Raw data set for camera display and software display. It'll save you card and hard drive space and then you can convert only the files you want in post.

Click to expand...

1. Most of the Jpegs are fine, no need for PP - time saved right there. Or the little retouching or cropping required can be done much faster in Picasa. I really only want to touch the Raw files when difficult lighting situations require it or I had the wrong settings for a particular shot.

2. Jpegs created by ViewNX are either too large or too small - ViewNX compression options really suck.

3. I shoot a lot on longer trips and upload the keepers to the internet with my Netbook or share with friends and family for viewing on their PCs or TVs. I don't have a Raw converter on my Netbook. I doubt ViewNX would run very well on it.
--

D5000 - Nikkor AF-S DX 18-105mm F/3.5-5.6G ED VR - Nikkor AF-S 50mm f/1.4 G - Nikkor AF-S 70-300mm VR - Canon PowerShot S3

Skipper494 said:

Convert RAW to tiff in ViewNX2, process tiff in whatever program pleases you and save, create jpegs as required. Don't process the RAW.

Click to expand...

Why would I want to handle 70mb files which contain none of the camera/lens-specific data? Is that what you do?
--

D5000 - Nikkor AF-S DX 18-105mm F/3.5-5.6G ED VR - Nikkor AF-S 50mm f/1.4 G - Nikkor AF-S 70-300mm VR - Canon PowerShot S3

Skipper494 said:

Convert RAW to tiff in ViewNX2, process tiff in whatever program pleases you and save, create jpegs as required. Don't process the RAW.

Click to expand...

That's not a very good plan from the standpoint of storage space (which is a concern of the OP). TIFF files are huge.

Lightroom comes close to View/Capture NX in terms of the initial RAW conversion and is superior in almost every other respect and doesn't bloat the files.

It really depends on ones own processing flow. If you use PhotoShop you'll need to convert to tiff and best at 16 bit so file sizes can get huge. Frankly, This discussion is taking on the wrong motivation, file size versus what is best for the image. It's a bit like buying groceries and saying the size of the bag you carry out is more important that what's in the bag.

Nikon software does the best job of NEF conversion since some of the raw code is proprietary and all other raw converters are guessing at what the final output should look like. Personally I think View NX or NX2 are not a raw converters, they are simply ways of shooting raw and converting to tiff or jpeg. If you want a real raw converter then get Capture NX2 and take advantage of all the raw features as well as Nik's Control Points.

If storage is an issue then burn a DVD of all your tiff's or get bigger drives.

PS A new version of View NX2 is out today 2.1.1 that may have fixed some of your issues.

R Ortiz said:

Skipper494 said:

Convert RAW to tiff in ViewNX2, process tiff in whatever program pleases you and save, create jpegs as required. Don't process the RAW.

Click to expand...

That's not a very good plan from the standpoint of storage space (which is a concern of the OP). TIFF files are huge.

Lightroom comes close to View/Capture NX in terms of the initial RAW conversion and is superior in almost every other respect and doesn't bloat the files.

Click to expand...

--
35 to 4 x 5 - NPS Member

They are not small, but concerning quality, tiff is the way to go.

R Ortiz said:

Skipper494 said:

Convert RAW to tiff in ViewNX2, process tiff in whatever program pleases you and save, create jpegs as required. Don't process the RAW.

Click to expand...

That's not a very good plan from the standpoint of storage space (which is a concern of the OP). TIFF files are huge.

Lightroom comes close to View/Capture NX in terms of the initial RAW conversion and is superior in almost every other respect and doesn't bloat the files.

Click to expand...

A 2 tb harddisk is only 85 euro's...over here...

JBP said:

It really depends on ones own processing flow. If you use PhotoShop you'll need to convert to tiff and best at 16 bit so file sizes can get huge. Frankly, This discussion is taking on the wrong motivation, file size versus what is best for the image. It's a bit like buying groceries and saying the size of the bag you carry out is more important that what's in the bag.

Nikon software does the best job of NEF conversion since some of the raw code is proprietary and all other raw converters are guessing at what the final output should look like. Personally I think View NX or NX2 are not a raw converters, they are simply ways of shooting raw and converting to tiff or jpeg. If you want a real raw converter then get Capture NX2 and take advantage of all the raw features as well as Nik's Control Points.

If storage is an issue then burn a DVD of all your tiff's or get bigger drives.

PS A new version of View NX2 is out today 2.1.1 that may have fixed some of your issues.

R Ortiz said:

Skipper494 said:

Convert RAW to tiff in ViewNX2, process tiff in whatever program pleases you and save, create jpegs as required. Don't process the RAW.

Click to expand...

That's not a very good plan from the standpoint of storage space (which is a concern of the OP). TIFF files are huge.

Lightroom comes close to View/Capture NX in terms of the initial RAW conversion and is superior in almost every other respect and doesn't bloat the files.

Click to expand...

--
35 to 4 x 5 - NPS Member

Click to expand...

Just tried to replicate this with some D5000 raw files in VNX2 and whilst the image changed the file size did not alter. Same for my D300 as well. Must be a reason why this is happening to you?

BTW there is an updated version 2.1 released have you tried that?

https://nikoneurope-en.custhelp.com/app/answers/detail/a_id/48681

JG777 said:

Just tried to replicate this with some D5000 raw files in VNX2 and whilst the image changed the file size did not alter. Same for my D300 as well. Must be a reason why this is happening to you?

BTW there is an updated version 2.1 released have you tried that?

https://nikoneurope-en.custhelp.com/app/answers/detail/a_id/48681

Click to expand...

It's not just me. Two posts in this thread have confirmed similar or worse file size increases with ViewNX and CaptureNX:

http://forums.dpreview.com/forums/read.asp?forum=1006&message=37791647

http://forums.dpreview.com/forums/read.asp?forum=1006&message=37795060

I just did another test shooting NEF only without the additional Jpeg. And on that one the file size increase just for having applied -0.3 exposure compensation and saved the file in ViewNX 2.0.3 was from 11.5 to 20.4 mb = 77%.

Now if you have a way of saving NEF files after making adjustments without a significant increase in file size (or as you say not even any), I would very much like to understand how that is possible and if it can be done with my installation (Win 7 Pro 32bit).

I do have difficulties understanding and accepting why such huge file size increases could be justified. Something else must be going on rather than just writing simple code to the file. Different or no NEF compression (I believe some compression is applied to NEF files in the camera, if I remember well)?

I would be quite happy to continue using ViewNX for now if it wasn't for the wasted harddrive space (including the 2 backups, on up 10.000 files per year).
--

D5000 - Nikkor AF-S DX 18-105mm F/3.5-5.6G ED VR - Nikkor AF-S 50mm f/1.4 G - Nikkor AF-S 70-300mm VR - Canon PowerShot S3

Do you happen to have your camera set up to store NEF's on your memory card in lossless compressed format? If so, I suspect that on first opening them, ViewNX will expand them to their full uncompressed size, which is much bigger. With the D300, for example, a lossless compressed 12-bit NEF is typically around 13.4MB in size, while an uncompressed NEF is 19.4 MB. That seems like about the amount of increase you're describing. Once the NEF has been decompressed, I doubt that ViewNX would bother re-compressing it before saving the changes, as the main function of the compression is to save capacity on the memory card, not on the computer hard disk. I'm also not sure how editing changes may impact the ability of the lossless compression algorithm to work properly.

Ray

After looking at both the D5000 User Manual and the in-depth product review on this site, it appears that the D5000 always stores NEF's in compressed format, at around 10MB per file. The size of an NEF+JPEG file with JPEG-fine is about 16 MB. So it could be either that View NX expands the file to a standard uncompressed size, or it may be that after opening and rendering the file, it always includes a JPEG-fine along with the saved NEF. Either might explain the size expansion you're seeing. But I still don't know for sure what's happening - I'm only speculating.

Ray

And the prize goes to Ray.

You hit the nail on the head, the camera is compressing the NEF's, the software isn't. I shoot uncompressed 14 bit files and there still is a bit of file expansion but not to the degree if I shoot compressed or lossless compressed.

As discussed earlier, get some bigger drives.

Ray Ritchie said:

Do you happen to have your camera set up to store NEF's on your memory card in lossless compressed format? If so, I suspect that on first opening them, ViewNX will expand them to their full uncompressed size, which is much bigger. With the D300, for example, a lossless compressed 12-bit NEF is typically around 13.4MB in size, while an uncompressed NEF is 19.4 MB. That seems like about the amount of increase you're describing. Once the NEF has been decompressed, I doubt that ViewNX would bother re-compressing it before saving the changes, as the main function of the compression is to save capacity on the memory card, not on the computer hard disk. I'm also not sure how editing changes may impact the ability of the lossless compression algorithm to work properly.

Ray

Click to expand...

--
35 to 4 x 5 - NPS Member