Reading mode: Light Dark

Uncompressed RAW and DNG converter

Go to last post
L

lorenzo190447

Well-known member
Messages
194
Reaction score
110
Location
IT
I just updated the FW of my A7rii and I was testing the new uncompressed RAW.

In my workflow I always convert the ARW files in DNG with DNG converter.

The strange thing is that if I convert 2 ARW files, one compressed and one uncompressed I get 2 DNG files that have the same file size (43MB).

I just updated the DNG converter but, even if in the preference I did not select any compression, I stil get 2 file of the same size!

Maybe we must only wait for a new update!
 
I found the same. The converted file are suspringly small at 29MB but can be opened by DNG editing programs. Also the folder view of the DNG converter shows no files to convert but when you tell the program to convert the contents of the selected folder all files are listed and converted. So the question is, what have we lost? :)

Bob.
 
BobP said:
I found the same. The converted file are suspringly small at 29MB

Bob.
Click to expand...
29MB? mine are 43,4MB very strange!
 
My guess would be that the DNG converter has to de-compress the lossy compressed file before it can convert it.

So the converted versions of both files will be the same size. The DNG files are then probably saved using a lossless compression such as LZW.

Any artefacts from Sony's lossy compression will still be there in the DNG file.
 
silentstorm said:
People are crying for uncompressed RAW so that they can DNG compress it?

The world must be coming to an end!! LOL!!!
Click to expand...
Yes you're missing something very important and crucial to the discussion. DNG offers lossless compression. Sony were enforcing mandatory lossy compression, which can be bas as data is lost. With lossless compression no data is lost so there is practically no downside.

However Sony aren't offering lossless compression with this firmware update - only uncompressed files, hence people are turning to DNG.
 
The result of compressing one uncompressed RAW to a ZIP file is 59MB so I'm not sure how a lossless 40MB file can be created without loosing something.
 
mgrum said:
silentstorm said:
People are crying for uncompressed RAW so that they can DNG compress it?

The world must be coming to an end!! LOL!!!
Click to expand...
Yes you're missing something very important and crucial to the discussion. DNG offers lossless compression. Sony were enforcing mandatory lossy compression, which can be bas as data is lost. With lossless compression no data is lost so there is practically no downside.

However Sony aren't offering lossless compression with this firmware update - only uncompressed files, hence people are turning to DNG.
Click to expand...
Perhaps you should drop ProfHankD a private message regarding DNG "lossless" compression.

There was a discussion here not long ago regarding this.

Loading…

www.dpreview.com

And Jose Viegas made a reply just below. Its a good point for discussion.

"The result of compressing one uncompressed RAW to a ZIP file is 59MB so I'm not sure how a lossless 40MB file can be created without loosing something."
 
silentstorm said:
Perhaps you should drop ProfHankD a private message regarding DNG "lossless" compression.

There was a discussion here not long ago regarding this.

http://www.dpreview.com/forums/post/56608165
Click to expand...
If you look at the thread you linked to you'll see that I do indeed discuss the issue with Hank. The conversion is not lossless at the bit level (it appears there's a normalisation step applied before the data is losslessly compressed), but it's worlds apart from Sony's lossless compression.
silentstorm said:
And Jose Viegas made a reply just below. Its a good point for discussion.

"The result of compressing one uncompressed RAW to a ZIP file is 59MB so I'm not sure how a lossless 40MB file can be created without loosing something."
Click to expand...
That could be explained by the fact DNG uses a compression algorithm designed for images (ITU-T81) as opposed to an algorithm designed to handle arbitrary data types (DEFLATE).
 
BobP said:
I found the same. The converted file are suspringly small at 29MB but can be opened by DNG editing programs. Also the folder view of the DNG converter shows no files to convert but when you tell the program to convert the contents of the selected folder all files are listed and converted. So the question is, what have we lost? :)

Bob.
Click to expand...
I've discovered the reason for the 29MB DNGs is that the camera Picture Profile was set to s-log2, with the profile set to OFF they're approximately 44MB. I'd previously been taking large dynamic range landscapes.

Bob.
 
I still don't get it... why make a DNG at all? Just keep it uncompressed.... Makes NO sense to me other than saving a little space.... but wasting a TON of time. Just delete the rejected photos after culling... smh
 
Zip has no knowledge about 14-bit align for a single information. Zip tries always to work on 8 bit informations, or multiples of 8 bit. So Zip is quite ineffective for a 14-bit RAW.
 
mgrum said:
silentstorm said:
Perhaps you should drop ProfHankD a private message regarding DNG "lossless" compression.

There was a discussion here not long ago regarding this.

http://www.dpreview.com/forums/post/56608165
Click to expand...
If you look at the thread you linked to you'll see that I do indeed discuss the issue with Hank. The conversion is not lossless at the bit level (it appears there's a normalisation step applied before the data is losslessly compressed), but it's worlds apart from Sony's lossless compression.
mgrum said:
And Jose Viegas made a reply just below. Its a good point for discussion.

"The result of compressing one uncompressed RAW to a ZIP file is 59MB so I'm not sure how a lossless 40MB file can be created without loosing something."
Click to expand...
That could be explained by the fact DNG uses a compression algorithm designed for images (ITU-T81) as opposed to an algorithm designed to handle arbitrary data types (DEFLATE).
Click to expand...
"The conversion is not lossless at the bit level..." meaning?? It is lossy? That normalisation step before compression is making the process lossy right? That's how I interpret your words.

I agree with ProfHankD more, Adobe is throwing away things that it doesn't understand or doesn't know how to interpret & process, to me that is a lossy compression. Upon decompressing from DNG, you can't revert back to original, to me that is lossy too. Perhaps this is the reason why LR/ACR can't reproduce the same hue gradations as the original software provided by their respective co. namely Canon DPP, Nikon Capture NX, Sony IDC & Oly Viewer.

BTW Sony new firmware says it is uncompressed, not losslessly compress. I think there is a thread too discussing about why can't Sony offer the 3rd option of losslessly compressing the RAW like canikon do.
 
Jose Viegas said:
The result of compressing one uncompressed RAW to a ZIP file is 59MB so I'm not sure how a lossless 40MB file can be created without loosing [sic] something.
Click to expand...
There are data compression techniques specifically for 2-D images that can achieve better compression ratios than can be attained by general-purpose data compression techniques for arbitrary data. And visually lossless image data compression techniques can do even better by e.g. not wasting space for noise.
 
silentstorm said:
"The conversion is not lossless at the bit level..." meaning?? It is lossy? That normalisation step before compression is making the process lossy right? That's how I interpret your words.
Click to expand...
It means that the bits you get out of converting to DNG and decompressing are not exactly the same bits as you get from decoding the original file. Now that alone doesn't mean the operation is lossy.

If it is as Hank suggests a scaling operation then it will be very very slightly lossy, in that the values you get back could be +/- 1DN, where rounding has occurred. In a situation where the minimum noise is almost 1DN you are highly unlikely to see this error in a file, especially if you are viewing an 8-bit representation, which you almost certainly will be.

More important is the loss of metadata, although depending on your workflow that too might make zero difference.

Maybe both of these things are important to you, in which case I wouldn't recommend converting to DNG.
silentstorm said:
BTW Sony new firmware says it is uncompressed, not losslessly compress. I think there is a thread too discussing about why can't Sony offer the 3rd option of losslessly compressing the RAW like canikon do.
Click to expand...
Yes, it's more work for them, possibly exacerbated by the fact that lossless compression was not in the spec when the hardware designs were finalised.
 
ThomasFromBavaria said:
Zip has no knowledge about 14-bit align for a single information. Zip tries always to work on 8 bit informations, or multiples of 8 bit. So Zip is quite ineffective for a 14-bit RAW.
Click to expand...
The 14-bit pixel values are expanded to 16-bit words in uncompressed RAW files.
 
silentstorm said:
People are crying for uncompressed RAW so that they can DNG compress it?

The world must be coming to an end!! LOL!!!
Click to expand...
You know DNG has lossless compression right? Sony's compression is lossy.
 
So, how big are these new uncompressed RAW files?

The regular RAW was consistently around 42MB for me, wondering how much it's grown now.
 
Cheyne said:
So, how big are these new uncompressed RAW files?

The regular RAW was consistently around 42MB for me, wondering how much it's grown now.
Click to expand...
 
Cheyne said:
So, how big are these new uncompressed RAW files?
Click to expand...
Cheyne said:
The regular RAW was consistently around 42MB for me, wondering how much it's grown now.
Click to expand...
About 83MiB. The per-pixel data doubles in size, from one byte per pixel to two bytes per pixel. The fixed overhead and small JPEG preview image are of course unchanged in size.

Yes, it's big. Yes, it's lame.
 
You must log in or register to reply here.

Keyboard shortcuts

f
Forum
m
My threads
Mobile site
About
Editorial content
Cameras & Lenses
Community
All content, design, and layout are Copyright © 1998–2025 Digital Photography Review All Rights Reserved.
Reproduction in whole or part in any form or medium without specific written permission is prohibited.
When you use DPReview links to buy products, the site may earn a commission.
©GPS Media - Guides, Products, Services.
Back
Top