Reading mode: Light Dark

Does RAW use lossy compresion?

Go to last post
M

Michal Szymanski

Member
Messages
34
Reaction score
0
Location
Gdansk, PL
One thing is for me unclear.
When I make RAW photo using my D70 I get approximately 6MB file
but when I calculate
3000x2000 pix * 36bit/pix (because RAW use 12bit/channel) = 25MB

25MB is almost 5 times higher than 6MB. Does RAW stores compressed image ?

I’ve noticed that for landscape with many detail (like forest) even for RAW small artifacts can be observed and sharpness of the picture is smaller than for picture with small numbers of details.

Can somebody give me answer how data are stored in RAW?

--
Michal Szymanski
http://podroze.szymanskich.net?lang=eng
 
Michal Szymanski said:
One thing is for me unclear.
When I make RAW photo using my D70 I get approximately 6MB file
but when I calculate
Click to expand...
There's not much point in trying to calculate. It's a lot more complex than you think.
Michal Szymanski said:
3000x2000 pix * 36bit/pix (because RAW use 12bit/channel) = 25MB
Click to expand...
12 bits per channel, but only one channel per pixel.
Michal Szymanski said:
25MB is almost 5 times higher than 6MB. Does RAW stores compressed
image ?
Click to expand...
Most RAW files use a compression scheme similar to the one used in a zip file, where there is compression but no information is discarded. RAW files from some Nikon cameras have an additional compression where some information is discarded, but afaik it's not really noticeable unless you try to do some very drastic highlight recovery.

There's a more detailed explanation here:
http://www.majid.info/mylos/weblog/2004/05/02-1.html

RAW files can also contain camera settings, a custom tone curve, EXIF info, and an embedded JPEG, just to complicate the matter further.
Michal Szymanski said:
I’ve noticed that for landscape with many detail (like forest) even
for RAW small artifacts can be observed and sharpness of the
picture is smaller than for picture with small numbers of details.
Click to expand...
What kind of artifacts? You might see maze/demosaic artifacts depending on which RAW converter you use.
Michal Szymanski said:
Can somebody give me answer how data are stored in RAW?
Click to expand...
http://www.dpreview.com/learn/?/Glossary/Digital_Imaging/RAW_01.htm
http://www.dpreview.com/learn/?/Glossary/Camera_System/sensor_linearity_01.htm
Michal Szymanski said:
--
Michal Szymanski
http://podroze.szymanskich.net?lang=eng
Click to expand...
 
I think Thom Hogan wrote something on this - you might want to search his site.

All I know is the D70 uses Huffman encoding on the final file, which remaps how colours are stored, so that the most common are mapped to a representation using the least number of bits, and so on (short, crummy explanation - if you want a more detailed one, ask). I know that this is lossless, and that you can get very high compression rates with this. I think I remember reading something about quantisation curves, but I can't remember, and don't know anything about this.

For waht it's worth, Nikon does say that it is lossless though ...
 
Michal,

there is a slight error in your calculations: the total amount of information captured by the sensor is 3008x2000 pix x 12 bit/pix, since each sensor pixel captures monochromatic information, with the colored filter in front deciding what kind of color that is (R, G or B).

Now, the RAW format D70 produces is indeed compressed. I don't remember the exact details, but there is a mapping from the 4096 discreet steps of 12bit ADC to 850 values that can be stored in the RAW file. There is some special algorithm that governs the distribution of those 850 sampling points, so that they are spread according to the sensor's sensitivity.

I remember seeing a reference to Thom Hogan's book for more info on that subject. You might want to look that up if you need more detailed information.

Regards,
--
Anestis Georgiadis
Web Gallery: http://pbase.com/mranest
 
In RAW each pixel have only one color.

It's during demosaic you get all three RGB-values for each pixel by using the values from the surrounding pixels.

So, the RAW data is 3008x2000x12bit, and the file also contains meta data, plus a fullsize JPEG Basic and a thunbnail.

That said, there is both lossy and lossless compression in compressed NEF files.
Michal Szymanski said:
One thing is for me unclear.
When I make RAW photo using my D70 I get approximately 6MB file
but when I calculate
3000x2000 pix * 36bit/pix (because RAW use 12bit/channel) = 25MB

25MB is almost 5 times higher than 6MB. Does RAW stores compressed
image ?

I’ve noticed that for landscape with many detail (like forest) even
for RAW small artifacts can be observed and sharpness of the
picture is smaller than for picture with small numbers of details.

Can somebody give me answer how data are stored in RAW?

--
Michal Szymanski
http://podroze.szymanskich.net?lang=eng
Click to expand...
 
Michal Szymanski said:
It means that real resolution of the picture is 1/3 of 6Mpix. I
know that camera uses special matrix to record RGB chanels but I
thought that 6Mpix = 6M*3 sensors.
Click to expand...
Sorry to burst your bubble but virtually all sensors have one color channel per photo site. Nearly all use a bayer pattern of red-green-blue-green. The Foveon sensor used in Sigma cameras has three colors per photosite. Sigma argues the same point that you are, that the camera is effectively 10.6 million sensor photo detectors while having only 3.4M pixels. .
 
3008 * 2000 12 = 72192000 bits
72192000 bits / 8 bits/byte = 9024000 bytes
9024000 bytes / 1024^2 bytes/MB = 8.6 MB

EACH PICTURE = 8.6 MB with no data compression.

No much compression to reach 6MB per picture, and with no losses as I believe.

José Luis (Spain)
 
joselu said:
3008 * 2000 12 = 72192000 bits
72192000 bits / 8 bits/byte = 9024000 bytes
9024000 bytes / 1024^2 bytes/MB = 8.6 MB

EACH PICTURE = 8.6 MB with no data compression.

No much compression to reach 6MB per picture, and with no losses
as I believe.
Click to expand...
The compression is lossy. Nikon's site says the compression is visually lossless, or small enough loss that you can't tell. I don't know whether that is true or not, because the D70 can't produce non-lossy NEF's as well to compare.
joselu said:
José Luis (Spain)
Click to expand...
--
D70 and photo discussion
D70 custom tone curves @ http://forum.mastersphoto.net
 
padeye99656 said:
three colors per photosite. Sigma argues the same point that you
are, that the camera is effectively 10.6 million sensor photo
detectors while having only 3.4M pixels. .
Click to expand...
In real life you, however, get about the same resolution from a 3Mpix FOVEON as you get from a 6Mpix Bayer Pattern sensor.
 
Michal Szymanski said:
It means that real resolution of the picture is 1/3 of 6Mpix. I
Click to expand...
No, it means that if there is only green light in the picture then only 3 million photocells will capture anything useful. If there's only red light then only 1.5 million photocells will see anything, and so on. If the picture is monochrome then all the photocells pick up information.
Michal Szymanski said:
know that camera uses special matrix to record RGB chanels but I
thought that 6Mpix = 6M*3 sensors.
Click to expand...
No, 6MP means 6 million photosites, 50% green, 25% red, 25% blue. I think the de-mosaic algorithm used to generate RGB data can vary, but it's not as simple as 1 R + 1 G + 1 B = 1 pixel. The R/G/B values for each pixel are interpolated in from several surrounding photosites.
Michal Szymanski said:
--
Michal Szymanski
http://podroze.szymanskich.net?lang=eng
Click to expand...
 
mszerg said:
Sorry, I'm the unbeliever :-)

please, give me the link where Nikon states that compression is
VISUALLY/VIRTUALLY/etc losless
Click to expand...
This is not from the mouth of Nikon, but it is close enough

http://www.bythom.com/D70REVIEW.HTM

The Compressed NEF format is the only one that comes close to retaining the full data set the D70 is capable of acquiring. I say "close to retaining" because the compression scheme Nikon uses is not lossless. Basically, the camera takes the highlight data and places them into groups (essentially a rounding of many of the data points), producing the equivalent to somewhere between 9 and 10 bits of data. When this is returned to 12-bit form, there's a bit of posterization in the highlight data. The reason this works as a visually lossless scheme is that our eyes really can't resolve more than about an 8-bit value can produce (and our eyes aren't linear in response to light, either). For the most part it isn't a big thing that the compression loses data, though there may be some post-processing manipulations that will render slightly differently because of the data rounding.

*********************

Warm regards,
Dad-of-four
Looking for a Smugmug account?
Enter code kHmP122izH9gg in the Referred by field to save $5.
 
pity, but not accepted, because original poster said "Nikon's site says".

what is said about compressed NEFs over the internet I know well, but that's not Nikon ;-)

...and I think that compression is really lossless, but quntization(or how do they call it) is the limitation of hardware processing algorithms, so uncompressed NEFs are the same "virtually" lossless. That's my opinion, cause I don't see any reason(even marketing purpose) to make it "virtually" lossles from lossles, even for 10% of used space reduction.

--
Best regards
http://mszerg.photosight.ru
 
Ok, but in the letest update it's mentioned about ADC which can work this way.

So the point is NOT IN COMPRESSED or uncompressed NEF, but in d70 itself.

So Nikon says truth when stating that NEF compression is lossless :-) but d70 uses slightly lossless ADC for recording data.

--
Best regards
http://mszerg.photosight.ru
 
You must log in or register to reply here.

Keyboard shortcuts

f
Forum
m
My threads
Mobile site
About
Editorial content
Cameras & Lenses
Community
All content, design, and layout are Copyright © 1998–2025 Digital Photography Review All Rights Reserved.
Reproduction in whole or part in any form or medium without specific written permission is prohibited.
When you use DPReview links to buy products, the site may earn a commission.
©GPS Media - Guides, Products, Services.
Back
Top