Why the huge range of raw file sizes?

Started 1 month ago | Discussions thread
ForumParentFirstPrevious
Flat view
Henry Richardson Forum Pro • Posts: 19,547
Why the huge range of raw file sizes?
6

I wonder why there is such a wide disparity in raw file sizes for different cameras with the same number of megapixels (or very close)? I understand that some may use no compression, some may use lossless compression, and some may even use lossy compression. Also, some are 12-bits/pixel and some are 14-bits/pixel.  And the embedded JPEG size can vary.  But, from looking at all that it doesn't explain the wide range, I think.  Here is just one example comparing Olympus and Panasonic, but further down I compare Sony, Fujifilm, Leica, Canon, and Nikon too.

Panasonic raw files are bigger than Olympus raw files. The GX7II (GX85), E-M10II, E-M10, and E-M5 are all 16mp, but the GX7II raw files are bigger:

E-M5/E-M10II/E-M10: 4608x3456 pixels, 16.1mp, embedded 3200x2400 jpeg, ~14-15mb file size generally

GX7II: 4592x3448 pixels, 15.8mp, embedded 1920x1440 jpeg, ~18-19mb file size generally

The GX7II with fewer pixels, a much smaller embedded jpeg, and lossy raw compression results in much bigger raw files than the E-M5/E-M10II/E-M10. Actually, the 16mp GX7II raw files are almost exactly the same size as the 20mp PEN-F raw files:

PEN-F: 5184x3888 pixels, 20.3mp, embedded 3200x2400 jpeg, ~18-19mb file size generally

Olympus uses lossless raw compression. Panasonic uses lossy raw compression. Olympus embeds a larger jpeg. Panasonic embeds a smaller jpeg. Yet, Olympus raw files are smaller. Weird and perplexing. I noticed the same thing with my 16mp G3 a few years ago. Here are just a few things I found concerning Olympus lossless raws and Panasonic lossy raws:

http://www.olympusamerica.com/crm/oneoffpages/crm_raw.asp

http://www.dpreview.com/forums/post/40154581

http://www.mu-43.com/threads/70068/

I noticed also that the 20mp G9 raw is much larger than the 20mp E-M1III raw:

G9: 5184 x 3888 raw + embedded 1920 x 1440 jpeg (23.1mb)

E-M1III: 5184 x 3888 raw + embedded 3200 x 2400 jpeg (16.9mb)

I wonder what is Panasonic doing or not doing that causes their raw files with smaller embedded jpegs to always be so much larger than Olympus raw files with bigger embedded jpegs? Very weird.

It turns out that 9 years ago when I first noticed this with my E-M5 and G3 kenw provided the answer and I had forgotten about that. Olympus uses lossless encoding compression and Panasonic uses lossy encoding compression:

https://www.dpreview.com/forums/post/42359338

I just looked at the dcraw source code and it appears ORF files do use Huffman coding for compression (this is a lossless compression method). More accurately they use difference encoding followed by Huffman encoding (the difference encoding transforms the data losslessly such that the Huffman encoding will be more effective at compressing).

This is definitely more effective than what Panasonic does. Panasonic does difference encoding to reduce the number of bits required for storage of most pixels and applies a lossy bit shift whenever the default number of bits is insufficient (again the Panasonic "lossy" compression is so low loss as to be inconsequential, this was explored in detail in a thread in the past year).

In general the difference in file sizes should be most extreme for a base ISO nearly black frame (Olympus will be significantly smaller than Panasonic). For a high ISO image or a bright but unclipped image the difference will be smaller.

Normally one would expect that lossy compression would be smaller than a superior lossless compression, but Panasonic really screws it up and their inferior lossy compression is actually much larger than the Olympus lossless compression.

These are 20mp except the Fujifilm is 16mp. Notice the raw file sizes:

These are 20mp except the Nikon is 21mp.

These are 16mp except the Canon is 18mp.

-- hide signature --

Henry Richardson
http://www.bakubo.com

Olympus OM-D E-M10 Olympus OM-D E-M10 II Olympus PEN-F Panasonic Lumix DC-G9
If you believe there are incorrect tags, please send us this post using our feedback form.
ForumParentFirstPrevious
Flat view
ForumParentFirstPrevious
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark MMy threads
Color scheme? Blue / Yellow