Sony: why does the A7R IV have no lossless compressed RAW option?

Started 4 weeks ago | Discussions
Brick Wall
Brick Wall Regular Member • Posts: 311
Sony: why does the A7R IV have no lossless compressed RAW option?
29

Listen up, Sony: I am completely baffled as to why, in 2019, you are still releasing cameras that have no lossless compressed RAW option.

No one enjoys 120MB A7R IV files. A single 4 sec second burst would take up the size of a DVD, on which they used to store whole entire high resolution movies!

Is the problem a lack of money? Know-how? Some combination of the two? Would you like me to start a Kickstarter campaign begging for donations on your behalf? That would be pathetic, wouldn't it?

Whatever the case might be, it's high time you got your act together.

And yes, that applies to your long-standing star eater problem, as well.

You have had more than enough time to address these issues. My patience is wearing thin.

-- hide signature --

"Gotta catch em all," he muttered. "Gotta catch em all,"...

Sony a7R
If you believe there are incorrect tags, please send us this post using our feedback form.
EarthQuake Senior Member • Posts: 2,797
Re: Sony: why does the A7R IV have no lossless compressed RAW option?
33

It's like an Onion article.

Man buys camera with more megapixels than anyone could ever possibly need, complains about file size.

Astrophotographer 10 Forum Pro • Posts: 13,235
Re: Sony: why does the A7R IV have no lossless compressed RAW option?
2

Is it technically very difficult to make a lossless compression? Or is it heavily protected by Patents? Don't know. It is an obvious omission so perhaps they are working on it and its not ready yet. It will become an achilles heel if they continue to increase mp without one.

With regards to the star eater Sony seems to have an attitude of its not important and to concentrate on what is important to the bulk of its market rather than the relatively small portion.

Although I note on these forums the large percentage of posts about cameras and saying they have an interest in shooting some nightscapes/astro. Its quite a common thing.

I also note Canon is rumoured to be releasing an astro version of the EOS R. Now they own this section of the market and its served them well over the years.

I wonder also if the star eater noise filter were able to be turned off are Sony worried that internet trolls will say how noisy their BSI sensors really are?

Greg.

 Astrophotographer 10's gear list:Astrophotographer 10's gear list
Sony a7R II Sony a7R III Sony FE 55mm F1.8 Zeiss Batis 85mm F1.8 Zeiss Loxia 21mm F2.8 +1 more
JimKasson
JimKasson Forum Pro • Posts: 26,237
Re: Sony: why does the A7R IV have no lossless compressed RAW option?
7

Astrophotographer 10 wrote:

Is it technically very difficult to make a lossless compression?

No.

Or is it heavily protected by Patents?

No.

Don't know. It is an obvious omission so perhaps they are working on it and its not ready yet. It will become an achilles heel if they continue to increase mp without one.

With regards to the star eater Sony seems to have an attitude of its not important and to concentrate on what is important to the bulk of its market rather than the relatively small portion.

Although I note on these forums the large percentage of posts about cameras and saying they have an interest in shooting some nightscapes/astro. Its quite a common thing.

I also note Canon is rumoured to be releasing an astro version of the EOS R. Now they own this section of the market and its served them well over the years.

I wonder also if the star eater noise filter were able to be turned off are Sony worried that internet trolls will say how noisy their BSI sensors really are?

Greg.

-- hide signature --
 JimKasson's gear list:JimKasson's gear list
Nikon D5 Sony a9 Sony a7 III Nikon Z7 Fujifilm GFX 100 +4 more
Luis Gabriel Photography
Luis Gabriel Photography Senior Member • Posts: 2,153
Re: Sony: why does the A7R IV have no lossless compressed RAW option?
4

Brick Wall wrote:

Listen up, Sony: I am completely baffled as to why, in 2019, you are still releasing cameras that have no lossless compressed RAW option.

No one enjoys 120MB A7R IV files. A single 4 sec second burst would take up the size of a DVD, on which they used to store whole entire high resolution movies!

Is the problem a lack of money? Know-how? Some combination of the two? Would you like me to start a Kickstarter campaign begging for donations on your behalf? That would be pathetic, wouldn't it?

Whatever the case might be, it's high time you got your act together.

And yes, that applies to your long-standing star eater problem, as well.

You have had more than enough time to address these issues. My patience is wearing thin.

When a company does not give me the product that fits my needs, I buy something else that fits my requirements better...simple.

 Luis Gabriel Photography's gear list:Luis Gabriel Photography's gear list
Sony a9 Sony a7R III Sigma 135mm F1.8 Art Sony FE 12-24mm F4 Sigma 35mm F1.2 DG DN +3 more
SQLGuy Veteran Member • Posts: 7,658
Re: Sony: why does the A7R IV have no lossless compressed RAW option?
2

A lot of the good compression algorithms are proprietary (license controlled and patented), or GPL (which Sony probably wouldn't want to use here). But there is an Apache Commons Compression library, and the ASF licenses are very easy to work with.

On the other hand, Nikon is only getting about 30% savings in file size with their lossless compression, so Sony may have felt that it just wasn't worth it... saving time would probably be about the same, and loading time would be longer, and it's more work for one of the CPUs.

-- hide signature --

A7R2 with SEL2470Z and a number of adapted lenses (Canon FD, Minolta AF, Canon EF, Leica, Nikon...); A7R converted to IR.

 SQLGuy's gear list:SQLGuy's gear list
Canon PowerShot G9 Canon PowerShot S100 (2000) Canon EOS-1D Konica Minolta Maxxum 7D Nikon D200 +17 more
jcb9001 Regular Member • Posts: 114
Re: Sony: why does the A7R IV have no lossless compressed RAW option?

A simple run-length encoding algorithm is easy. But, it is yet another step to perform in the journey from chip to buffer to memory card. My guess (just a guess, mind you) is that even a simple algorithm would take too long to do on a 61 MP image. That, or they experimented with it and found the extra time spent isn't worth the average compression ratio.

Horshack Veteran Member • Posts: 7,970
Because BIONZ X doesn't support it
11

Sony's existing raw compression logic likely resides in their image processing ASIC, which they call BIONZ X. From what I can tell Sony hasn't upgraded this chip in quite a few generations, probably to save on R&D. Recent advancements in processing speed and AF appear to have been implemented in a second ASIC they added, which they generically call a "Front End LSI". As a result we won't see enhancements to their raw compression algorithm until they rearchitect BIONZ.

Chris Crevasse Contributing Member • Posts: 700
Re: Sony: why does the A7R IV have no lossless compressed RAW option?

I know that we (and I am including me in that "we") frequently complain about Sony's lossy compressed raw, but it seems that Sony has decided the benefits of its lossy raw outweigh the detriments.  Are we certain that Sony is wrong?

 Chris Crevasse's gear list:Chris Crevasse's gear list
Sony RX1 Panasonic Lumix DMC-GF1 Sony a7R II Canon EF 50mm f/1.2L USM Canon EF 85mm f/1.2L II USM +10 more
JimKasson
JimKasson Forum Pro • Posts: 26,237
Re: Sony: why does the A7R IV have no lossless compressed RAW option?
10
  1. Chris Crevasse wrote:

I know that we (and I am including me in that "we") frequently complain about Sony's lossy compressed raw, but it seems that Sony has decided the benefits of its lossy raw outweigh the detriments. Are we certain that Sony is wrong?

It isn’t a matter of right and  wrong. It’s what’s appropriate for the situation. Craw is fine for most things, but not for everything. Nikon offers six different raw file formats.

Choice is good. Procrustean beds aren’t.

-- hide signature --
 JimKasson's gear list:JimKasson's gear list
Nikon D5 Sony a9 Sony a7 III Nikon Z7 Fujifilm GFX 100 +4 more
Horshack Veteran Member • Posts: 7,970
Re: Sony: why does the A7R IV have no lossless compressed RAW option?
10

Chris Crevasse wrote:

I know that we (and I am including me in that "we") frequently complain about Sony's lossy compressed raw, but it seems that Sony has decided the benefits of its lossy raw outweigh the detriments. Are we certain that Sony is wrong?

The benefits of a lossy algorithm are space savings, so Sony is right about that. However others like Nikon have implemented lossy algorithms with similar space savings but without the perceptual artifacts visible in Sony's algorithm. And for that reason Sony is wrong wrt to the implementation choice of their lossy algorithm. And Sony is also wrong for not providing a losslessly compressed algorithm, both for the corner cases where IQ issues exist even for a good lossy algorithm and also just as an engineering backup in case issues with the algorithm are discovered later...as was the case in Sony's existing lossy algorithm.

Brick Wall
OP Brick Wall Regular Member • Posts: 311
Re: Sony: why does the A7R IV have no lossless compressed RAW option?

Luis Gabriel Photography wrote:

Brick Wall wrote:

Listen up, Sony: I am completely baffled as to why, in 2019, you are still releasing cameras that have no lossless compressed RAW option.

No one enjoys 120MB A7R IV files. A single 4 sec second burst would take up the size of a DVD, on which they used to store whole entire high resolution movies!

Is the problem a lack of money? Know-how? Some combination of the two? Would you like me to start a Kickstarter campaign begging for donations on your behalf? That would be pathetic, wouldn't it?

Whatever the case might be, it's high time you got your act together.

And yes, that applies to your long-standing star eater problem, as well.

You have had more than enough time to address these issues. My patience is wearing thin.

When a company does not give me the product that fits my needs, I buy something else that fits my requirements better...simple.

Years ago, due perhaps in large measure to user feedback, Sony finally came out with an uncompressed RAW option.

Hopefully, they will listen again. I'm a big fan of the progress Sony has made with its FF mirrorless. Like many others here, I'd them to address a gap in their feature-set.

Having to store and back up larger than necessary files--whether locally or to the cloud--does nobody any favors.

Jim K and Horshack have said it best: choice is good.

-- hide signature --

"Gotta catch em all," he muttered. "Gotta catch em all,"...

mujana Veteran Member • Posts: 6,032
Re: Sony: why does the A7R IV have no lossless compressed RAW option?

Can lossless compressed raw be implemented by firmware update/upgrade?

 mujana's gear list:mujana's gear list
Sigma DP3 Merrill Sony a7R IV Zeiss Batis 85mm F1.8 Sony FE 16-35mm F2.8 +5 more
SigZero
SigZero Regular Member • Posts: 385
Re: Because BIONZ X doesn't support it

Horshack wrote:

Sony's existing raw compression logic likely resides in their image processing ASIC, which they call BIONZ X. From what I can tell Sony hasn't upgraded this chip in quite a few generations, probably to save on R&D. Recent advancements in processing speed and AF appear to have been implemented in a second ASIC they added, which they generically call a "Front End LSI". As a result we won't see enhancements to their raw compression algorithm until they rearchitect BIONZ.

I was for long time also convinced that this is Bionz-X limitation and without new image processor it won't happen. I understand that there might be no place (additional buffer, compute power) to lossless compress data stream - in contrast Sony cRAW is linear, has fixed compression ratio, can be processed "in-line". But on the other hand - it is the same chip that is able to produce varied compression ratio JPGs with different scaling and picture effects, so there is a compute power available (I won't be offended if lossless RAW compression disables the JPG output). The question is if the CXD4236 is programmable to that degree that new compression protocol can be implemented by firmware update. Let's remember that uncompressed RAW option was added by firmware update - so there is at least some programmability there.

Br, Pawel

-- hide signature --

My photo-blog: http://pawel.online

 SigZero's gear list:SigZero's gear list
Sony FE 85mm F1.8 Sony FE 100-400mm F4.5-5.6 Sony FE 24-105mm F4 Sony a7R IV
Magnar W
Magnar W Senior Member • Posts: 4,155
Re: Sony: why does the A7R IV have no lossless compressed RAW option?

Astrophotographer 10 wrote:

Is it technically very difficult to make a lossless compression? Or is it heavily protected by Patents?

Maybe slowing down the speed of the camera too much? A source for tons of new complains ...

With regards to the star eater Sony seems to have an attitude of its not important and to concentrate on what is important to the bulk of its market rather than the relatively small portion.

How many do use these cameras, or system cameras, for scientific grade astro work?

Also, for many kind of astrophoto, faint stars are removed, and bright stars are made smaller, so that deep sky objects stand better out. Even Photoshop has a filter for doing this.

How much do stars looking like hot pixel add to amateur night sky photos taken for aesthetic purposes? For me, I rather want to get rid of these stars anyway.

Although I note on these forums the large percentage of posts about cameras and saying they have an interest in shooting some nightscapes/astro. Its quite a common thing.

The spatial filtering is a non-issue for this group of users. They don't use a tracker to avoid elongated star, and they don't wait for night with good seeing to capture as faint stars as possible, which also take some carefully post processing if you want to keep them.

I also note Canon is rumoured to be releasing an astro version of the EOS R. Now they own this section of the market and its served them well over the years.

I wonder also if the star eater noise filter were able to be turned off are Sony worried that internet trolls will say how noisy their BSI sensors really are?

Probably. The best solution might be better spatial filtering algorithms.

 Magnar W's gear list:Magnar W's gear list
Sony a7 Sony a7R III Sony FE 90mm F2.8 macro Zeiss Loxia 21mm F2.8 Zeiss Loxia 35 +2 more
joger
joger Veteran Member • Posts: 5,596
surprise - surprise - 60.2 MP have larger file sizes ;-)
3

EarthQuake wrote:

It's like an Onion article.

Man buys camera with more megapixels than anyone could ever possibly need, complains about file size.

YMMD !! 

no further arguments needed!

Years ago I met a guy who put Ferrari sized tires on his Porsche and complained about the tire price. I guess that is the kind of cognitive dissonance I simply don't understand.

Before I buy something I think twice, three times or sometimes 20 times and sometimes  wait quite a long time to really be sure about the purchase.

E.g. I did not buy the Batis 135 even though I wanted so badly a fast native mount 135 mm lens and same goes with the G 200 .. 600 - I wanted so badly a long native mount sharp tele lens. Both wishes just took a bit longer than wanted but the result is two outstanding lenses that fully satisfy in their targeted uses cases.

I am looking forward to the A9 II which might get 16 bit readout at 36 MP - maybe this is the better camera compare to the A7R IV - tests will proof how good these two new contenders are - at this point in time it might be wise to wait a bit longer to fully understand the technical implications of the sensors.

To be surprised by the file size on a 60.2 MP camera is a bit - well - surprising

We live in fantastic times - so many new tools and segments being addressed that everybody should find his/her best suitable device.

Once we have the A9 II we can all lean back and wait for the initial comparisons.

File size is personally my least concern of all.

Samsung recently released their spec of the new stacked flash chips. Very soon 512 GB UHS-II cards will cost less than today's fast 128 GB cards.

just my 2 CT

-- hide signature --

__________________________________
A7R III - one camera to rule them all
ISO 9000 definition of quality: 'Degree to which a set of inherent characteristic fulfills requirements'
I am the classic “Windows by Day, Mac by Night user'
“The horizon of many people is a circle with zero radius which they call their point of view.” Albert Einstein
"In the beginning the Universe was created. This has made a lot of people very angry and been widely regarded as a bad move." Douglas Adams

 joger's gear list:joger's gear list
Sony a7R III Zeiss Loxia 21mm F2.8 Sony FE 85mm F1.4 GM Sony FE 135mm F1.8 GM Sony FE 200-600 F5.6-6.3 +9 more
Horshack Veteran Member • Posts: 7,970
Re: Because BIONZ X doesn't support it

SigZero wrote:

Horshack wrote:

Sony's existing raw compression logic likely resides in their image processing ASIC, which they call BIONZ X. From what I can tell Sony hasn't upgraded this chip in quite a few generations, probably to save on R&D. Recent advancements in processing speed and AF appear to have been implemented in a second ASIC they added, which they generically call a "Front End LSI". As a result we won't see enhancements to their raw compression algorithm until they rearchitect BIONZ.

I was for long time also convinced that this is Bionz-X limitation and without new image processor it won't happen. I understand that there might be no place (additional buffer, compute power) to lossless compress data stream - in contrast Sony cRAW is linear, has fixed compression ratio, can be processed "in-line". But on the other hand - it is the same chip that is able to produce varied compression ratio JPGs with different scaling and picture effects, so there is a compute power available (I won't be offended if lossless RAW compression disables the JPG output). The question is if the CXD4236 is programmable to that degree that new compression protocol can be implemented by firmware update. Let's remember that uncompressed RAW option was added by firmware update - so there is at least some programmability there.

Br, Pawel

The algorithm would be baked into the ASIC so aside from some possible parameters that firmware can feed into that logic it's likely immutable. Uncompressed was likely already supported by the logic and just unlocked when Sony had to respond to the complaints about their raw compression algorithm.

moffatross
moffatross Senior Member • Posts: 1,114
Re: Sony: why does the A7R IV have no lossless compressed RAW option?
4

JimKasson wrote:

Choice is good. Procrustean beds aren’t.

Jim, I had to look it up its meaning and provenance, but thanks I love that word and have added it to my vocabulary.

Cheers, Ross

splashy
splashy Veteran Member • Posts: 3,127
Re: Sony: why does the A7R IV have no lossless compressed RAW option?

When the specs of the camera went public the first people where talking about was the huge size of the files.

Nobody is forced to buy this camera, but when you buy it, this camera is years ahead in time, so don't complain about it. If you want smaller files select extra fine jpeg.

Yes it might fill up a DVD in a few seconds shot, not so long ago we used (floppy) disks and back then we had the same problem.

https://www.google.com/search?source=hp&ei=dFGDXYw0iLeTBYCqlIAK&q=capacity++floppy+disk&oq=capacity++floppy+disk&gs_l=psy-ab.3..0j0i22i30l9.2224.2224..3546...0.0..0.72.72.1......0....2j1..gws-wiz.YAQTKabZmkU&ved=0ahUKEwiMvoGN0NzkAhWI26QKHQAVBaAQ4dUDCAU&uact=5

It will be a question of years to find this filesize for a picture normal and we will need faster computers and more chips space to handle this size fast.

 splashy's gear list:splashy's gear list
Sony a7R III Sony FE 28mm F2 Sony FE 24-240mm F3.5-6.3 OSS Sony FE 50mm F1.8 Sony FE 85mm F1.8 +2 more
mostlyboringphotog Veteran Member • Posts: 8,884
Re: Sony:... compress your hard drive (n/t)

 mostlyboringphotog's gear list:mostlyboringphotog's gear list
Panasonic Lumix DMC-GM1 Pentax 645Z Nikon 1 J5
Keyboard shortcuts:
FForum MMy threads