M-Raw and S-Raw - a really bad idea.

Most people greatly understate the possible utility of megapixels. I like to have 45 MP just for an 8x10. On top of that, sRAW has several downsides:
I am not worthy to argues with you, you are very intelligent and I agree with you that more megapixels is better. However... right now in 2009 for business reasons I want a 20mp+ sensor that allows for making a 10mp raw file. It makes for nice customer satisfying 8x10 prints (not good enough for me and you of course). At the rate I am shooting racks of 2TB hard drives will not be enough to store all the images I want to archive for my customers. I am seriously thinking about just tossing the raw file and keeping the jpeg for storage.
 
Here's the downsampled version (with the expected contamination of aliasing artifacts added by the resampling algorithm)
I'm afraid you downsampled the upsampled sRaw version .

LOL, now that's a solid analysis.
OK, no problem. Here is 100% crop of sRAW (not resampled in anyway) compared with the downsampled raw and 100% crop raw (both from Graeme through ACR). You'll see the result is the exact same as above: aliasing at lower spatial frequencies in the sRAW:

100% sraw:



raw downsampled:



100% raw:


Anyway, I fail to see there any practical issue and I have never seen yet any user complaining about that.
Well, the opinion of whether it is a practical issue or not depends on your personal taste and how much you desire or dislike aliasing.

Some photographers pay $30,000 for the privilege of cameras that have the worst aliasing artifacts of all (MFDB), and they hold up the missing OLPF as an advantage. Others buy Foveon or pay good money to remove their AA filter for the same reason. Some are content with just adding aliasing artifacts in post production through junky downsampling filters like Nearest Neighbor.

These are not just a few rare photographers. It seems to me that most photographers like aliasing artifacts. Just as most of them like over-saturation, over-sharpening, high contrast, etc. So of course you are not going to hear any complaints about aliasing, because most people like it.

But I dislike aliasing, so to me it is considered harmful.
Let's make something clear: Graeme Nattress's diatribe against Canon's raw image data design is partly justified,
FWIW, the text of the post is mine, only the zone plate image is from Graeme.
On the other hand, his ISO 1600 only needs 9 bits to record every last ounce of data from the camera is ridiculous, and the plead for writing the image data with loss has been discredited many times.
Let's leave that for another discussion; suffice it to say that I respectfully differ from your position on the matter.
All in all, whatever you accept from that diatribe, that regards the raw data, not the sraw .
True, but the way it relates to sraw is this: if one accepts the diatribe, then raw file size becomes the same as current sraw, making it acceptable for the same conditions that people use sRAW for now. (Of course, the new sRAW would be even smaller still.)

--
Daniel
 
Most people greatly understate the possible utility of megapixels. I like to have 45 MP just for an 8x10. On top of that, sRAW has several downsides:
for business reasons I want a 20mp+ sensor that allows for making a 10mp raw file. It makes for nice customer satisfying 8x10 prints
I don't think you really want a 10 mp raw file. You want smaller raw files. sRAW is an inferior method for getting smaller files (because it has stronger aliasing as well as other disadvantages). Of course it's the only choice that Canon has given us.

--
Daniel
 
Most people greatly understate the possible utility of megapixels. I like to have 45 MP just for an 8x10. On top of that, sRAW has several downsides:
for business reasons I want a 20mp+ sensor that allows for making a 10mp raw file. It makes for nice customer satisfying 8x10 prints
I don't think you really want a 10 mp raw file. You want smaller raw files. sRAW is an inferior method for getting smaller files (because it has stronger aliasing as well as other disadvantages). Of course it's the only choice that Canon has given us.
Most here, except you, ejmartin and the OP seem to be assuming that reducing the number of pixels is the only way to reduce raw file size. Maybe if anyone here has experience of both sRAW modes and Nikon's lossy compression, we could make some sort of judgement about which was best. It would be nice to see if lossy compression produced a better small image than sRAW when downsampled, and if sRAW upsampled looked better than lossy compression - is there anyone with both Nikon and Canon who can help us out?
 
Most people greatly understate the possible utility of megapixels. I like to have 45 MP just for an 8x10. On top of that, sRAW has several downsides:
for business reasons I want a 20mp+ sensor that allows for making a 10mp raw file. It makes for nice customer satisfying 8x10 prints
I don't think you really want a 10 mp raw file. You want smaller raw files. sRAW is an inferior method for getting smaller files (because it has stronger aliasing as well as other disadvantages). Of course it's the only choice that Canon has given us.
Most here, except you, ejmartin and the OP seem to be assuming that reducing the number of pixels is the only way to reduce raw file size. Maybe if anyone here has experience of both sRAW modes and Nikon's lossy compression, we could make some sort of judgement about which was best. It would be nice to see if lossy compression produced a better small image than sRAW when downsampled, and if sRAW upsampled looked better than lossy compression - is there anyone with both Nikon and Canon who can help us out?
Some seems to think that a 10,1mp mRAW file from 7D doesn't contain enough information, but what then about a 20mp mRAW file from a 80mp 7D III? Would it still make sense to save the full 80mp (100+ MB) file? Or even worse: what about a lossy compressed mRAW file?
 
Some seems to think that a 10,1mp mRAW file from 7D doesn't contain enough information, but what then about a 20mp mRAW file from a 80mp 7D III? Would it still make sense to save the full 80mp (100+ MB) file?
20mp mRAW: 25 MB
80mp intelligent compression: 25 MB.

The compressed 80mp will have compression artifacts, sure (especially in worst cases). But they will occur at a higher spatial frequency than the 20mp file. The difference is that you can always go from 80mp to 20mp and throw away the resolution (and compression artifacts with it) in the 80mp file. But you can never go from 20mp back to 80mp, even if you decide that you are now willing to accept a few compression artifacts in compromise.

So instead of completely throwing away all high frequency detail, you just let that high frequency detail have lower quality. Both methods give you the same high quality at low frequencies, but one is more flexible.

This is not just theoretical, either. Look at REDCODE. For most common scenes, it compresses 9.5 MP into 2 MB raw files, and with typical post processing (e.g. 8 stops of dynamic range) I have a hard time seeing the compression artifacts. But even with extreme scenes or more demanding post processing, the compression artifacts generally disappear by the time you downsample to what a 2 MB raw file would have been (less than 2mp with Canon's bloat).

--
Daniel
 
Some seems to think that a 10,1mp mRAW file from 7D doesn't contain enough information, but what then about a 20mp mRAW file from a 80mp 7D III? Would it still make sense to save the full 80mp (100+ MB) file?
20mp mRAW: 25 MB
80mp intelligent compression: 25 MB.

The compressed 80mp will have compression artifacts, sure (especially in worst cases). But they will occur at a higher spatial frequency than the 20mp file. The difference is that you can always go from 80mp to 20mp and throw away the resolution (and compression artifacts with it) in the 80mp file. But you can never go from 20mp back to 80mp, even if you decide that you are now willing to accept a few compression artifacts in compromise.

So instead of completely throwing away all high frequency detail, you just let that high frequency detail have lower quality. Both methods give you the same high quality at low frequencies, but one is more flexible.

This is not just theoretical, either. Look at REDCODE. For most common scenes, it compresses 9.5 MP into 2 MB raw files, and with typical post processing (e.g. 8 stops of dynamic range) I have a hard time seeing the compression artifacts. But even with extreme scenes or more demanding post processing, the compression artifacts generally disappear by the time you downsample to what a 2 MB raw file would have been (less than 2mp with Canon's bloat).
Moreover, if no strides are made in read noise per unit area, and given that current ~20MP Canon DSLR's are only worthy of 12 bit encoding, an 80MP file starts off needing only 10 bits per pixel. Even without lossy compression, the file would be at least 30% smaller than the ~100MB that one would infer from scaling up current Canon RAW file sizes by the ratio of MP counts.

I don't know what method RED uses, but intelligent image compression algorithms look for a sparse data representation, where only a few coefficients are non-negligible. An example is JPEG-2000, which is based on a wavelet transform. Interestingly, wavelet encoding is one of the common methods of noise reduction (far better than pixel binning). One of the simplest versions is called wavelet thresholding, whereby small wavelet coefficients are set to zero (the theory being that large coefficients represent actual image data while small ones encode noise).

By using only the largest wavelet coefficients to represent the image data, one achieves a high level of compression, with a high level of fidelity to the original image, and at the same time achieving noise reduction in a manner similar to wavelet thresholding.

--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/
 
20mp mRAW: 25 MB
80mp intelligent compression: 25 MB.
You are missing the point here by acting as if the primary advantage of mRAW or sRAW is reduced file sizes.

If a photographer only needs a 10 MP image, he may only want to process a 10 MP image. The 80 MP image compressed to 25 MB will require more processing time to decompress and the photographer (or photo editor) will still be stuck with a 80 MP file once it is decompressed. Sure, it can be reduced but first the entire 80 MP image miust be decompressed and loaded. Ever heard of a short deadline?

The other potentially important advantage of the smaller RAW formats is the increased buffer depth over full RAW images. A more "intelligent" compression scheme will certainly use more processor cycles and reduce buffer depth (even when compared to full RAW). This point will be easily understood by anyone who has ever exhausted their camera,s image buffer right before the real important action happened. So, a more "intelligent" compression scheme will not fulfill the reason many use reduced RAW formats in the first place.

The answer is not always as simple as it may appear to detractors (particularly if they don't grasp the reasons for sRAW and mRAW in the first place).

--
Mike Mullen
 
20mp mRAW: 25 MB
80mp intelligent compression: 25 MB.
You are missing the point here by acting as if the primary advantage of mRAW or sRAW is reduced file sizes.

If a photographer only needs a 10 MP image, he may only want to process a 10 MP image. The 80 MP image compressed to 25 MB will require more processing time to decompress and the photographer (or photo editor) will still be stuck with a 80 MP file once it is decompressed. Sure, it can be reduced but first the entire 80 MP image miust be decompressed and loaded. Ever heard of a short deadline?
You have that backwards. It is faster to demosaic an 80mp bayer file into a 20mp image than to demosiac a 20mp bayer image. Current software is built to spend as much time as necessary doing slow and laborious calculations to extract the maximum amount of detail, because most photographers still desire more resolution. All that can be skipped if you have double the resolution to start with.
The other potentially important advantage of the smaller RAW formats is the increased buffer depth over full RAW images. A more "intelligent" compression scheme will certainly use more processor cycles and reduce buffer depth (even when compared to full RAW).
How about 100 FPS and unlimited buffer depth? That's what REDCODE does now using expensive parts, and that's what the Scarlet will do cheaply with mass-produced ASIC: 5mp at 100fps with unlimited buffer depth in a $3,000 camera that will have far lower sales volume than the Canon xxD series.

--
Daniel
 
I keep most of my pictures on my computer and share with internet friends. I do not need to print large prints, but I do like to have RAW to open in Photoshop elements so I can adjust WB and exposure there......I cannot do this with a JPEG. If it were not for this I would shoot everything in JPEG on my 20D. I hope to get a 50D for the bigger LCD for my old eyes. But I doubt I will ever need the full 15 or 18 MP RAW, although it is nice to have if I ever want it.....I will just go buy a really big CF card. So from my understanding Canon has done me a favor by having both full RAW and sRAW.
whvick
--

 
The other potentially important advantage of the smaller RAW formats is the increased buffer depth over full RAW images. A more "intelligent" compression scheme will certainly use more processor cycles and reduce buffer depth (even when compared to full RAW).
How about 100 FPS and unlimited buffer depth? That's what REDCODE does now using expensive parts, and that's what the Scarlet will do cheaply with mass-produced ASIC: 5mp at 100fps with unlimited buffer depth in a $3,000 camera that will have far lower sales volume than the Canon xxD series.
Don't try to change the subject here. Look what happens to the buffer depth on current Canon cameras when you switch from RAW to sRAW. It goes up, way up.

'nuff said.

--
Mike Mullen
 
The other potentially important advantage of the smaller RAW formats is the increased buffer depth over full RAW images. A more "intelligent" compression scheme will certainly use more processor cycles and reduce buffer depth (even when compared to full RAW).
How about 100 FPS and unlimited buffer depth? That's what REDCODE does now using expensive parts, and that's what the Scarlet will do cheaply with mass-produced ASIC: 5mp at 100fps with unlimited buffer depth in a $3,000 camera that will have far lower sales volume than the Canon xxD series.
Don't try to change the subject here. Look what happens to the buffer depth on current Canon cameras when you switch from RAW to sRAW. It goes up, way up.

'nuff said.
I did not intend to change the subject. You said that sraw had one advantage over raw and intelligently-compressed raw: greater buffer depth. I agree that it has such an advantage over raw, but I do not agree that it has such an advantage over intelligent compression. That is why I pointed to an example that has unlimited buffer depth at 100 FPS that's coming to the same price as a 5D2 (yet 5 times more pixels per second).

--
Daniel
 
Some seems to think that a 10,1mp mRAW file from 7D doesn't contain enough information, but what then about a 20mp mRAW file from a 80mp 7D III? Would it still make sense to save the full 80mp (100+ MB) file?
20mp mRAW: 25 MB
80mp intelligent compression: 25 MB.

The compressed 80mp will have compression artifacts, sure (especially in worst cases). But they will occur at a higher spatial frequency than the 20mp file. The difference is that you can always go from 80mp to 20mp and throw away the resolution (and compression artifacts with it) in the 80mp file. But you can never go from 20mp back to 80mp, even if you decide that you are now willing to accept a few compression artifacts in compromise.

So instead of completely throwing away all high frequency detail, you just let that high frequency detail have lower quality. Both methods give you the same high quality at low frequencies, but one is more flexible.

This is not just theoretical, either. Look at REDCODE. For most common scenes, it compresses 9.5 MP into 2 MB raw files, and with typical post processing (e.g. 8 stops of dynamic range) I have a hard time seeing the compression artifacts. But even with extreme scenes or more demanding post processing, the compression artifacts generally disappear by the time you downsample to what a 2 MB raw file would have been (less than 2mp with Canon's bloat).
The point as I see it is that almost all the high frequency 'detail' in the 80mp image will be noise, because the sensor 'resolves' the noise much better than the real detail (caused by the CFA and AA-filter, and also by 'secondary' things like diffraction, camera shake, lens blur etc.), so why bother with large, soft and noisy files if a cleverly downsampled (and maybe even compressed) 20mp mRAW file contains 98% of the real detail/information, with much lower noise and better per pixel sharpness than the original 80mp file? (And if the downsampling should result in a bit of moiré when shooting test charts, so be it! ;-))
 
for business reasons I want a 20mp+ sensor that allows for making a 10mp raw file. It makes for nice customer satisfying 8x10 prints
I don't think you really want a 10 mp raw file. You want smaller raw files. sRAW is an inferior method for getting smaller files (because it has stronger aliasing as well as other disadvantages). Of course it's the only choice that Canon has given us.
Yes, you are correct.
 
The point as I see it is that almost all the high frequency 'detail' in the 80mp image will be noise, because the sensor 'resolves' the noise much better than the real detail (caused by the CFA and AA-filter, and also by 'secondary' things like diffraction, camera shake, lens blur etc.),
The fact that some users will not take advantage of the increased resolution is not a reason to hamstring everyone else. We shouldn't go with the lowest common denominator. According to that logic, digicams should be 1.5 MP (6 micron pixels) instead of 15 MP (2 micron pixels). Even the most casual digicam user gets sharp 15 MP images with little noise when they shoot in sunlight. DSLR users will get sharp images with less noise even more often.
so why bother with large, soft and noisy files if a cleverly downsampled (and maybe even compressed) 20mp mRAW file contains 98% of the real detail/information, with much lower noise and better per pixel sharpness than the original 80mp file?
First, because there is no harm in doing so. Whatever file size you can get by downsampling you could also get by compressing.

Second, because the compressed high-resolution file would have far greater flexibility in post processing even if you can only use 20mp in your display (e.g. 12x18 print). Noise reduction, hot pixel removal, deconvolving diffraction, rotation, correction of aberrations such as C.A., distortion, lens-specific PSF, changing rectilinear projection/adding shift-lens correction, fisheye, etc. all benefit from having higher resolution than what you need for display.

Third, because there are plenty of users who will shoot in ample light or with a tripod, and will enjoy taking advantage of the increased resolution.
(And if the downsampling should result in a bit of moiré when shooting test charts, so be it! ;-))
:) Moiré does not show up very often in real life shots, though it already ruined several day's worth of timelapse photography that a friend did. When your computer struggles under the load of 5,000 raw frames per day of shooting, sRAW looks mighty tempting, but not after finding the shots ruined by hot pixels and moiré which would not have occurred with raw.

Moiré is not the only aliasing artifact, either. I assure you that dozens of other aliasing artifacts do show up in real life shots. Test charts just allow you to measure exactly how bad it is. But as I said earlier, most people like aliasing.
--
Daniel
 
But as I said earlier, most people like aliasing.
It's settled then!

sRAW and mRAW improve the image (at least to the preferences of most people).

--
Mike Mullen
 
I find Sraw can be useful at high ISO, the Raw (left) has been reduced to the same as the Sraw (Right).



And in this example (though the photos were not intended for comparion rather I was running low on CF space) the Sraw image is ready for use for my PC wallpaper and TV slide show. I don't bother converting to Jpeg as Vista & Win7 convert on the fly as required.

Opened in DPP, Sraw viewed at 100%, Raw at 50% then using sniptool


sRAW1 (exp -4 in ACR): I can read your argument
It's not about size, it's about IQ
--

 
Some seems to think that a 10,1mp mRAW file from 7D doesn't contain enough information, but what then about a 20mp mRAW file from a 80mp 7D III? Would it still make sense to save the full 80mp (100+ MB) file?
20mp mRAW: 25 MB
80mp intelligent compression: 25 MB.

The compressed 80mp will have compression artifacts, sure (especially in worst cases). But they will occur at a higher spatial frequency than the 20mp file. The difference is that you can always go from 80mp to 20mp and throw away the resolution (and compression artifacts with it) in the 80mp file. But you can never go from 20mp back to 80mp, even if you decide that you are now willing to accept a few compression artifacts in compromise.

So instead of completely throwing away all high frequency detail, you just let that high frequency detail have lower quality. Both methods give you the same high quality at low frequencies, but one is more flexible.

This is not just theoretical, either. Look at REDCODE. For most common scenes, it compresses 9.5 MP into 2 MB raw files, and with typical post processing (e.g. 8 stops of dynamic range) I have a hard time seeing the compression artifacts. But even with extreme scenes or more demanding post processing, the compression artifacts generally disappear by the time you downsample to what a 2 MB raw file would have been (less than 2mp with Canon's bloat).
The point as I see it is that almost all the high frequency 'detail' in the 80mp image will be noise, because the sensor 'resolves' the noise much better than the real detail (caused by the CFA and AA-filter, and also by 'secondary' things like diffraction, camera shake, lens blur etc.),
This would imply that the SNR at 80MP linear equivalent would be zero - I think this is unlikely to be the case.
so why bother with large, soft and noisy files
why bother with L series lenses, or anything else that improve image quality, because you kght destroy it with camera sahke, misfos, etc
if a cleverly downsampled (and maybe even compressed) 20mp mRAW file
but it doesn't seem to be cleverly downsampled. On the basis of the artifacts,it seems to be crudely and nastily downsampled.
contains 98% of the real detail/information,
Not at all clear that is does.
with much lower noise and better per pixel sharpness than the original 80mp file?
The noise is exactly the same, just downsampled, and the 'per pixel' sharpnes is a bit irrelevant if the pixels are a lot bigger.
(And if the downsampling should result in a bit of moiré when shooting test charts, so be it! ;-))
But if you could downsample yourself, cleverly, you might avoid the moire.

I think the OP was a bit condescending to users of sRAW mode, but it does seem, if Daniel and ejmartin are right, that Canon might have done something better.
 
Some seems to think that a 10,1mp mRAW file from 7D doesn't contain enough information, but what then about a 20mp mRAW file from a 80mp 7D III? Would it still make sense to save the full 80mp (100+ MB) file?
20mp mRAW: 25 MB
80mp intelligent compression: 25 MB.

The compressed 80mp will have compression artifacts, sure (especially in worst cases). But they will occur at a higher spatial frequency than the 20mp file. The difference is that you can always go from 80mp to 20mp and throw away the resolution (and compression artifacts with it) in the 80mp file. But you can never go from 20mp back to 80mp, even if you decide that you are now willing to accept a few compression artifacts in compromise.

So instead of completely throwing away all high frequency detail, you just let that high frequency detail have lower quality. Both methods give you the same high quality at low frequencies, but one is more flexible.

This is not just theoretical, either. Look at REDCODE. For most common scenes, it compresses 9.5 MP into 2 MB raw files, and with typical post processing (e.g. 8 stops of dynamic range) I have a hard time seeing the compression artifacts. But even with extreme scenes or more demanding post processing, the compression artifacts generally disappear by the time you downsample to what a 2 MB raw file would have been (less than 2mp with Canon's bloat).
The point as I see it is that almost all the high frequency 'detail' in the 80mp image will be noise, because the sensor 'resolves' the noise much better than the real detail (caused by the CFA and AA-filter, and also by 'secondary' things like diffraction, camera shake, lens blur etc.),
This would imply that the SNR at 80MP linear equivalent would be zero - I think this is unlikely to be the case.
so why bother with large, soft and noisy files
why bother with L series lenses, or anything else that improve image quality, because you kght destroy it with camera sahke, misfos, etc
if a cleverly downsampled (and maybe even compressed) 20mp mRAW file
but it doesn't seem to be cleverly downsampled. On the basis of the artifacts,it seems to be crudely and nastily downsampled.
contains 98% of the real detail/information,
Not at all clear that is does.
with much lower noise and better per pixel sharpness than the original 80mp file?
The noise is exactly the same, just downsampled, and the 'per pixel' sharpnes is a bit irrelevant if the pixels are a lot bigger.
(And if the downsampling should result in a bit of moiré when shooting test charts, so be it! ;-))
But if you could downsample yourself, cleverly, you might avoid the moire.

I think the OP was a bit condescending to users of sRAW mode, but it does seem, if Daniel and ejmartin are right, that Canon might have done something better.
Well, time will tell, but it'll probably take a while before we'll have a 80mp camera to play with. In the meantime it'll be interesting to see how 7D's RAW and M-RAW compares, and there's also the 5DII with its 10mp sRAW1 (or sRAW2?) output. (Doesn't Daniel have a 5DII?)
 

Keyboard shortcuts

Back
Top