Dotphoton Raw / 'Rawsie' - any opinions on this raw compression witchcraft?

Hmm... yet another proprietary lossy compression scheme?  Unless you become the standard, is there a market for that? And if you don't, will folks be able to open their files in 10 years? Even then...

These applications are all using the new Adobe DNG Cinema standards (1.5.0 I think), so I would guess they use 10 or 12 bit gamma encoding, and then either lossless or JPEG encoding. JPEG encoding has since ~2004 (again from memory) supported 8/10/12b data in the standard, the problem has been that very few encoding/decoding engines has chosen to include that option.

The only compatibility problem I can see is if they've chosen to include some of the options that RED cinema has chosen to start a patent disagreement about....

Quality-wise, the differences are down to the +/-1 value on a 12-bit gamma encoded file. Think of it as a maximum (approximately!) of a 1-bit error if an AdobeRGB or Prophoto RGB was encoded in a file format that supported 12-bit encoding.

The problem there isn't in the highlights, where rounding errors in color space translations and photon noise even in the best of sensors at base ISO are a factor of ~100 larger than the encoding error. The problem with "destroying captured data" is at, and around the blackpoint.

AFAIK the DNG standard sets a blackpoint and truncates data at that point, meaning that read-noise from the electronics in the camera, that can be both negative and positive, gets all the negative values truncated to zero in a truly "black" area. This creates some problems that are certainly measurable, but I've yet to see it affect a real photo used for any real purpose other than measurements. That's even considering that many cameras today are good enough to accurately capture single-photon values when averaged over a smallish area.

Most of the Cinema DNG basics are covered here in Adobe's spec sheet.

I wouldn't be to worried about quality, but as others have mentioned - you might lose some universality in what softwares you can use to decode the raws in a few years. All of Adobe's programs will certainly continue to support the format, and programs like Capture One too.

KyleMayPhoto wrote:

Hey folks, first post here. Looking for opinions on Dotphoton Raw from people with real knowledge of this stuff before I blow £80 on a Windows pre-order. Apparently it can reduce RAW files down to JPEG sizes without any loss of quality or information. Something to do with removing 'quantum noise' which sounds a bit snake-oily:

https://rawsie.co

Looks like they recently rebranded - Rawsie is a terrible name though

It hasn't had much coverage but the stuff I've seen has been positive, but for some crazy reason it was Mac-only (ugh) so I've not been able to try it.

I'm still a bit dubious about the tech. Maybe the change isn't perceptible with normal shooting but I do some fairly extreme pushes of shadows and exposing for the highlights, I'm not convinced it wouldn't throw away at least some of the detail. Anyone here tried it?

Always keep the original raw files.

One of the reasons is that metadata is very much unknown. Moving it to a non-standard location, messing with offsets, or deleting "unnecessary" metadata can render it useless, especially in perspective.

Same goes to engineering areas like optical black. It must be kept intact.

Finally, changing raw data is also not a good thing because future software may be capable of using the original raw data in better ways.

Your archival format needs to be the original raw.

Hey Martin,

Thanks for posting the Fstoppers article. It's still a great one, but it was the first review ever and the product evolved quite a lot since then. There are more reviews out there, I'm not posting it here out of respect for the rules of the forum.

As to rebranding: Rawsie is a product by Dotphoton, it used to be called Dotphoton Raw before Spring 2020. We decided to rename it as it was a bit difficult to pronounce and memorize. As to the domain zone, it's just one of those fancy short domains lots of startups use today, nothing to do with Columbia (probably our website builder service is hosted in UA, not quite sure), but we're still based out of Switzerland.

Hey Jack, the output files are DNG, no decompression or active license is needed to use them as outlined in the first section of our FAQ.

We're pretty sure that in 10 years from now your chances of being able to open Rawsie-compressed files are absolutely the same, if not higher, than in your camera's proprietary format, as dng is used not only by just one manufacturer. Don't you agree?

Hello Mark, they are indeed. The software is mostly designed for photographers who take 100-200k+ images a year, some of them have contractual obligations to store raw files. At the same time, they tend to upgrade their cameras every now and then, while the average raw file size is not getting smaller from year to year obviously. But it's not only about storage, people find various benefits of using it.

Hi The_Suede, I am one of the devs of 'Rawsie', sorry for the late reply, I was just made aware of your post. Indeed, we use the lossless JPEG coder already present in the DNG SDK. Coding is done over the native bit-depth of the camera (DNG supports up to 16 bits). The way that we achieve better results is that we calibrate each individual camera, at each iso and most shutter speed, in the lab. From that we generate custom coding per camera and per iso. The quality is not bit-for-bit lossless, but the error generated with respect to the noise is approximately 1/10th of that of lossy JPEG at the same file size.

You are completely right about black point: one must be particularly careful around it because, as you mention, even if the signal is smaller than the noise, it can be recovered by denoisers, or even visually by the human eye. We found that it's best to keep values around the black point untouched, as anyway they don't eat up much space.

As far as I see, values below the black point are preserved in the DNG, and it's then up to the development software to decide what to do about them, I agree with you that it's best not to clip them.

About RED and patent issues, my understanding is that DNG pre-dates the RED patents. My superficial understanding is that the RED patent is quite broad, but that even though some of their claims may not hold, Sony and Apple implemented systems that were remarkably similar to RED's i.e. Split the R,G,B image planes, apply amplitude scaling per-channel, apply wavelet compression, de-compress and scale back, with the purpose of having compression errors become invisible to the human eye.

Personally, I hope that DNG will remain supported for a long time. Most raw files, including DNG, are basically TIFF. The DNG SDK is open source, and, although it seems to lack some documentation in the code, once you understand it, it is rather nice to use. Somehow, all my favourite cameras use it: Ricoh, Sigma, Leica. Also, of course, I do love to be able to easily open the files and try to write my own image development software :-), after working with DaVinci Resolve for video, no photo editing software makes sense to me anymore ...

Just tested compressed dngs with originals in Capture One 20 and Adobe Camera Raw. In ACR there is no visible shift in color and brightness, but in C1 there is very visible. So, it doesnt fit to my c1 workflow :/

boris-rawsie wrote:

Hey Jack, the output files are DNG, no decompression or active license is needed to use them as outlined in the first section of our FAQ.

We're pretty sure that in 10 years from now your chances of being able to open Rawsie-compressed files are absolutely the same, if not higher, than in your camera's proprietary format, as dng is used not only by just one manufacturer. Don't you agree?

Difficult to predict the future is...

Best regards

Erik