R5m2 In Camera Upscaling versus other scaling methods

[Karl_Guttag]

In a prior topic, I compared the R5m2 scaling to the R5's IBIS on a high-resolution test pattern that would cause more. The prior analysis showed the problem of rescaling a HEIF (or JPEG), which is that you have already lost a lot of "information" from the RAW file. But I know people will say, "what about a "real" picture?"

I tried to set up a "typical" person shot, a person in a group, or a portrait full-body shot. (Before this, I tried a headshot with a much bigger eye. There was almost no difference between the same scaling methods, as the eye and eyelashes were big enough that the scaling method was not much of a factor).

It was a quick and dirty setup to get a representative picture of my eye (using the R5m2's eye tracking). The picture below is to show framing and sizing, so I have blurred out the background and resized the whole image. Its not a great setup, but I only wanted the image of the one eye.


Picture to show framing with the eye shown in the red rectangle

The matrix below shows various scaling methods. I have scaled everything to the same size. The Original picture was taken with RAW. I covered the RAW to HEIF with the HEIF quality set to 10 (best). There are three areas that stand out: the eyebrows, eyelashes, and IRIS. The description of the rows is as follow:

1. For the HEIF, I have enlarged it by pixel replication (just to make the pixels bigger) and using Photoshop Bicubic scaling. You can see how the eyebrows are both lost and take on a pixelated/jaggy effect. The IRIS has lost resolution, and some of the eyelashes are all but gone.
2. This row has the R2m2 in-camera upscale from the HEIF and a simple bicubic enlargement followed by an Unsharp Mask with a radius of 1.5 at 100%. I find it very difficult to tell the difference between the two pictures.
   1. Note that not only is all the detail in the eyebrows lost, but there are some jaggy artifacts.
   2. The eyelashes are blotchy and some of all but missing
   3. There is a lack of detail in the eye
3. This row shows Photoshop Super Resolution from the original RAW file (ACR "enhance Super Resolution" works on the R5m2 CR3 files).
   1. There is a dramatic improvement in the eyelashes over the R5m2 in-camera scale
   2. There is more detail in the eyelashes
   3. There is detail in the ISIR not seen in the in-camera scaling
   4. Adobe Camera RAW is noisy and the USM sharpening enhances that noise.
4. This row shows PS bicubic scaling from a RAW file (left) and USM r1.5 100% sharpening on the right.
   1. The eyebrow detail is lost (about the same as the in-camera scaling)
   2. The eyelashes are between the in-camera scaling and Adobe Super Resolution.
   3. The Iris looks similar to the Adobe Super Resolution and is much more detailed than with the in-camera scaling.

Conclusions:

Theoretically and based on my testing, the R5m2's in-camera scaling is worthless for anything other than making an image bigger. It has been so far to eliminate IBIS High Res and try and substitute this abomination. Hopefully, Canon will, as they did with the R5, add IBIS HR with a firmware update (and save the multiple RAW files rather than a single JPEG)

Almost any decent scaling program on a computer could do better, and an upsizer works from RAW, it would be much better. The fact that the scaling only works from JPEG or HEIF makes it ridiculous to put it "in camera." Frankly, even if the in-camera works from RAW files, it would be better done with a PC.

Working from the RAW file retains some "information" in the image that is lost when the image is deBayered/converted. This extra information helps with upscaling, so it is better to upscale directly from the RAW.

 

[Alastair Norcross]

Thanks for this detailed test. I'm sure I'll never be using this feature. I find that Topaz does a pretty good job in the latest version of Photo AI, on the rare occasions when I want to upscale. Even with my 24MP R6II, I almost never want to upscale. The 45MP of the R5II will give me even fewer occasions when I want to. Perhaps the odd time when I have to crop heavily. But it's very rare that a heavy crop gives an image worth keeping.

 

[Karl_Guttag]

Thanks for this detailed test. I'm sure I'll never be using this feature. I find that Topaz does a pretty good job in the latest version of Photo AI, on the rare occasions when I want to upscale. Even with my 24MP R6II, I almost never want to upscale. The 45MP of the R5II will give me even fewer occasions when I want to. Perhaps the odd time when I have to crop heavily. But it's very rare that a heavy crop gives an image worth keeping.

Thanks,

As far as I understand it, Topaz AI does not work directly from the RAW files. This would seem to be a scaling limitation as "information" is lost in converting from RAW.

 

[Wing2]

Thanks for this detailed test. I'm sure I'll never be using this feature. I find that Topaz does a pretty good job in the latest version of Photo AI, on the rare occasions when I want to upscale. Even with my 24MP R6II, I almost never want to upscale. The 45MP of the R5II will give me even fewer occasions when I want to. Perhaps the odd time when I have to crop heavily. But it's very rare that a heavy crop gives an image worth keeping.

Thanks,

As far as I understand it, Topaz AI does not work directly from the RAW files. This would seem to be a scaling limitation as "information" is lost in converting from RAW.

I am afraid you may be misinformed or got outdated information. I have Topaz Gigapixel AI and Topaz Photo AI, Both of them work directly with RAW files.

 

[BobKnDP]

Thanks for this detailed test. I'm sure I'll never be using this feature. I find that Topaz does a pretty good job in the latest version of Photo AI, on the rare occasions when I want to upscale. Even with my 24MP R6II, I almost never want to upscale. The 45MP of the R5II will give me even fewer occasions when I want to. Perhaps the odd time when I have to crop heavily. But it's very rare that a heavy crop gives an image worth keeping.

Thanks,

As far as I understand it, Topaz AI does not work directly from the RAW files. This would seem to be a scaling limitation as "information" is lost in converting from RAW.

I'm not sure what you mean.

A RAW file has to be converted to an actual image before further processing makes sense. At least as I imagine it. A RAW has to be de-mosaiced, for one. That adds unreal information.

Photo AI has no explicit 8 and 16 bit modes, unlike Photoshop. It allows images to be exported as 16 bit TIFFs. I don't know whether those really contain proper 16 bit data. I must be an optimist, as I've always thought they were genuine.

The older Topaz trinity did an ugly job of opening CR3 files. The current software (Photo AI, Gigapixel) does much better.

 

[Karl_Guttag]

Thanks for this detailed test. I'm sure I'll never be using this feature. I find that Topaz does a pretty good job in the latest version of Photo AI, on the rare occasions when I want to upscale. Even with my 24MP R6II, I almost never want to upscale. The 45MP of the R5II will give me even fewer occasions when I want to. Perhaps the odd time when I have to crop heavily. But it's very rare that a heavy crop gives an image worth keeping.

Thanks,

As far as I understand it, Topaz AI does not work directly from the RAW files. This would seem to be a scaling limitation as "information" is lost in converting from RAW.

I am afraid you may be misinformed or got outdated information. I have Topaz Gigapixel AI and Topaz Photo AI, Both of them work directly with RAW files.

Thanks, I stand corrected. I misunderstood what they were saying on their website.

 

[Karl_Guttag]

Thanks for this detailed test. I'm sure I'll never be using this feature. I find that Topaz does a pretty good job in the latest version of Photo AI, on the rare occasions when I want to upscale. Even with my 24MP R6II, I almost never want to upscale. The 45MP of the R5II will give me even fewer occasions when I want to. Perhaps the odd time when I have to crop heavily. But it's very rare that a heavy crop gives an image worth keeping.

Thanks,

As far as I understand it, Topaz AI does not work directly from the RAW files. This would seem to be a scaling limitation as "information" is lost in converting from RAW.

I'm not sure what you mean.

A RAW file has to be converted to an actual image before further processing makes sense. At least as I imagine it. A RAW has to be de-mosaiced, for one. That adds unreal information.

The de-mosaicing process inherently loses resolution information and, in effect, low-passes/blurs the source. It combines photosites from many surrounding pixels to decide on the color and intensity of any one pixel.

There is "real" detail/resolution lost in the demosaicing. Ideally, you want to scale the from the RAW information.

Photo AI has no explicit 8 and 16 bit modes, unlike Photoshop. It allows images to be exported as 16 bit TIFFs. I don't know whether those really contain proper 16 bit data. I must be an optimist, as I've always thought they were genuine.

The older Topaz trinity did an ugly job of opening CR3 files. The current software (Photo AI, Gigapixel) does much better.

Thanks, I am trying it as I only have a 2022 Sharpener AI software. I don't think it "understands" R5m2 CR3 files yet.

 

[BobKnDP]

Thanks for this detailed test. I'm sure I'll never be using this feature. I find that Topaz does a pretty good job in the latest version of Photo AI, on the rare occasions when I want to upscale. Even with my 24MP R6II, I almost never want to upscale. The 45MP of the R5II will give me even fewer occasions when I want to. Perhaps the odd time when I have to crop heavily. But it's very rare that a heavy crop gives an image worth keeping.

Thanks,

As far as I understand it, Topaz AI does not work directly from the RAW files. This would seem to be a scaling limitation as "information" is lost in converting from RAW.

I'm not sure what you mean.

A RAW file has to be converted to an actual image before further processing makes sense. At least as I imagine it. A RAW has to be de-mosaiced, for one. That adds unreal information.

The de-mosaicing process inherently loses resolution information and, in effect, low-passes/blurs the source. It combines photosites from many surrounding pixels to decide on the color and intensity of any one pixel.

There is "real" detail/resolution lost in the demosaicing. Ideally, you want to scale the from the RAW information.

I admit that I don't know the algorithms used to de-mosaic. Apparently, there is no single standard.

Bayer filter - Wikipedia

I also don't know anything about the anti-aliasing filters present in the Canon R5. It can't be too strong, else the IBIS based super resolution feature would not have been possible.

Photo AI has no explicit 8 and 16 bit modes, unlike Photoshop. It allows images to be exported as 16 bit TIFFs. I don't know whether those really contain proper 16 bit data. I must be an optimist, as I've always thought they were genuine.

The older Topaz trinity did an ugly job of opening CR3 files. The current software (Photo AI, Gigapixel) does much better.

Thanks, I am trying it as I only have a 2022 Sharpener AI software. I don't think it "understands" R5m2 CR3 files yet.

 

[homebodyMacro]

In a prior topic, I compared the R5m2 scaling to the R5's IBIS on a high-resolution test pattern that would cause more. The prior analysis showed the problem of rescaling a HEIF (or JPEG), which is that you have already lost a lot of "information" from the RAW file. But I know people will say, "what about a "real" picture?"

I tried to set up a "typical" person shot, a person in a group, or a portrait full-body shot. (Before this, I tried a headshot with a much bigger eye. There was almost no difference between the same scaling methods, as the eye and eyelashes were big enough that the scaling method was not much of a factor).

It was a quick and dirty setup to get a representative picture of my eye (using the R5m2's eye tracking). The picture below is to show framing and sizing, so I have blurred out the background and resized the whole image. Its not a great setup, but I only wanted the image of the one eye.

The matrix below shows various scaling methods. I have scaled everything to the same size. The Original picture was taken with RAW. I covered the RAW to HEIF with the HEIF quality set to 10 (best). There are three areas that stand out: the eyebrows, eyelashes, and IRIS. The description of the rows is as follow:

1. For the HEIF, I have enlarged it by pixel replication (just to make the pixels bigger) and using Photoshop Bicubic scaling. You can see how the eyebrows are both lost and take on a pixelated/jaggy effect. The IRIS has lost resolution, and some of the eyelashes are all but gone.
2. This row has the R2m2 in-camera upscale from the HEIF and a simple bicubic enlargement followed by an Unsharp Mask with a radius of 1.5 at 100%. I find it very difficult to tell the difference between the two pictures.
   1. Note that not only is all the detail in the eyebrows lost, but there are some jaggy artifacts.
   2. The eyelashes are blotchy and some of all but missing
   3. There is a lack of detail in the eye
3. This row shows Photoshop Super Resolution from the original RAW file (ACR "enhance Super Resolution" works on the R5m2 CR3 files).
   1. There is a dramatic improvement in the eyelashes over the R5m2 in-camera scale
   2. There is more detail in the eyelashes
   3. There is detail in the ISIR not seen in the in-camera scaling
   4. Adobe Camera RAW is noisy and the USM sharpening enhances that noise.
4. This row shows PS bicubic scaling from a RAW file (left) and USM r1.5 100% sharpening on the right.
   1. The eyebrow detail is lost (about the same as the in-camera scaling)
   2. The eyelashes are between the in-camera scaling and Adobe Super Resolution.
   3. The Iris looks similar to the Adobe Super Resolution and is much more detailed than with the in-camera scaling.

Conclusions:

Theoretically and based on my testing, the R5m2's in-camera scaling is worthless for anything other than making an image bigger. It has been so far to eliminate IBIS High Res and try and substitute this abomination. Hopefully, Canon will, as they did with the R5, add IBIS HR with a firmware update (and save the multiple RAW files rather than a single JPEG)

Almost any decent scaling program on a computer could do better, and an upsizer works from RAW, it would be much better. The fact that the scaling only works from JPEG or HEIF makes it ridiculous to put it "in camera." Frankly, even if the in-camera works from RAW files, it would be better done with a PC.

Working from the RAW file retains some "information" in the image that is lost when the image is deBayered/converted. This extra information helps with upscaling, so it is better to upscale directly from the RAW.

One request: take the same baseline photo at half ish the distance to match the framing of the upscaled images. That way, you can see how real or fake the invented "details" are from the upscaling algorithms.

Also, I am philosophically inclined to dislike the R5II upscaling by its lack of an empirical basis for inventing features, but I must also say: I'm actually shocked at how bad the results are in the handful of examples I've seen.

Even one that was in marketing copy in another thread looked like trash. I'm open to counterexamples, but so far it looks like a dumpster fire that belongs in the dumpster. Literally worse than no upscaling.

Thank you for sharing your testing!

 

[Ali]

...

Conclusions:

Theoretically and based on my testing, the R5m2's in-camera scaling is worthless for anything other than making an image bigger. It has been so far to eliminate IBIS High Res and try and substitute this abomination. Hopefully, Canon will, as they did with the R5, add IBIS HR with a firmware update (and save the multiple RAW files rather than a single JPEG)

Almost any decent scaling program on a computer could do better, and an upsizer works from RAW, it would be much better. The fact that the scaling only works from JPEG or HEIF makes it ridiculous to put it "in camera." Frankly, even if the in-camera works from RAW files, it would be better done with a PC.

Agreed that this feature seems pretty useless - my observations with cat and lizard as well. I still want to try some landscapes and the moon, but I don't have high hopes.

Maybe Canon targets this feature mostly for the event/sports photographer who takes whole streams of JPEGs and uploads them from the field. Especially in cases where cropping may be needed this gives them a quick way to get 4x photos. The way the feature is set up to work in bulk may also support this usage pattern.

If so, it is a different feature altogether from IBIS high res, and we can still hope that feature comes back someday.