State of the art in upsizing algorithms?

Started Jun 9, 2014 | Discussions thread
Mark Scott Abeln
OP Mark Scott Abeln Forum Pro • Posts: 15,716
Frequency domain

JimKasson wrote:

Mark Scott Abeln wrote:

Is there any research on going directly from a mosaiced Bayer raw file to an upsampled image? Lately I’ve gotten comfortable with doing my own processing in Octave with good results. Clearly there will be at least some data loss when doing the demosaicing and upsizing steps independently.

I've long thought that was the way to go, but I haven't seen any evidence of commercial tools working that way. Demosaicing involves building a model of the scene based on sparse samples, and that model should be valuable in producing an upsampled result; it seems a shame to throw it away.

Yes it does seem to be a shame. But every time I think about how some aspect of image processing could be improved, the solution seems to be turning the image processing pipeline into a mathematical construct of Byzantine complexity. Not that it is necessarily a bad thing, just that it is difficult for me to conceive.

Take a look at some of the frequency-domain methods in this survey paper:

If you're adventurous, you could use one of the frequency domain methods and effectively do your resampling when converting back to the xy domain. However, my experience with image frequency domain processing is quite limited, and I have seen minor divergence from expected results, even with 64-bit floating point precision.

If you stay in the space domain, there's a possible win in not going to 16-bit integer precision between demosaicing and resampling. That win should be more significant as the scaling methods and sharpening filters get more complex.

My experience with frequency domain processing was disappointing, but I admit that I really don’t remember most of the math, except vaguely. Certainly, loss of precision is important, and I’m seeing a number of places where lack of precision in general harms the final image. Perhaps this is a good opportunity to experiment with extended precision arithmetic in Mathematica or whatever else is out there — since I’m concerned with a one-off image, long processing times is hardly an issue.  However, I do need speed for the typical large project that I do a couple times a year.

My current favorite raw processing software is Raw Therapee, which still has some problems, sadly, particularly the white balance when using CIECAM02. It can use 64 bit processing, and I’m generally pleased by the fine gradation of tones that it can create when doing shadow brightening.

 Mark Scott Abeln's gear list:Mark Scott Abeln's gear list
Nikon D200 Nikon D7000 Nikon D750 Nikon AF-S DX Nikkor 35mm F1.8G Nikon AF Nikkor 50mm f/1.8D +2 more
Post (hide subjects) Posted by
(unknown member)
(unknown member)
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark MMy threads
Color scheme? Blue / Yellow