State of the art in upsizing algorithms?

Started Jun 9, 2014 | Discussions thread
hjulenissen Senior Member • Posts: 2,269
Re: State of the art in upsizing algorithms?
1

JimKasson wrote:

I've long thought that was the way to go, but I haven't seen any evidence of commercial tools working that way. Demosaicing involves building a model of the scene based on sparse samples, and that model should be valuable in producing an upsampled result; it seems a shame to throw it away.

If this model (or something equally good) can be built from the demosaiced image, then one could equally well do the processing in stages. The question is if there is some unique information that needs to be propagated between the traditional blocks.

I think that demosaicing would benefit from insight into the noise, but also from knowing how the denoised samples would look like. I think that deconvolution might benefit from something similar. So perhaps some iterative process could reduce the problem that every stage would ideally like to work on the improved output (or not) of every other stage?

If you had access to a practical camera (say, a D800) and an impractical camera (say, a D800 using 20x20 stitched images at base ISO and using some multi-spectral pre-filter), then you could generate two images of the same scene. One test-image, and one reference. The question then is what nonlinear, signal-adaptive transformation (i.e. raw development) would make the test image as similar as possible to the reference image (in some numeric or perceptual fashion).

If you're adventurous, you could use one of the frequency domain methods and effectively do your resampling when converting back to the xy domain. However, my experience with image frequency domain processing is quite limited, and I have seen minor divergence from expected results, even with 64-bit floating point precision.

I see "the frequency domain" as just a linear transform. So why should we expect something like a 2d DFT to matter much for image quality (processing speed is a different matter). Why not some time/space transform tradeoff (wavelet, filterbank,...)?

I believe that straight image FFT processing often has issues with padding. Specifically, the inherent assumption that the input is infinitely periodically repeated gives less than ideal results.

Perhaps those more into statistics than myself could offer some ideas. If each image sensel is a filtered, noisy sample of some process of specific stochastic parameters (unknown, but perhaps guessable, hopefully locally stationary), then one might be able to "guess" the image (possibly at a denser grid) while minimizing some expected error criterion. Again, I am sceptical (even if you could do the mathematics) that minimizing a simple error will end up looking good.

Or is this all moot, current cameras provide "good enough" accuracy under good conditions, and we all end up doing unrealistic colors, over-sharpening etc because that is what we want in the first place?

-h

Post (hide subjects) Posted by
(unknown member)
(unknown member)
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark MMy threads
Color scheme? Blue / Yellow