Resampling sharp Foveon images

Hi all,

Deep buried in a thread some of us discussed a very important topic, resampling of pixel sharp images.

Resampling you do if you make some distortion of the image, e.g. restitution, rotation, stitching, lens distortion correction, etc

Even resizing involves resampling, but its not all that relevant for this discussion.

---

The problem with resampling is loss of Foveon super resolution.

Foveon (without AA and correctly focussed with a good lens and held steadily) can produce stunning sharpness. The aliasing you get will even further enhance the feeling for sharpness. If you can stand the aliasing, then you get a look that some people think is to die for.

Now, if you resample (and not at the same time is scaling up the image greatly) then it is impossible to maintain that sharpness. You just have to think about all 1 pixel large details, dots, lines, edges and complicated patterns. If you resample those, trying to put the image in another pixel grid, they will totally break. I think you can imagine what happens.

Now, this is not only theories. One poster claimed that he did not lose any detail while stitching Foveon images. So - he showed an example. And ... there was lots of loss of detail. Lots of small things that just was blur in the stitched image. And everything else is impossible.

---

There is one solution, as hinted above. And thats upscaling. While doing the distortion of the image you also upscale. Then its possible to maintain details. Now, of course, if it is pixel sharpness you want, then upscaling will destroy that. But ... the details will still be there.

But stitching and upscaling at the same time? Woa! That will result in big images!

---

There is another solution: downscaling. Then you will lose detail, but you will get pixel sharpness. You choose.

--
/Roland
X3F tools:
http://www.proxel.se/x3f.html
https://github.com/rolkar/x3f

when making panoramas with Foveon Cameras.

And I shot much over 1000 Panoramas with SD9,SD10,SD14,SD15, DP1. Even several Gigapixel Panos.

I have never seen any sharpness or detail loss.

So I doubt it will be diffrent when using the Merill Imager.

Sadly the new generation of Sigma Cameras is not really useful for massive Panorama work. As they are so slow.

--
This review is from: Nikon F6 Professional SLR 35mm Film Camera Body (Electronics)
Camera will not work with CompactFlash or other digital media cards. You must buy a cartridge of tape, which allows for just 24 shots. No LCD screen for image playback. Extremely frustrated and returned item.

Resampling is not a problem if the image does not contain frequencies over the Nyquist frequency.

In fact if all frequencies are below that limit you can perfectly resample the image (using a high order Lanczos filter). Resampling an image with contains frequencies higher than that frequency will either introduce (ringing) artifacts or suppress high frequencies (= blur the image).

Bayer images are usually unable to capture information near that frequency (which we all detest as Bayer blur) while Foveon sensors still have a good frequency response (which we all love as Foveon sharpness). From which it follows that resampling a Bayer image will not imply such a loss of information or introduction of artifacts as resampling a Foveon image.

So much for the signal theory... I am not stating anything about the impact in the practice.

Joerg

With today's huge image sizes, downsizing might be important for a few people.

Hands up all those who alwaysdownsize "bicubic sharper"

 . It compares the Photoshop methods with examples, advocates a little smoothing before downsizing and proves by example why that is necessary.hereOne of the better articles on downsizing can be read 
Has anybody used ImageMagick?

--

Best Regards,
Ted.

If you mean aliasing, then yes, you can't resample/recreate aliasing artifacts successfully.

If these is no aliasing in source images (what means that you have no pixel-size elements either), then you in theory do not lose information while resampling; unfortunately you have to very carefully choose subsequential sharpening algorithm. SPP can do wonders on original raw data, creating for example pixel-sized elements from non-aliased data; for resampled images you can't use SPP.

I have very minor experience with resampling.
Sometimes I need to rotate images a bit; I have yet to found workflow, which doesn't kill original sharpness. Silkypix, which should be able resample data on raw level, can't retain original sharpness (actually it can't often match SPP results anyway).
Sure original images are little aliased too, SD14 pixels are big enough

For panos I'm usually giving up some sharpness, no problem for PC-only images.

Are you having troubles with foveon images in your lens corrector?

--
Arvo
Sigma/Foveon information collection and little galleries:
http://www.stv.ee/~donq/sigma.htm
http://www.stv.ee/~donq/images.htm

As you said the solution is up-sampling before any operations are done, and in the end don't down-sample, just keep the image size. That's what's done to satellite imagery, planetary imagery by probes, or any other field of science where images must be geometrically corrected but all the information is precious.

Upsampling 2x is more than enough, and even 1.5x is ok if you use good algorithms after.

If you upscale by exactly doubling the width and height, there will be no loss of pixel sharpness. Each pixel becomes four identical pixels.

After stitching or distorting, you could then halve the width and height.

Yes - I mean aliasing. Foveon sharpness is based on aliasing.

My post really is meant to remind the users that this (false) sharpness does not survive resampling.

OK - all correct.

Yes, you have to. Those believing otherwise are wrong. If you want to preserve the aliased sharpness you have to leave the image alone regarding resampling. Thats it!

The same kind of problems as when doing panoramas of course But ... I assume that if you scale the image up, maybe a factor 2, then it is not a big problem.

But, I dont have any Foveon camera, so it does not affect me, except ... that Pentax now has a DSLR without AA filter. That camera will have the same kind of problems and ... the AA filters you find in Bayer cameras are not perfect.

--
/Roland
X3F tools:
http://www.proxel.se/x3f.html
https://github.com/rolkar/x3f

Hmmmmm ... do you mean upscaling with bicubic or with nearest neighbour?

No, you cant halve the width. Then you get the same problem as just doing resampling without upscaling.

--
/Roland
X3F tools:
http://www.proxel.se/x3f.html
https://github.com/rolkar/x3f

This is beacuse you have not looked close enough. If the original image is super sharp, a la Foveon aliasing, then it is impossible (without serious upscale) to map it to another grid without losing detail.

You can take 3 times as few images and get the same resolution. Do you say that the new cameras are 3 times slower than the old ones?

--
/Roland
X3F tools:
http://www.proxel.se/x3f.html
https://github.com/rolkar/x3f

Sounds reasonable

--
/Roland
X3F tools:
http://www.proxel.se/x3f.html
https://github.com/rolkar/x3f

I'm confused, Roland.

Could you take a minute to explain your view of the relationship between sharpness and aliasing as it applies to Foveon-based images?

Thanks,

--
Best Regards,
Ted.

For a full spherical panorama I need 8 shots.  Which took a minute with a SD9 SD10, SD14, SD15 Than the SD1 is even slower

When doing the same as HDR I need 24 shots.

How much longer will I need compared to a SD15?

When doing a Gigapixel with 500 Shots, I needed around 2 hours with a SD14/SD15, will I even finish this in one day with a SD1?

Not to talk about all the fuzz with moving clouds during wait time etc.

Those new Cameras are single shot cameras sadly.

And I looked very close on my pictures. Back in 2003/2004 stitching was a bit more complicated than nowadays and I always controlled each panorama for it's quality. And they where always good enough that they where shown on trade shows like PMA at the Sigma booth. They where also good enough that even a feature about my work was written in the Rangefinder Magazine. So I stand for it, there is zero loss in quality when stitching a panorama.

By the way my NpParallaxpoints are mesured for a 10th of a millimeter, are those people which are complaining also using a dedicated Panohead and exact values for the NPP?

In my experience mostly errors in panos are user errors and you will see a dramataic loss in quality when doing freehand panos. Just because of parallax errors.

--
This review is from: Nikon F6 Professional SLR 35mm Film Camera Body (Electronics)
Camera will not work with CompactFlash or other digital media cards. You must buy a cartridge of tape, which allows for just 24 shots. No LCD screen for image playback. Extremely frustrated and returned item.

Hi Roland,

Here we have a classic issue between theory and practice. In theory you are correct, however, remember that with an electronic image we can actually "see" the pixels. In "practice," even though the initial incredibly "sharp" pixels are "mutated," we can't actually "see" that result in the print.

Once we get from the electronic pixel level view to the printer, things are quite different because unless we are printing billboards, we really never reach a situation where individual "pixels" are visible. Huge prints from Foveon captures which are printed at A0 and larger can be visually flawless even when using fairly common and mundane interpolation such as bicubic.

Using more sophisticated interpolation algorithms such as available with PhotoZoom Pro, (Max Spline, etc.) even the often maligned "aliasing" can almost always be totally ameliorated to the point of being inconsequential.

As you know, printers do not create tiny little "rectangles" of ink, they variously create lines or "dots" and they print on paper or other media which does not perfectly hold the shape of the tiny "dot" or line but bleed very slightly so that what our eyes eventually see on the printed result and what our eyes see on the electronic display are measurably different.

I too have made numerous panoramas and I have never been able to detect any deterioration in the printed result. Yes, if one zooms in to 400% it's possible to see a deterioration in the pixel, but what can easily be seen on an electronic display just doesn't pertain to the printed result in my experience. Foveon panoramas are, in my experience, measurably "better" than my CFA panoramas in terms of "sharpness".

The widely circulated large prints from Rick and Laurence are ample testimony to the quality of "resampled" Foveon large prints.

Best regards,

Lin

--
learntomakeslidshows.net

Hi Lin,

Your post makes the classic assumption that everyone here sends their work to a printer. Some of us only view and publish on-screen and a very tiny few (probably only me) post their shots cropped, but not re-sized. In which case sharpness can be a concern at pixel level.

In fact, someone here (guess who) once put a lot of time and effort into researching which watch shot is sharper: 2268x1512 re-sized, or 1134x756 straight out of the camera.

--
Best Regards,
Ted.

I am not surprised that you are confused. This is not easy stuff. For a good read on the sampling theorem, you can read this.

http://en.wikipedia.org/wiki/Sampling_theorem

In order to make an image that can be reconstructed you have to band limit the signal before sampling. But ... that is not enough. You also have to use a rather elaborate method for reconstruction. And, of course, then you can only reconstruct the band limited version of the original.

Now ... no one I know of uses the sinc function trick for reconstructing images. Moreover, I am quite sure of that the sinc trick assumes the output space is continuous, and not sampled, which of course is totally bogus for reconstruction in computers.

So - this my take on real world image sampling. The sampling theorem is some kind of theoretical triva. In practice you dont use the tricks mentioned there ... you only look at the pixels ... or some smoothed and/or rescaled version. So you get rather different behaviour.

What will happen in the real world sampling is that each pixel will record the value it sees at that coordinate. And ... if you have no AA filter ... and a very sharp lens ... this reading will alias heavily. This means that details that are much smaller than the pixel can heavily affect the outcome.

A typical example of this is grass. In a camera with AA filter, grass (at some distance) will be a green mush, without any detail. But, without an AA filter you will get structure. This structure is false, but (according to some) it looks better. The same goes for the needles in conifers, and maybe also the leaves in trees with leaves. And also the small leafless branches in trees in the winter.

All this is irregular details. When it comes to regular patterns, then you might get problems.

Just a word of warning. A grating of frequency F has MUCH higher frequencies than F. If you band limit that grating to F you will get a sinus with frequency F, and when reconstructing it will still be a sinus. This sinus wil also just have a fraction of the energy of the original grating. Hardly looks like a grating. It might even be almost invisible.

--
/Roland
X3F tools:
http://www.proxel.se/x3f.html
https://github.com/rolkar/x3f

You are totally correct Lin. And your opinion correspond to the one of Laurence. And actually to most sensible photographers You take your tools. Do the best that you can. And if it is good - it is good. Thats it really.

My post only affects those claiming that they need the superior Foveon sharpness My message to those is that they shall be weary of resampling. Resampling do NOT preserve that sharpness.

Then - just as Arvo says - the sharpness left might be more than enough.

--
/Roland
X3F tools:
http://www.proxel.se/x3f.html
https://github.com/rolkar/x3f

Without studying all the theory of sampling, I found this experiment might give indication of the effects in practice.

• 1. Take a Foveon Merrill shot of leaves, around a meter away. You can run the experiment on a subject more relevant to you.
• 2. Open a TIF out of SPP in Fast Stone, the donation-only viewer program.
• 3. Rotate the image by a small angle and save it as a new file.
• 4. Use the FastStone Compare operation to look at both files side by side at 100%.

My eye could not see a difference. However, I noticed that my viewing angle was important. If you are directly in front of one image then look at the other image at an angle off the perpendicular, you might see a difference in some feature. When I moved my head directly in front of the second image, it looked just like the first image looked.

Thanks Roland,

The penny dropped with that last paragraph which reminded me of my Hi-Fi audio days, where your grating would have been equivalent to a square-wave test input containing all the odd (or was it even?) harmonics of the fundamental frequency. Similarly, sawtooth waveforms.

It also explained (indirectly) why even a perfect (artificial) sharp edge presented to an MTF program still gives a curve of approx 64% value at Nyquist. And why the author in the link I posted earlier used a sine-wave target for his examples.

--
Best Regards,
Ted.

Some programs have a simple algorithm for doubling which just repeats each pixel four times. That's the one you want.

Bicubic and other clever algorithms would give some blur.

Note that this second paragraph did _not_ include "there will be no loss of sharpness". Those who find the big file size too much may be ready to lose a little sharpness.