A problem with RAW files

[peweuk]

I haven't used RAW and was just beginning to experiment with it.
I have taken a number of shots over the last week or two, using RAW+JPG.

Most of the RAW shots are OK, but there are a few which display the 'clipping' you can see in the corners of the image below which is a screen capture of an FZ150 RAW file displayed in Raw Therapee (latest version 4.0.5.2) on my PC.

The camera processed JPG version of the image is OK and shows no problems, and the RAW file opens with no clipping in Photoshop Camera RAW.

I have no idea what is causing this and all the examples where this appears seem to be when using 25mm Focal Length.

Can anyone with RAW experience explain what might be going on?

 

[David247]

That is odd looking. To be honest, that looks like the petal lens hood is twisted somewhat and not properly locked in place. With the lens hood petals aligned diagonally, it will create that exact effect at max wide angle, 25mm, and not at longer focal lengths.

But why it would show on the RAW files and not on the JPEG I can't explain unless the camera somehow recognized the issue on the image and cropped the image some for the JPEGs.

--
David
Galleries: http://dwrobinson.zenfolio.com

 

[Detail Man]

David247 wrote:

That is odd looking. To be honest, that looks like the petal lens hood is twisted somewhat and not properly locked in place. With the lens hood petals aligned diagonally, it will create that exact effect at max wide angle, 25mm, and not at longer focal lengths.

Spot on. A "proboscis problem" from the get-go. The lens-hood on the FZ28 (with only one single little threaded screw part used to affix the lens-hood to the lens-assembly) loves to work itself loose, and I have learned to watch it "like a hawk" prior to each and every individual usage (especially at full wide-angle):





It does seem odd, though, that this is (for the most part) only appearing on the right-hand side of the recorded image-frame. (At first blush), this would seem to only occur in the case of a lens-hood that was so loose on it's "mounts" that it was "detaching" more so on the right-hand (as opposed to the left-hand) side of the lens-assembly ...

But why it would show on the RAW files and not on the JPEG I can't explain unless the camera somehow recognized the issue on the image and cropped the image some for the JPEGs.

The RW2 image-files have extra pixels (photo-sites, actually, prior to de-mosaicing into "pixels") available that RAW Therapee 4.x (similar to DxO 6.x) is very likely accessing.

While ExifTool shows that (FZ150 4:3 Aspect Ratio RW2s) report image-dimensions of width=4008 and height=3008, they also report sensor-dimensions of width=4144 and height=3016. Thus (in the horizontal dimensions, anyway), there exists plenty of extra photo-sites to be sampled. DxO reveals a significant quantity of them - and it does not surprise me that the savvy developers of RAW Therapee appear to (also) be including extra photo-sites to the maximum extent that is possible (considering the maximum sensor-height limits).

 

[peweuk]

David,

I don't have the petal lens hood fitted to the camera.

The RAW images are slightly bigger than the OOC jpg's, although in some instances the clipping seems to be within the 4000x3000 area. Picassa shows the RAW size as 4144x3016.

What is puzzling is that the clipping does not show up in PS Camea RAW which shows the RAW file as 4000x3000.

I have not worked with RAW before, so I don't understand why there are differences in the programs using the same images.

 

[Detail Man]

peweuk wrote:
I don't have the petal lens hood fitted to the camera.

Wow ...

Picassa shows the RAW size as 4144x3016.

Those are the "sensor-dimensions" (not the "image-dimensions") as reported in FZ150 EXIF data.

The RAW images are slightly bigger than the OOC jpg's, although in some instances the clipping seems to be within the 4000x3000 area.

What is puzzling is that the clipping does not show up in PS Camea RAW which shows the RAW file as 4000x3000.

That's an interesting (and seemingly contradictory) pair of statements (quoted above).

I have not worked with RAW before, so I don't understand why there are differences in the programs using the same images.

It does not surprise me that RAW Therapee (like DxO) is using more photo-sites than Lightroom or Silkypix do.

But ... (if there was no lens-hood attached), I think that your lens-system is showing some serious physical alignment problems surrounding how the (optical) image-circle is being projected onto the surface of the image-sensor.

Note how much sharper the details along the left edge of the image-frame are (as compared to the fuzzy and blurred foliage-detail and more in a significant portion of the areas located near the right-most edge of the image-frame). It is obvious and significant. It is non-trivial, and (in my eyes) unacceptable, indeed.

Note : The presence or the absence of "distortion correction" applied (manually, or using RT 4.x's automatic rectilinear-distortion corrections) has (essentially) nothing to do with what I am observing and here describing.

I would either return/exchange the camera (if I was within a return-time-window), or (sigh) send it down to the McAllen Texas chamber of horrors for what Panasonic refers to as "prompt warranty service". On a camera this expensive (as on the LX3), their policy is very likely attempting to repair your original unit (as opposed to the preferred scenario of simply replacing it with a brand-new unit that is not defective).

Note that another poster has reported (and posted) example-images of FZ150 lens-system de-centering that clearly affects (by the posters information, not just one, but two separate FZ150s).

It (de-centering) surely happened with some of the (incredibly, by Panasonic QC tested) LX5s ...

I don't think that anybody is "dreaming" (about "non-existent" problems) here ...

Good Luck,

DM ...

 

[Steen Bay]

David,

I don't have the petal lens hood fitted to the camera.

The RAW images are slightly bigger than the OOC jpg's, although in some instances the clipping seems to be within the 4000x3000 area. Picassa shows the RAW size as 4144x3016.

What is puzzling is that the clipping does not show up in PS Camea RAW which shows the RAW file as 4000x3000.

I have not worked with RAW before, so I don't understand why there are differences in the programs using the same images.

RT shows how the image looks before distortion correction.

 

[Steen Bay]

David,

I don't have the petal lens hood fitted to the camera.

The RAW images are slightly bigger than the OOC jpg's, although in some instances the clipping seems to be within the 4000x3000 area. Picassa shows the RAW size as 4144x3016.

What is puzzling is that the clipping does not show up in PS Camea RAW which shows the RAW file as 4000x3000.

I have not worked with RAW before, so I don't understand why there are differences in the programs using the same images.

RT shows how the image looks before distortion correction.

P.S. - In the thread linked below is a similar example, showing the huge difference between a Canon S100 ooc JPEG image, and a RAW image converted in Raw Therapee (or DCRaw) without distortion correction. The RT image looks almost like an image shot with a fisheye lens. Other converters, like ACR, will correct the distortion automatically, and quite a bit of the original pixels are lost in the process.

http://forums.dpreview.com/forums/read.asp?forum=1010&message=39634085

 

[peweuk]

Thanks for responses.

OK, below are 3 images

The first is the OOC jpg at full size.

The second is a screen capture of the same image displayed as RAW in Raw Therapee.

The third is the screen capture of the raw file as displayed in Photoshop Camera RAW.

Note that as well as the difference in colours, Raw Therapee is showing more of the RAW image than PSCR, and also there is distortion, most obvious along the left edge.

So is the RAW file actually as shown in RT and the camera and PSCR are adjusting it, or is RT displaying it incorrectly for some reason.

I need to be sure if there is actually a problem with the camera, or if it is just variations in the software before deciding on next steps. I'd hate to send the camera back for them to tell me there is nothing wrong with it. (Here in the UK I'm told Panasonic charge if there is no found problem - not sure if that's true).

The actual RAW file is here if anyone has the time to download it an check it out.
http://www.uko.com/files/P1010478.RW2

Thanks.

 

[Nicholas Bell]

Exact same problem when I was using a 32 bit version of RT which did not support the FZ150 RW2 at the time (mid Oct). When I updated to 4.0.4.2 64 bit later in October, the induced vignetting (or whatever is it) disappeared. ....I just downloaded 4.0.5.2 64 bit and I am not seeing the problem.

 

[peweuk]

Nicholas

I am using the 32bit V4.0.5.2 as I am using Win7 32 Bit.

What OS are you on - I guess 64bit of Win?? for the 64bit version of RT to work.

 

[Detail Man]

peweuk wrote:

So is the RAW file actually as shown in RT and the camera and PSCR are adjusting it, or is RT displaying it incorrectly for some reason.

I have the RW2 open in RT 4.0.4.2 (the version previous to 4.0.5.2, which I am holding back to wait for bug-reports about to trickle in to the RT Forum), and it looks just the same. No image-processing applications (can or do) "invent" photo-site data from the image-sensor data output that doe not in actuality exist.

There is even a small "hint of it it the OOC JPG, and a more substantial "hint" of it in the Capture One processed image. Both of these images [with or without rectilinear distortion corrections, which proceed with 10 or less extra pixel-locations above and below the (nominal, default) image-frame] show that they are merely excluding what RT 4.x is including. Viewing your RW2 using XnView 1.982 (which still does not have the color-rendering right, as it was released prior to the release of the FZ150) show exactly the same (RW2) image-size (and phenomena) that RT 4.0.5.2 does in your case (and RT 4.0.4.2 does in my case).

I need to be sure if there is actually a problem with the camera, or if it is just variations in the software before deciding on next steps. I'd hate to send the camera back for them to tell me there is nothing wrong with it. (Here in the UK I'm told Panasonic charge if there is no found problem - not sure if that's true).

Panasonic markets the FZ150 as having RW2 recording capabilities. However, I see that the (free) Silkypix SE 3.181 is "clipping-off" your RW2 to an extent that appears to be identical to that of the in-camera produced JPG ...

(While they should quite rightly be "hung-up" by their bootstraps and left out to dry if they were to try to pull this), Panasonic might be so harsh and myopic as to try (at least) to convince you that they only guarantee RW2 results when using the marginal "Siklypizz" that they have the poor judgment (or outright frugality) to "bundle" into the retail sales price of your camera. This would be ludicrous, but is (I suspect) possible ...

However, you can note to them that the problem actaully does (albeit to a minor degree) appear in the OOC JPG (as well as tthe SP SE 3.x) representations.

Far more importantly, however (and regardless of the barrel-distortion corrections being performed or not), is the fact that your FZ150 lens-system shows "glaring" evidence of lens-system de-centering problems in general that extend well into the right-side of the image frame. Take some more test-shots having plenty of fine foliage-detail in the left as well as the right extremes of the recorded image-frame (RW2 or JPG), and these problems will be amply demonstrated to you.

You could burn those images onto a CD/DVD disc to accompany your (required) written "Panasonic Lumix DMC-FZ150 Request for Service Completion Notice". In the US, Panasonic Warranty Service has a way of seemingly liking to ignore evidence that is not "thrust in front of their eyes". It seems unlikely that they would (or reasonably could ) ignore such hard-evidence provided to them ! ...

Nicholas Bell wrote:

Exact same problem when I was using a 32 bit version of RT which did not support the FZ150 RW2 at the time (mid Oct). When I updated to 4.0.4.2 64 bit later in October, the induced vignetting (or whatever is it) disappeared. ....I just downloaded 4.0.5.2 64 bit and I am not seeing the problem.

Please take note ! I surely am (seeing the effect) with my (32-bit) installation of RT 4.0.4.2 ... Nicholas.

peweuk wrote:
I am using the 32bit V4.0.5.2 as I am using Win7 32 Bit.
What OS are you on - I guess 64bit of Win?? for the 64bit version of RT to work.

You are looking in the wrong place. I strongly suspect that this has nothing whatsoever to do with whether it is a 32-bit of 64-bit OS version of RT 4.x !

You owe it to yourself to take some more test-shots (as I have recommended), and have a good look at what your (misaligned/de-centered) FZ150 lens-system is doing to the overall image-quality of your recorded (RW2 as well as JPG) images ! ... (IMO), it is not pretty, and it (should rightly be) unacceptable .

DM

 

[bugbait]

Hello.

I am running 64bit Win 7 and have the latest RawTherapee, 4.0.4.2 64 bit.

This solved that problem for me in general but there are still just a hint of such corner darkness/distortion in RW2 files loaded in RT.

So if this is a file I am going to keep and not just a practice image, I load it into PS, which in turn loads it in Camera Raw 6.5. (If your Camera Raw is older, go to Adobe and get the latest.) I save it from there in TIF, then load that TIF in RawTherapee. All seems fine then.

 

[Detail Man]

As Nicholas Bell reports a (64-bit/32-bit) difference RT 4.x difference, and bugbait (may, not clear) be reporting seeing a substantively similar(?) phenomena, I am re-thinking my previous dismissal of that (RT 4.x, 64-bit vs 32-bit) as a possible cause of (at least some of these) problems.

One thing is for sure. Your previously (originally) posted indoor shot should not show a the large brown-colored area of no image-data that it does (on the left edge-area) ! There surely is a limit (due to the optical image-circle) that RT 4.x (or any processor) can utilize ... maybe (for some reason), (it would appear, both of the 32-bit) versions of RT 4.0.4.2 and RT 4.0.5.2 do indeed have a serious malady.

I felt like I owed it to you ( peweuk ) after declaring evidence of lens-system de-centering to have (and to make available) a closer look at the outdoor (yard) shot that you have posted a JPG rendition of. I downloaded the largest size available from your DPR Gallery.

As it was (largely because of the open sky included in the image-frame) highly over-exposed in the Blue color-channel, I processed all of the samples displayed below in PSP9 - performing only an equal amount of linear-scaling of the "brightness" of all of the image-data (to a level which brought the full-size and first displayed below image to a lower "brightness", for the purpose of being able to better see what image-details exist).

I made 416x416 pixel-size crops of the upper-right, upper-left, and lower-left corner-areas of the image, displayed below. I used Gaussian (with little or no re-sampling artifacts introduced) 3x upwards re-sampling in the case of the three corner-area crops displayed (so that they are 1248x1248 in pixel-size). I will keep these images posted for a day or two (in my DPR Gallery, and on this thread):

















To my eyes, they all look rather marginal, with the quality decreasing in direct proportion to the distance of the (finely-detailed) subject-matter from the camera.

It seems very unlikely that (any) of this subject matter (at full wide-angle) would likely be out of the Depth of Field of focus. If that were the case, it would be expected to be the closest subject matter that would suffer the most - yet it is the closest subject-matter that appears to be the sharpest of all ...

To keep things in perspective, it may(?) be that you performed little (or no) adjustments to the FZ150 RW2 image in RT 4.x processing. Additionally, if a software-version-related flaw does exist (which I concede may indeed somehow be the case), little or nothing should rightly be concluded from these results.

So, I do hope that your FZ150 lens-system does not have de-centering problems, and that you will not have a need to try to exchange it or have it serviced. Nevertheless, it would be a wise practice (in general, and with or without these problems being found to be influenced by the RT 4.x image-processing) to carefully assess the real performance of your lens-system now (and not later on down the line). I do sincerely hope that everything is just "peachy" !

DM ...

 

[Gerry Pasternack]

I'm using RT 4.5.0.2 under W7-64. I do not see this problem.
--
Gerry

 

[Detail Man]

As this problem surrounding 32-bit versions of RAW Therapee 4.x may well (for reasons unknown) be specific to FZ150 RW2s, I would note that using the 32-bit versions of RT 4.0.3.4 and 4.0.4.2 (on FZ28 RW2s), I have not seen any program behavior even resembling what peweuk has (it may, it seems) been struggling with (in the case of RW2s from a FZ150):

FZ28 RW2, RT 4.0.3.4 (32-bit) :





I often hate it when nimrods plaster a thread with their idea of "demonstrated perfection" of some particular (this or that, perceived to have been somehow maligned) camera-model ... but I thought that it might be somewhat informative (and possibly supportive of a FZ150 RW2 + RT 4.x in 32-bit vs 64-bit hypothesis) to demonstrate that RT 4.x (32-bit) is not (in itself) implicitly functionally flawed ...

RT 4.x (in general) is an increasing impressive (albeit RAM resource-hungry) freeware application !

While Dave Coffin (the developer of the DCraw code that is the core of the RAW Therapee software-routines) lists the latest release of DCraw to be compatible with the FZ150 RW2s , there likely is a lot of "development mileage" in between DCraw binary-code and the RT developers' implementations with it. Reporting the problems on the RAW Therapee Forum might well go a long way toward calling any problems experienced with FZ150 RW2s to the RT developers' attention at : http://www.rawtherapee.com/forum/

 

[Detail Man]

Well, when I open this Imaging Resource FZ150 RW2 image-file (using RT 4.0.4.2 32-bit) at:

http://www.imaging-resource.com/PRODS/FZ150/FZ150hHOUSE.RW2.HTM

... there are no problems at all. There are lots of other FZ150 RW2s on the same IR web-page at:

http://www.imaging-resource.com/PRODS/FZ150/FZ150A7.HTM

Yet, when I open peweuk's FZ150 RW2, I get the same old thing that we all have seen ... ... ...

Dunno. Time for peweuk to get busy with RT 4.0.5.2 32-bit, and have a look at the Imaging Resource FZ150 RW2 (or the Photography Blog Image Samples page). It will be interesting to hear about what peweuk finds when these other FZ150 RW2s are opened in RT 4.0.5.2.

But ... I see that there is only one FZ150 RW2 that was recorded at full wide-angle (4.5mm actual focal-length) here:

http://www.imaging-resource.com/PRODS/FZ150/FZ150hVFAWB.RW2.HTM

and it has the infamous "vignetting" effect (more so on the right than on the left edges, once again). Oddly the shape of the pattern (of the "vignetted" corners does not look like an "image-circle" ... it looks more like an "image-rectangle" ... ? ... ... This is weird - and I would not count out the FZ150 design itself ...

For some reason my browser has not been able to access the Photography Blog site for several days now. Same thing happened with the "lumixuser.ruk" site.

Try seeing if any of the PB FZ150 RW2s are (also) recorded at full wide-angle, check it out (with RT 5.052 or 4.042), and let us know what you find ! ...

Note: Have to finishing my editing of the post. Will try the infamous full wide-angle FZ150 RW2 from IR in LR 3.5, and post the results ... (drum-roll) ...

 

[Steen Bay]

Well, when I open this Imaging Resource FZ150 RW2 image-file (using RT 4.0.4.2 32-bit) at:

http://www.imaging-resource.com/PRODS/FZ150/FZ150hHOUSE.RW2.HTM

... there are no problems at all. There are lots of other FZ150 RW2s on the same IR web-page at:

http://www.imaging-resource.com/PRODS/FZ150/FZ150A7.HTM

Yet, when I open peweuk's FZ150 RW2, I get the same old thing that we all have seen ... ... ...

But what happens if you open one of the IR RAW images shot at 4.5mm (25mm equivalent)? And couldn't it be considered to be more like a "feature" than a "problem", to be able to see how the uncorrected RAW image really looks?

Dunno. Time for peweuk to get busy with RT 4.0.5.2 32-bit, and have a look at the Imaging Resource FZ150 RW2 (or the Photography Blog Image Samples page). It will be interesting to hear about what peweuk finds when these other FZ150 RW2s are opened in RT 4.0.5.2.

 

[Detail Man]

Lightroom 3.5 does not show the problem. When I apply a Transform - Distortion - Amount (in RT 4.042) of -0.16 or a more negative value, the problem almost (but not quite) fully recedes.

I think that the RT 4.x developers are trying to include too many extra photo-sites in the horizontal dimension that are available in the FZ150 image-files. ExifTool finds a report in FZ150 RW2s (when in 4:3 Aspect Ratio, at least) of a Sensor Width of 4144 photo-sites, and Sensor Height of 3016 photo-sites. They are close, but need to shave some off.

Why it would be that the 64-bit version of RT 4.x appears to behave differently is a real mystery ... but I take the statements of the several reports posted at face value. Seems pretty weird, indeed

Steen Bay 's comments on this thread have turned out to have some validity. Prior to performing the geometric distortion correction in RT 4.042 and 4.052 (32-bit), the image looks rather "vignetted" (yet, rather strangely, much more so on the right-edge than on the left-edge in this case of the FZ150 RW2s ... ). And, as I mentioned, the shape of the "vignetting" effect appears more like an "image-rectangle" than it does an "image-circle".

peweuk , is it true that a brown colored wall appears near the far-left edge in your indoor shot posted ? It's rather hard for me to tell by looking. I had originally interpreted that to be an absence of image-information (due to it's solid "uniform-field" appearance) ...

Steen, (see above, and) it appears that you did not read my post (in full) here (posted 37 minutes prior to your recent post):

http://forums.dpreview.com/forums/read.asp?forum=1033&message=39880012

DM

 

[Brad Morris]

A raw file actually has more detail at the edges than is processed in the main stream raw converters such as ACR or silkypix. Raw therapy is not cropping off teh very edges captured in the raw file.

If you compare a raw therepee converted file with the jpg or ACR converted file. you will actually find it larger in both height and width.

Take a look at the shot you posted with the rafters and you will see there is more of the beam showing in the Raw therapee shot. If you crop the shot back to match the jpg size image you will find that you cannot see the vignette.

I haven't used RAW and was just beginning to experiment with it.
I have taken a number of shots over the last week or two, using RAW+JPG.

Most of the RAW shots are OK, but there are a few which display the 'clipping' you can see in the corners of the image below which is a screen capture of an FZ150 RAW file displayed in Raw Therapee (latest version 4.0.5.2) on my PC.

The camera processed JPG version of the image is OK and shows no problems, and the RAW file opens with no clipping in Photoshop Camera RAW.

I have no idea what is causing this and all the examples where this appears seem to be when using 25mm Focal Length.

Can anyone with RAW experience explain what might be going on?

--
http://www.flickr.com/photos/k-blad/

 

[Steen Bay]

Lightroom 3.5 does not show the problem. When I apply a Transform - Distortion - Amount (in RT 4.042) of -0.16 or a more negative value, the problem almost (but not quite) fully recedes.

I think that the RT 4.x developers are trying to include too many extra photo-sites in the horizontal dimension that are available in the FZ150 image-files. ExifTool finds a report in FZ150 RW2s (when in 4:3 Aspect Ratio, at least) of a Sensor Width of 4144 photo-sites, and Sensor Height of 3016 photo-sites. They are close, but need to shave some off.

Why it would be that the 64-bit version of RT 4.x appears to behave differently is a real mystery ... but I take the statements of the several reports posted at face value. Seems pretty weird, indeed

Steen Bay 's comments on this thread have turned out to have some validity. Prior to performing the geometric distortion correction in RT 4.042 and 4.052 (32-bit), the image looks rather "vignetted" (yet, rather strangely, much more so on the right-edge than on the left-edge in this case of the FZ150 RW2s ... ). And, as I mentioned, the shape of the "vignetting" effect appears more like an "image-rectangle" than it does an "image-circle".

peweuk , is it true that a brown colored wall appears near the far-left edge in your indoor shot posted ? It's rather hard for me to tell by looking. I had originally interpreted that to be an absence of image-information (due to it's solid "uniform-field" appearance) ...

Steen, (see above, and) it appears that you did not read my post (in full) here (posted 37 minutes prior to your recent post):

http://forums.dpreview.com/forums/read.asp?forum=1033&message=39880012

DM

No, I didn't notice your edit. Found this old thread, showing how an uncorrected LX3 WA image looks :

http://forums.dpreview.com/forums/read.asp?forum=1033&message=33487113