How do I processing RAW images in the same way my camera processes JPEGs?

[stephband]

I have a Panasonic Lumix S5. It captures a RAW file and saves both that and a JPEG version of the same image. I have filters switched off. When I import the two files into an image editor they look completely different. Sometimes, I want to recreate the way my camera has processed the JPEG while editing the RAW image but I'm struggling to understand what it is doing and why.

An example. Here's an image I shot last night. It's not terribly good, but it illustrates the problem...




the JPEG produced by the camera




A screenshot of the RAW image



We can see that that lava river in the JPEG produced by the camera is yellow, as it was on the night, whereas the RAW image has regions that are bleached white. Presumably it still carries the data to unbleach those white patches.

At first I thought this was simply a question of white balance, which I assume is not applied to the RAW image. But adjusting the white balance in an image editor does not solve the problem.

So what is the camera doing and how can I recreate what it is doing?

 

[Digital Nigel]

Yes, I do too. I have longer exposures of the same image I was going to stack to create something like this.

You shouldn't need to stack them. I produced this version with a lightened foreground from your single raw file. I kept the high contrast around the lava flow for greater drama:


Lighter foreground to show the people watching, but still dark around the lava flow. [16:9 aspect ratio]

 

[s_grins]

n/t

 

[lcubed11]

Thank you for doing this. I like the SilkyPix import, it has the closest colouring to how I remember the scene on Saturday. (I like how you edited the last one too - I have multiple exposures I was going to stack to create something like this).

Overall, the conclusion to this thread seems to be, "buy more tools", which I'm slightly surprised about, because until today I thought they were all doing much the same thing, and assumed MacOS processes to be among the best.

SilkyPix explicitly supports my camera, but seems to have multiple websites which is a bit confusing.

there's should be a link to the free panasonic specific version of SILKYPIX in the materials included with your S5.

 

[Digital Nigel]

Thank you for doing this. I like the SilkyPix import, it has the closest colouring to how I remember the scene on Saturday. (I like how you edited the last one too - I have multiple exposures I was going to stack to create something like this).

There's no need for stacking to create images like that. A single raw frame is adequate.

Overall, the conclusion to this thread seems to be, "buy more tools", which I'm slightly surprised about, because until today I thought they were all doing much the same thing, and assumed MacOS processes to be among the best.

No, definitely not. You can certainly get better raw processors. For one thing, you're not even getting the full width of the raw images in your JPEG. Using a better raw processor is like a very low cost improvement to your lens and body hardware. Your UWA lens gets wider and sharper, and you get less noise in low light images.

SilkyPix explicitly supports my camera, but seems to have multiple websites which is a bit confusing. ON1 on the face of it looks like it might be a richer tool for composing HDR stacks, and it appears to support Photoshop plugins (Panasonic has a plug-in for importing HLG images, another thing I wanted to play with). But it's hard to tell. Time to download and try them out!

Thanks for your help!

 

[ggbutcher]

Thank you for doing this. I like the SilkyPix import, it has the closest colouring to how I remember the scene on Saturday. (I like how you edited the last one too - I have multiple exposures I was going to stack to create something like this).

Overall, the conclusion to this thread seems to be, "buy more tools", which I'm slightly surprised about, because until today I thought they were all doing much the same thing, and assumed MacOS processes to be among the best.

SilkyPix explicitly supports my camera, but seems to have multiple websites which is a bit confusing. ON1 on the face of it looks like it might be a richer tool for composing HDR stacks, and it appears to support Photoshop plugins (Panasonic has a plug-in for importing HLG images, another thing I wanted to play with). But it's hard to tell. Time to download and try them out!

Thanks for your help!

Neat image!

I just had to mess with it... all raw channels are blown at some place. Here's a screenshot of the brightest part, without correction:



Note the brightest part of the lava renders magenta, that's because the white balance took all three pileups of blown channel values and skewed them according to the camera-supplied WB multiplers 2.211, 1.000, 1.629. Turn on the highlight reconstruction tool and it paints those pixels with appropriate color:



So yes, processing a raw file to approximate the JPEG render takes a bit of work. Most raw processing software (camera firmware included) will default to something acceptable, which may be acceptable to you, until it's not. Then, the world of the raw file opens to you, with myriad choices, especially with an image such as this. You're seeing that in some of the renders posted here; me, I didn't know there were people in the frame until that one render...

About stacking, if you don't want to invest a lot of money to start, there are a couple of free alternatives. LuminanceHDR comes to mind, not exactly the easiest UI but the the merging algorithms are pretty good. What little I've done has been with HDRMerge, a command line program that (for me, anyway) is a bit more straightforward to use and will provide a nice linear amalgam render to which you will then need to apply a tone curve with other software. "Free" comes at a price...

 

[TorsteinH]

I dont know if it's of any interest, but here are the dxo PL 5 version with deep prime, default S5 rendering, no other adjustment.







--
Torstein

 

[stephband]

Yup. Version 8 SE. The interface and responsiveness is reeeally clunky. Current version of full app is version 11, I'll try that soon.

 

[stephband]

It IS of interest. I might as well add another processing app to the list since I seem to be trying them all!

That's by default?? I'm surprised by how much of the darker colours it lightens up in that image. It brings out the people in a way the other apps are not doing by default. Also, it has given us the full frame, not the 16:9 crop stored by the camera (that's a good thing, I can't seem to get the full frame back in other apps if I want it), and without lens correction (my version of SilkyPix is applying lens correction I cannot take off).

Thanks!

 

[TorsteinH]

It IS of interest. I might as well add another processing app to the list since I seem to be trying them all!

That's by default?? I'm surprised by how much of the darker colours it lightens up in that image. It brings out the people in a way the other apps are not doing by default. Also, it has given us the full frame, not the 16:9 crop stored by the camera (that's a good thing, I can't seem to get the full frame back in other apps if I want it), and without lens correction (my version of SilkyPix is applying lens correction I cannot take off).

Thanks!

Good that it was useful! In my view, DXO Elite PL5 is among the best RAW developers, if not the best!

 

[Digital Nigel]

It IS of interest. I might as well add another processing app to the list since I seem to be trying them all!

That's by default??

The defaults in PhotoLab are whatever you choose. You can set up your own preset with any default settings you prefer.

I'm surprised by how much of the darker colours it lightens up in that image. It brings out the people in a way the other apps are not doing by default. Also, it has given us the full frame, not the 16:9 crop stored by the camera (that's a good thing, I can't seem to get the full frame back in other apps if I want it), and without lens correction (my version of SilkyPix is applying lens correction I cannot take off).

Why do you think there's no lens correction in that image? I've shown a number of PhotoLab 5 renderings in this thread, all with lens correction applied.

 

[Henry Richardson]

I have a Panasonic Lumix S5. It captures a RAW file and saves both that and a JPEG version of the same image.

I think Panasonic gives you a free raw processing program and that should be able to process the raw to look like the camera JPEG.

 

[TorsteinH]

It IS of interest. I might as well add another processing app to the list since I seem to be trying them all!

That's by default??

The defaults in PhotoLab are whatever you choose. You can set up your own preset with any default settings you prefer.

Thats true, but I have selected to keep DXO's settings as default, the only change I made to the preset are for cropping, so it's actually the deafult development in DXO PL.

I'm surprised by how much of the darker colours it lightens up in that image. It brings out the people in a way the other apps are not doing by default. Also, it has given us the full frame, not the 16:9 crop stored by the camera (that's a good thing, I can't seem to get the full frame back in other apps if I want it), and without lens correction (my version of SilkyPix is applying lens correction I cannot take off).

Why do you think there's no lens correction in that image? I've shown a number of PhotoLab 5 renderings in this thread, all with lens correction applied.

 

[Digital Nigel]

It IS of interest. I might as well add another processing app to the list since I seem to be trying them all!

That's by default??

The defaults in PhotoLab are whatever you choose. You can set up your own preset with any default settings you prefer.

Thats true, but I have selected to keep DXO's settings as default, the only change I made to the preset are for cropping, so it's actually the deafult development in DXO PL.

OK, that's interesting. Here's a comparison of your image using DxO's original default settings, and a rendering with my default settings:




DxO default settings




My default PL5 settings (which I would normally tweak with every image)

And a tweaked version, giving more prominence to the spectators, and apply ViewPoint's volume deformation correction, to stop the people being stretched at the edges:




Tweaked version, making use of DxO and the FilmPack and ViewPoint controls.

 

[Digital Nigel]

It IS of interest. I might as well add another processing app to the list since I seem to be trying them all!

That's by default?? I'm surprised by how much of the darker colours it lightens up in that image. It brings out the people in a way the other apps are not doing by default. Also, it has given us the full frame, not the 16:9 crop stored by the camera (that's a good thing, I can't seem to get the full frame back in other apps if I want it), and without lens correction (my version of SilkyPix is applying lens correction I cannot take off).

I did a comparison of the ON1 and DxO renderings. The DxO PL5 rendering recovers more of the original raw frame than the ON1 (and other raw and OOC) renderings. In other words, it makes your UWA lens even wider — that's a very cheap hardware upgrade!

I overlaid the ON1 version with 50% opacity on the larger DxO image:


DxO uses more of the original raw image than the OOC JPEGs and other raw converters, so it makes the lens wider. The image is also less noisy that the others, effectively improving the sensor quality.

As you can see, the images don't quite align. That's because DxO uses its own, more accurate lens distortion corrections, whereas the other raw converters just use Panasonic's approximate corrections.

So, as Torstein correctly said, PhotoLab is the best raw developer available.

 

[TorsteinH]

Of course, if you lift the shadows you get a different picture. But as OP stated, he wanted something as close as possible to the original proseccing in camera as possible, so that was what I gave him. ( As far as default in DXO gives that...)

 

[Digital Nigel]

Of course, if you lift the shadows you get a different picture. But as OP stated, he wanted something as close as possible to the original proseccing in camera as possible, so that was what I gave him. ( As far as default in DXO gives that...)

Sure, but I was simply comparing my PL5 defaults with your image that used DxO's original defaults. I didn't explicitly lift the shadows, it's just what my preset does by default. Neither image replicates what the camera produced, and nor should you expect it to. I certainly aim to produce better images than OOC JPEGs, and almost always tweak the settings after my defaults have been applied, as I did here. Depending on whether I wanted the people to be seen, I would tweak it differently.

This is an illustration of how raw development isn't just an automatic conversion process, but one where you modify the appearance of the exported images, depending on what you want them to look like.

 

[stephband]

SilkyPix. The version they offer is out of date (8, current version 11) and pretty clunky.

 

[stephband]

Yes, it is giving us more picture than ANY other editor that has been tried in this thread.

I went to the DXO website to look for supported cameras/lenses, and when I searched for Lumix S5 with Sigma 14-24mm 2.8f DN it said 'no support'. So are you sure that is applying any lens correction at all? Because not applying lens correction would be a sure way to get the whole image back. I have been trying to find ways to switch it off in other apps!

 

[Digital Nigel]

Yes, it is giving us more picture than ANY other editor that has been tried in this thread.

I went to the DXO website to look for supported cameras/lenses, and when I searched for Lumix S5 with Sigma 14-24mm 2.8f DN it said 'no support'. So are you sure that is applying any lens correction at all? Because not applying lens correction would be a sure way to get the whole image back. I have been trying to find ways to switch it off in other apps!

This is the module it downloaded and installed

Is that not your lens?

This is the image produced with distortion correction turned off:




Distortion correction turned off




Distortion correction turned on

And this is the two images super-imposed:




Distortion corrected and uncorrected images overlaid with 50% opacity. Clearly, the distortion correction is having an effect.

Perhaps you have another detailed image with straight lines (eg, buildings or just a carefully shot, rectilinear picture of tiles) that we can check? Upload the OOC JPEG (and maybe one from another raw developer) and share the raw file.

 

[Henry Richardson]

SilkyPix. The version they offer is out of date (8, current version 11) and pretty clunky.

Well if you want what you said you want then you will have to use the camera brand's software that has the profiles that are the same as in the camera when it makes a JPEG.