My New Approach To RAW Processing

Mads Bjerke wrote:

Unfortunately the screen shots I have posted are leading you astray as the quality gets compromised if you try to scale them.

Which of these do you prefer? One is your TIFF, and one is my messing around with your RAW file in Lightroom.

You can probably tell which one is yours from the colours; I couldn't quite match what you had in the TIFF.

For me, some parts of the TIFF you created are better than what I could do in Lightroom, especially vegetation. However, other parts are worse to my eye. For example, the shadow area of that large boulder near the bottom is muddy and indistinct in your TIFF, but better (to my eye) in Lightroom.

Your TIFF is on the left. The shadows near the water look smeared and muddy to me.

I find these exercises exceptionally useful and I really appreciate that you're doing this. Your workflow produces excellent results -- undoubtedly. However, I wouldn't change my workflow (yet) based on what I've seen so far.

Much of what you call finer detail looks like more local contrast to me. If you like it, fine.

My experience is that DxO Photo Lab defaults to a considerable amount of local contrast.

You can do much the same thing by running a .tif through Raw Therapee (which does not require a raw file for many of its transformations) and playing with its Contrast by Detail Levels. Other programs, too, have several ways to apply local contrast.

Some people will find these alternatives simpler than the circuit between programs that you use.

We all know that sharpening can be overdone, and also that some people confuse it with revelation of more detail. Probably most of us have a brief phase like this early in our hobby. The phenomenon here is not oversharpening; it is the local contrast counterpart of it.

Mads Bjerke wrote:

I too enjoy these exercises as they push us to re-evaluate our processes and perhaps find improvements.

This scene is challenging as there is very high contrast from the midday sun.

Judging by these two images I do have a preference for the image on the left (confirmation bias?), but given that we are looking at a 100-200% crop both are acceptable to me.

It would be interesting if you could share your sharpening settings in LR for others to try.

Where you able to match to your satisfaction the perceived detail and sharpness from the vegetation in general?

As for the shadows, I have a preference for smooth transitions into the darkest part of an image.
If you are used to more traditional contrast curves then my version will seem a bit flat in comparison.
I use a linear curve now to process my raw as I feel the transition from mids to lows are more natural and lifelike. Totally subjective, for sure.

I think that's well put Mads. The idea of "smooth transition" actually works for this conversation too.

I think that if 100 people worked on your file, and we compared them all, we'd probably notice a threshold:

• On one side are the ones that most experienced people would find objectionable for a reason we could agree on, describe using common language (e.g., ugly sharpening artifacts like halos or 'crunchiness').
• On the other side, there would be a "smooth transition" into versions that most experienced people would consider were good, but which had qualities that some people preferred. Sometimes these qualities could be articulated clearly; your preference for the treatment of shadows is a great example. Other times it would be less tangible, but the person preferring that version could reliably choose the images that had the quality in question.

This situation seems to be quite common by the way. I have read, and been part of, exactly the same kind of conversation, but about black and white image photography.

As to what I did in Lightroom to try to match what you'd done in your TIFF, it's nothing fancy:

• With sharp modern lenses like the one you used, I like less sharpening Amount than Lightroom uses by default; in this case I lowered Amount to 33
• I don't normally use Radius 1.9 and Detail 83, but I was trying to match what I was seeing in your file so that's what I used here.
• I set Masking at 31
• After using Detail adjustments to try to match what you'd done, I reduced global Contrast to -39
• Sometimes I use a bit of Texture to create more local contrast and the appearance of sharpness. This is often necessary with the older, lower contrast lenses I use. I didn't do it in this case because there was plenty of local contrast already.

Hi,

I have played using DxO Photo Lab a bit. The image shown here was shot on the A7rIV with a Zeiss Loxia 25/2.4 lens at f/8.

The left side here has been processed in DxO Photo Lab and the right side with Lightroom. The DxO image is pretty much default while the right side was processed to my taste in Lightroom.  I could probably adjust the images for a better match.

I think that the two images have different contrast and tone curves. Regarding sharpness, there is a huge difference in the close foreground. I guess that DxO applies stronger sharpening in off axis areas, or it even take some asymmetri of the PSF into account.

I would also think that DxO's sharpening will have less 'halo' effect compared with LightRoom (*).

Best regards

Erik

(*) I have done a bit of cheating here, doing MTF analysis based on DPreview's studio image.

I've also been playing around with it and after importing the DNG back into C1 I've got it to almost match another I developed earlier this week, but not quite there yet. It's been an interesting exercise and the thing I notice most is how clean the conversion is noise wise. the DNG is super clean. Not seeing any noticeable difference in detail in my images though but I'll keep looking.

Trying something with another person's file is one thing, but I find the best way to know if a tool works for you is to try it on your files.

DXO offers a 30 day trial, so I downloaded a copy and tried it on some files.

Short version:

For the files I tried, with the lenses I used on my GFX 50R, both programs did a great job. However, (1) I can't get the full benefit of DXO's sharpening tools because my lenses are not profiled; and, (2) there's something happening on the noise front when the DNG produced by DXO comes into Lightroom.

Long version:

The first thing I discovered right out the gate is that the Lens Sharpness tool is only available if your lens is in DXO's database. Not a single lens I use is in DXO's database. Your only option in that case is to rely on DXO's sharpening tools.

Mads: I was trying to duplicate your settings exactly, so I couldn't do that. Keep that in mind when you look at these samples.

I ended up exporting the file I was trying it on two ways: with Unsharp Mask off (and then sharpening in Lightroom), and with Unsharp Mask on (using the default settings, and then no sharpening in Lightroom). I used all the other settings you specified except vignette.

Here I'm only showing the "sharpened in LR" version because the ones that came out of DXO with the default sharpening were not good. I'm sure someone who knows how to use DXO better than I do could get better results.

The test files I used for this exercise were made with a new lens: an SMC Pentax 67 200mm f/4 with the 1.4x teleconverter, so effective focal length of 280mm. To make this a more realistic test for me, I worked on an f/11 file (an aperture I use a lot), and an f/22 file (an aperture I normally never use because it's crap, but I was curious).

Here's a long view down a street at f/11 on a hazy day. Lightroom is on the left, and DXO on the right. There are some differences, but to my eye, both programs do a great job. Keep in mind that whatever secret sauce DXO might have applied had I used a profiled lens is not being applied here.

Pentax 67 200mm f/4 with the 1.4x teleconverter -- so 280mm at f/11.  200% magnification

Every new lens that comes into my hands ends up in my kitchen sooner or later. There's lots of light, and different kinds of surfaces and shapes. This is a knob on my toaster oven at f/22.

I don't know about you, but this should look like crap -- a soft mess due to diffraction. I'm astonished by how good this lens is. Remember, this is a decades old film-era telephoto lens with a teleconverter at f/22. Amazing.

Both tools do a great job as far as I'm concerned. They both pulled out the circular striations in the knob and the brushed texture of the steel. I was not expecting to see that.

Pentax 67 200mm f/4 with the 1.4x teleconverter -- so 280mm at f/22 if you can believe it. 400% magnification

Here's the more interesting result. Both images are in Lightroom, and noise reduction is off. The only difference is that the one on the right came out of DXO with Deep Prime noise reduction.

Importantly, inside DXO it doesn't look that bad. Is something happening to the file once it comes into Lightroom that makes it look like this?

Comparing noise treatment in an area of smooth texture. Magnification is 400% in this screenshot

Did I see anything in DXO that would make me want to use it to develop my RAFs? No, I did not. However, it does a great job, and maybe with profiled lenses it would pull ahead of Lightroom. But for someone like me, Lightroom is still the tool of choice.

Mads, thanks again for starting this thread. Benchmarking our tools and techniques every once in a while is incredibly valuable. You helped me do that.

That's an interesting result compared to what I saw doing the same exercise with C1. I saw a clear improvement in noise reduction in the DNG produced by DXO compared to the native C1 converted RAW. I still like the overall result in C1 better and I can run it through denoising tools after the fact but I'm not done trying to match it and probably can do better with practice. This is with the 100S.

Native C1 converted RAW viewing sky at 200%. This is after doing post processing to taste and isn't really noticeable unless at this extreme type of magnification.

View: original size

RAW developed by DXO with just denoise and lens corrections applied. I then imported the DNG back into to C1 and tried to match the result I got with C1. In the end I still like the C1 version better and I got there quickly but I'm sure I can learn to do this better. The DNG is much cleaner.

View: original size

Mads Bjerke wrote:

Erik Kaffehr wrote:

Hi,

I have played using DxO Photo Lab a bit. The image shown here was shot on the A7rIV with a Zeiss Loxia 25/2.4 lens at f/8.

The left side here has been processed in DxO Photo Lab and the right side with Lightroom. The DxO image is pretty much default while the right side was processed to my taste in Lightroom. I could probably adjust the images for a better match.

I think that the two images have different contrast and tone curves. Regarding sharpness, there is a huge difference in the close foreground. I guess that DxO applies stronger sharpening in off axis areas, or it even take some asymmetri of the PSF into account.

I would also think that DxO's sharpening will have less 'halo' effect compared with LightRoom (*).

Best regards

Erik

(*) I have done a bit of cheating here, doing MTF analysis based on DPreview's studio image.

That is a very noticeable improvement in sharpness and perceived detail.
I don't know what algorithm DxO is applying, but it could have something to do with lens profiles in conjunction with demosaicing.

My understanding is that DxO measures the Point Spread Function (PSF) across the image in the lens profiling process. The PSF describes how blur affects a point in the image. If the PSF is known, the point can be restored from blurred image, using a process called deconvolution.

The PSF may be estimated by some well known function, like a gaussian, or the PSF may be calculated from the image itself.

The sample I posted is a bit interesting, as I think that the foreground is significantly blurred due to defocus, due to limited depth of field. It seems that DxO achives some impressive sharpening here, too.

Now, sharpening tends to:

• Increase noise 
• Create artifacts like haloes or ringing

I looked at slanted edge based MTF on sharpened images using Lightroom and DxO:

This example is based on DPReview Studio image, with the Canon EOS 5R using the RF 50/1.2L lens. It seems that both DxO and LR yield the same sharpness at the pixel level (see the red vertical line). It could be argued that LR, with the parameters used' yields a slightly sharper image at lower detail, but that may not be noticeable.
But, LR shows significant halo effect in the highlights. DxO mark shows a 'negative' halo in the darks, ut that would not be noticeable,

Best regards
Erik

Mads Bjerke wrote:

Here's the more interesting result. Both images are in Lightroom, and noise reduction is off. The only difference is that the one on the right came out of DXO with Deep Prime noise reduction.

Importantly, inside DXO it doesn't look that bad. Is something happening to the file once it comes into Lightroom that makes it look like this?

Comparing noise treatment in an area of smooth texture. Magnification is 400% in this screenshot

Did I see anything in DXO that would make me want to use it to develop my RAFs? No, I did not. However, it does a great job, and maybe with profiled lenses it would pull ahead of Lightroom. But for someone like me, Lightroom is still the tool of choice.

Mads, thanks again for starting this thread. Benchmarking our tools and techniques every once in a while is incredibly valuable. You helped me do that.

I do not see this in my files, even at 1000% zoom.
It looks like a sharpening artefact to me.
Did you apply sharpening to the file after the NR was applied?

In my DxO converted files I do not add any sharpening other than the Global Lens Sharpen I outlined in my first post.

That's what it was Mads.

In that example, I couldn't apply sharpening in DXO because the lens I used is not profiled, so the image had to be sharpened in Lightroom. Lightroom appears to be balancing its noise reduction and sharpening better than it's balancing its sharpening and DXO's noise reduction.

After a coffee this morning it finally dawned on me that while I don't have any profiled lenses now, I did before. That means I can do a proper comparison on one of my GFX 50R files.

This time I chose an image at 320 ISO made with the GF 45-100 and did the comparison using the same settings you use in your preset. This is the whole scene. There are in-focus areas with lots of detail, and also out of focus areas in shadows.

Fujinon GF 45-100 f/4 @ f/11, ISO 320. Focus is on the bark of the cedar trees at right

My experience again is that both programs are doing a good job on parts of the image that look like this (in focus, lots of detail and texture). I'm seeing a tiny bit more detail in the Lightroom version, but also a tiny bit more coarseness. Both could be adjusted more to address these points I'm sure.

DXO version on the left. They both look good to me in this 100% view.

Where DXO starts to look different is in an out of focus area in shadows. At high magnification, the DXO version is unquestionably smoother. The smoothness could read as less detail at this enlargement.

An out of focus area in shadow at 300%.

The smoothness comes at the cost of a bit of detail, or at least the appearance of a bit less detail, as shown in this 100% view.

At 100%, the DXO version is a bit smoother, but also looks a bit less detailed

Once again, DXO seems to be an excellent tool. I don't see a reason to include it in my workflow (mostly because I don't shoot high ISO and because I am not using profiled lenses). But it's good to know tools are out there for special situations.

rbf wrote:

That's an interesting result compared to what I saw doing the same exercise with C1. I saw a clear improvement in noise reduction in the DNG produced by DXO compared to the native C1 converted RAW. I still like the overall result in C1 better and I can run it through denoising tools after the fact but I'm not done trying to match it and probably can do better with practice. This is with the 100S.

As you'll see from my follow-up post to last night's work, I came to exactly the same conclusion for Lightroom after trying again with a profiled lens (GF 45-100).

Mads isn't asking usto change our workflows to use DXO!

Nonetheless, I find it very useful and constructive to work through exercises like this just to check if better results are possible using other tools, and if so, whether it's worth the huge effort that would be required to incorporate new tools. Sometimes it is.

Mads Bjerke wrote:

My understanding is that DxO measures the Point Spread Function (PSF) across the image in the lens profiling process. The PSF describes how blur affects a point in the image. If the PSF is known, the point can be restored from blurred image, using a process called deconvolution.

The PSF may be estimated by some well known function, like a gaussian, or the PSF may be calculated from the image itself.

The sample I posted is a bit interesting, as I think that the foreground is significantly blurred due to defocus, due to limited depth of field. It seems that DxO achives some impressive sharpening here, too.

Now, sharpening tends to:

• Increase noise
• Create artifacts like haloes or ringing

I looked at slanted edge based MTF on sharpened images using Lightroom and DxO:

This example is based on DPReview Studio image, with the Canon EOS 5R using the RF 50/1.2L lens. It seems that both DxO and LR yield the same sharpness at the pixel level (see the red vertical line). It could be argued that LR, with the parameters used' yields a slightly sharper image at lower detail, but that may not be noticeable.But, LR shows significant halo effect in the highlights. DxO mark shows a 'negative' halo in the darks, ut that would not be noticeable,

Best regards
Erik

That is very interesting and confirms my hunch, if correct.
I cannot replicate the results in LR for many images.
The tools just aren't there.

It does seem to affect some images more than others, your example shows a very pronounced improvement. Others seem to find less of an improvement on some of their images.

It could also be more effective on certain types of raw files and sensor types.
I am finding good improvement on my Canon CR3 files, but less obvious on my GFX100 files.
It could also be resolution dependent with higher megapixel sensors already having more detail natively. I don't know.

Hi,

It is hard to know...
It seems that the GFX lenses are very well corrected, according to what Jim Kasson says.

Having smaller pixels means less blur, but when viewed at the pixel level that should not be very noticeable.
The Canon EOS 5R has a quite advanced OLP filter, with 4 layers instead of 2. Not obvious what effect that may have.

Comparing zooms and primes may be a complex issue.

I have for very long felt that Adobe needs to improve their demosaic algorithms regarding images with aliasing, but in the end aliasing is an issue that camera makers should handle. I would give some honors to Canon for keeping OLP filters on their cameras.

Looking at edge profiles and MTFs give a lot of insight into the working of things, that may be valuable when printing large. I very seldom print larger than A2, like 16"x23", which is pretty much my print size.
In the end, I don't care a lot about things like resolution, DR or sharpness. I have been using digital cameras since 2004 and I have never found that 24 MP was not enough. Some of my larger prints are from just 10 MP.

Having the right subject and getting it right matters more than other things.

I also think that prints are much more forgiving than pixel peeping on screen.
In the real world, everything is a compromise. We have only one plane of perfect focus. That really means that only parts of the image will be optimally sharp.

This image has been hanging on my wall for a dozen years. It was shot on Sony A900 with a Sigma 12-24 zoom. No one ever complained about it not being sharp, folks may be polite...

Best regards

Erik

Leica CL owners are well aware of DxO's support limitations. They refused to do profiles for the camera, despite many, many email conversations (including at least 3 with me) and a write-in campaign of sorts. They do great work, and I own a couple of their products. I just don't understand their decisions about support.

I'm not nearly so technically gifted as those posting here. I mainly invert negatives in LR or PS and try not to do too much in post that robs my photos of their overall filmic look.

But I am so very impressed with Topaz. Their Sharpen and Denoise programs have improved dramatically with each major release. For my limited aesthetic goals, they more than serve the purpose.

When printing large, I spring for professional drum scanning and 600mps files, and even then Topaz can help a bit.

But I am so very fascinated by this thread! Thanks, all.

Bags27 wrote:

Leica CL owners are well aware of DxO's support limitations. They refused to do profiles for the camera, despite many, many email conversations (including at least 3 with me) and a write-in campaign of sorts. They do great work, and I own a couple of their products. I just don't understand their decisions about support.

I'm not nearly so technically gifted as those posting here. I mainly invert negatives in LR or PS and try not to do too much in post that robs my photos of their overall filmic look.

But I am so very impressed with Topaz. Their Sharpen and Denoise programs have improved dramatically with each major release. For my limited aesthetic goals, they more than serve the purpose.

When printing large, I spring for professional drum scanning and 600mps files, and even then Topaz can help a bit.

But I am so very fascinated by this thread! Thanks, all.

I would think profiling a camera and lens system is a significant effort, so DxO probably goes after market share.

Also, I think that camera makers may be more or less involved in supporting DxO-s efforts.

Best regards

Erik

One thing I noticed from Mads' original post is the amazing success of DXO at cleaning aliasing.

The last time I investigated this, I compared Lightroom, Capture 1 and Iridient to what the JPEG/TIFF engine in my 50R can do. It wasn't close. Only the camera repaired what you see here on the left.

But look how well DXO did. Amazing. Mind you, it's hit and miss. It cleaned one silo (left) nearly perfectly, but did less well with squat silo on the right.

DXO (on the right) does a much better job with aliasing than Lightroom (left).