LX3 and dcraw workshop

Started Oct 9, 2008 | Discussions
Conversio Forum Member • Posts: 94
LX3 and dcraw workshop

I think the image above speaks for itself. The main reason of this post is not to criticize Silkypix but to establish a workflow for rw2 files after dcraw development. The use of dcraw has some cons, so it would be nice if we can share our tips in order of create a fluid workflow. Main problems TIFF output has are the following:

-Barrel distortion.
-Flat colors (nothing PP can't solve).
-Presence of vignetting due to extra 0'1 megapixels Silkypix discards.
-Chroma noise.

It's hard to correct this for each isolated photo. How about if we try to do that process easier?

By the moment I can contribute with these things:

-After some trial/error attempts I think -S 7000 works better than dcraw default saturation.

-I have found that it's better to utilize -b 4 in dcraw command line than trying to correct brightness in PP.

-The development process can be speeded up by creating a batch file (windows users only):

@echo off
c:
cd dcraw
dcraw -v -w -H 1 -o 0 -q 3 -T -b 4 -S 7000 -P badpixels.txt %1

Just save it as lx3 h1.bat in c:\dcraw or wherever dcraw.exe is located and drag and drop single rw2 files.

-If you want to recover highlights you have to use -h2 or -h[3-9]. It is easy to create another batch file for this.

Things to do:

-Make a profile for PTLens for each aspect ratio. I believe this should be the main task, as it can solve the barrel distortion and vignetting problems with one click.

-Make the profiles for Noise Ninja. The idea is to apply this the first, so the TIFF output should be used "as is" for the calibration.

-Update dcrawg (dcraw with gamma compensation by Guillermo Luijk) so we can develop to a 16-bit Adobe RGB 1998 TIFF. I will write to Guillermo soon.

Additional info here:

- http://epaperpress.com/ptlens/ (check "Calibration").
- http://www.picturecode.com/profile_upload.php
- http://www.guillermoluijk.com/article/gamma/index.htm

Do you like the idea? Do you have any tips for post-processing the flat TIFF? Any suggestion will be welcome!

-Conversio

ficofico Junior Member • Posts: 27
Re: LX3 and dcraw workshop
-- hide signature --

Great, I have passed this day finding how to set saturation in dcraw using Histogrammar and reading all about Guillermo Luijk ....

This post may be very usefull!!!!

OP Conversio Forum Member • Posts: 94
Re: LX3 and dcraw workshop

Ouch, "-S 7000 -b 4" were the values I used for testing purposes and shouldn't actually be used. It's always better to use "-S 4095 -b 2"! In fact, dcraw sets "-S 4095" by default with LX3, so specifying the saturation level is redundant.

Ficofico, thanks for your comment! Let's hope more people are interested.

Liz Z.
Liz Z. Senior Member • Posts: 1,272
Re: LX3 and dcraw workshop

Conversio wrote:

Things to do:

-Make a profile for PTLens for each aspect ratio. I believe this
should be the main task, as it can solve the barrel distortion and
vignetting problems with one click.

I am already on this one! I have been in touch with Tom Niemann and am about to send him a set of LX3 images for him to calibrate for PTLens. He said the process might be more complex due to the three different aspect ratios, but it can and will be done!

Liz

 Liz Z.'s gear list:Liz Z.'s gear list
Sony RX100 Adobe Photoshop CS6 +1 more
ficofico Junior Member • Posts: 27
Only from Dcraw
-- hide signature --

It's important that images came from dcraw, that not apply any distortion correct....

Liz Z.
Liz Z. Senior Member • Posts: 1,272
Re: Only from Dcraw

Well, I'm sending him jpegs, per his request, and he'll tell me later if he wants RAW images as well.

I don't have DCRAW, nor do I know how to use it........

Liz

ficofico wrote:

 Liz Z.'s gear list:Liz Z.'s gear list
Sony RX100 Adobe Photoshop CS6 +1 more
ficofico Junior Member • Posts: 27
Re: Only from Dcraw
-- hide signature --

Dcraw it'a a simple exutable file, you can download for example from my site

http://www.rigenerator.com/lx3.bat.rar

hinting
hinting Contributing Member • Posts: 857
Re: LX3 and dcraw workshop

So what is with the panasonic sensor/sillkpix with blue led light source???

I took photos of the watercube in Beijing at night, it also shows this high contrast banding that I didn't see with my own eyes..

Why????

:hinting
hinting.multiply.com

 hinting's gear list:hinting's gear list
Leica M Monochrom (Typ 246) Panasonic Lumix DMC-L1 Panasonic Lumix DMC-GX1 Olympus OM-D E-M5 Leica M-Monochrom +69 more
OP Conversio Forum Member • Posts: 94
Re: Only from Dcraw

Liz Z., it's wonderful you have already started the work! If you don't know how to use dcraw maybe you can upload to rapidshare the raw files so we can develop them (assuming you have shot in RAW+jpg the files you send to Tom Niemann).

So what is with the panasonic sensor/sillkpix with blue led light source???

I don't know why, but this is unacceptable for me. This is the kind of problem that appears when processing a jpg several times but shouldn't exist in the developed TIFF.

By the way, I see that you all use the 16 bit output in dcraw. How do you recover the TIFF afterwards? "-4" option is incompatible with brightness adjustment, and I am unable to achieve a decent result in Photoshop.

torrent99 Regular Member • Posts: 101
Re: Only from Dcraw

Hmmm. Is it me or does any one else prefer the Silkypix version?

I'm a fan of dcraw and would like to get it to produce good results for the FZ28,

however at the moment I really can't seem to get the colours to come anywhere near the JPEG version.

At the moment I'm thinking of writing some code to do a pixel by pixel comparison of the JPEG and DCRAW output to try to produce a set of custom curves...

Juan Trujillo Regular Member • Posts: 486
Re: Only from Dcraw

Conversio wrote:

By the way, I see that you all use the 16 bit output in dcraw. How do
you recover the TIFF afterwards? "-4" option is incompatible with
brightness adjustment, and I am unable to achieve a decent result in
Photoshop.

Interesting thread. I started to try DCRAW and arrived to something but frankly I need more time to arrive to some conclussion so for now I can contribute only with the processing of 16bits gamma 1 files from DCRAW.

First read the stuff by Guillermo Luijk at http://www.guillermoluijk.com/tutorial/dcraw/index_en.htm (look under CONVERSION TO A COLOUR SPACE AND GAMMA)

And a more comfortable method (that I learned at a workshop with Guillermo) is having a gamma-1 - DCRAW profile that is assigned to the tiff file.

The thing goes like this:

0) Download the profile: http://jtrujillo.net/DCRAW/DCRAW_gamma_1.icc

1) Install that profile like any other regular profile (C:\WINDOWS\system32\spool\drivers\color)

2) Run PS and load the 16bit TIFF

3) ASSIGN the profile to that image (Edit-> Assign Profile and choose DCRAW gamma1)

Now PShop is aware that the image is in gamma1 so it corrects the luminosity of the image. Notice that here you are working in a linear image, there are operations that cand take advantage of this although PShop is not thought to work with this. What I do at this point is convert the image to a regular colorSpace+gamma by
4) Convert to profile -> Your working profile.

The advantage here is that you have a native 16 bit image and eventually you can do some stuf in linear.

Regards,
Juan
http://TheCharmOfLight.com

Juan Trujillo Regular Member • Posts: 486
Re: Only from Dcraw

Notice that jpegs (and Silkypix) do by themseves correction of the distortion so the jpegs you sent to Tom Nieman are already somehow corrected (I don't know if thats an usual feature, I never seen that before and was surprised when I did my first raw conversion with DCRAW. You may want to warn Tom Nieman of this - for the case that already distortion processed files make any difference)

Regards,
Juan
http://theCharmOfLight.com

OP Conversio Forum Member • Posts: 94
Re: Only from Dcraw

Before buying the LX3 I used to develop with dcrawg, which apply itself the gamma compensation, so it's nice to know that there is the possibility to get an usable and native 16-bit output with an unmodified copy of dcraw. I will try it this afternoon. Many thanks for the info and for the detailed explanation, Juan!

torrent99 wrote:

At the moment I'm thinking of writing some code to do a pixel by pixel comparison of the JPEG and DCRAW output to try to produce a set of custom curves...

That would be awesome! It seems too complex for me... Can you really do that?

torrent99 Regular Member • Posts: 101
Re: Only from Dcraw

At the moment I'm thinking of writing some code to do a pixel by pixel comparison of the JPEG and DCRAW output to try to produce a set of custom curves...

That would be awesome! It seems too complex for me... Can you really
do that?

I don't know yet! My theory is that I'll extract the embbeded JPEG.

The produce the DCRAW image (hopefully with the extra border info removed) in PPM.

Then I go thru each pixel mapping it to the corresponding pixel in the JPEG (scaling will be needed here). I'll take each individual channel and make a note of the corresponding channel value in the JPEG. If there are several instances of the same source value I'll average the destination values.

Hopefully this will produce a curve that maps input (dcraw) to output (jpeg).
Wish me luck!

Initially I look at the curve in Excell or similar, then draw it in PS (until I figure out the curve file format).

Cheers

Steve

OP Conversio Forum Member • Posts: 94
Re: Only from Dcraw

I would like to understand programming at pixel level... Maybe some day.

It seems like a hard work. Of course I wish you luck!

Juan Trujillo Regular Member • Posts: 486
Re: Only from Dcraw

torrent99 wrote:

At the moment I'm thinking of writing some code to do a pixel by pixel comparison of the JPEG and DCRAW output to try to produce a set of custom curves...

I thought something like this to get some kind of color mapping but then is when I discovered amazed and deceived at the same time that the jpegs are distortion corrected so there was an spectacular difference between the two bitmaps.... I only tried with a wide angle shot and got lost on other things so maybe larger focal lehghts may give something meaningful.... Good luck!, let us know if you get anything...

Regards,
Juan
http://TheCharmOfLight.com

Liz Z.
Liz Z. Senior Member • Posts: 1,272
Re: Only from Dcraw

Conversio wrote:

Liz Z., it's wonderful you have already started the work! If you
don't know how to use dcraw maybe you can upload to rapidshare the
raw files so we can develop them (assuming you have shot in RAW+jpg
the files you send to Tom Niemann).

Alas, I just shot them in jpg.... Eventually Tom will ask me for more shots (these were all shot at full wide angle, but in three size formats), and then I can go back to the building I photographed and do them in jpeg + RAW. He told me he is also in conversation with someone else about the RAW files.

-- hide signature --

Liz

 Liz Z.'s gear list:Liz Z.'s gear list
Sony RX100 Adobe Photoshop CS6 +1 more
OP Conversio Forum Member • Posts: 94
Re: Only from Dcraw

Liz Z. wrote:

Alas, I just shot them in jpg.... Eventually Tom will ask me for more
shots (these were all shot at full wide angle, but in three size
formats), and then I can go back to the building I photographed and
do them in jpeg + RAW. He told me he is also in conversation with
someone else about the RAW files.

Liz, thanks anyhow, it doesn't matter. Do you know who is sending those raws to Tom? If we could develop them in dcraw we would save a lot of work!

-Conversio

Liz Z.
Liz Z. Senior Member • Posts: 1,272
Re: Only from Dcraw

In the end I sent Tom the RAWs. The other person didn't live near a good target building.

Do you want me to send you the RAWs I sent Tom? I'd be glad to, though I still don't understand exactly what will save time for what. This thread veers beyond my level of technical comprehension!

I had to upload my files to Tom's ftp site, and it took about two hours, so I don't think emailing them would work--each one is 11.7MB, and there were 8 of them! So if you do want them, tell me how I can get them to you. I could use Yousendit.

Liz

Conversio wrote:

Liz Z. wrote:

Alas, I just shot them in jpg.... Eventually Tom will ask me for more
shots (these were all shot at full wide angle, but in three size
formats), and then I can go back to the building I photographed and
do them in jpeg + RAW. He told me he is also in conversation with
someone else about the RAW files.

Liz, thanks anyhow, it doesn't matter. Do you know who is sending
those raws to Tom? If we could develop them in dcraw we would save a
lot of work!

-Conversio

 Liz Z.'s gear list:Liz Z.'s gear list
Sony RX100 Adobe Photoshop CS6 +1 more
OP Conversio Forum Member • Posts: 94
Re: Only from Dcraw

I think I didn't explain myself clear enough from the beginning, sorry. The fact is that Silkypix corrects by default the barrel distortion of the lens, and dcraw leave that distortion unchanged, so there is a noticeable difference between the output of each developer. An illustrative example at 3:2 & 24mm, both extracted from the same raw file:

What I want to do is to sent Tom the samples of a valid calibration target that have been developed with dcraw, instead Silkypix. So if you have already the right raws but don't know how to use dcraw (if you are interested this is the place to ask!) I will be grateful if you could sent (Yousendit or Rapidshare are ok) them to me so I can develop and upload them to the ftp.

Many thanks for your patience!

-Conversio

Keyboard shortcuts:
FForum MMy threads