Canon Pixel Problems - Mapping a Possibility?

It has been suggested that here may be the place to ask for some help.

After posting earlier and discovering that indeed all 2000 odd images from yesterday's wedding contain a dead/hot pixel line, I hope that someone here an answer.

https://www.dropbox.com/s/z6vo4ksjv8qehlx/IMG_0326.CR2?dl=0

The images all contain a single line in two directions which cross at a particular point in every shot. This was a brand new camera body which was sent back today however i still need to do the edit in the wedding.

Can I 'map out' the issue or is this simply a case of closing and trying to create an action in PS? It was suggested that a code could be made to map it out but that is beyond my capabilities! (even if I knew what it was)

Only pixel software I found is Windows only. (mac here)

What are my options? Anything other than PS marathon session?

I don't use PS and am a Windows user, but if PS has a macro recorder it should be possible to record a macro, edit the macro to finesse it and then batch process all the images with this macro.

Assuming you took the shots all at the same size, the images would first need to be split into two groups: Portrait; Landscape. There probably is a EXIF tag that indicates if a shot was taken in Portrait mode. If so, you could filter on this.

Otherwise, use any program that can sort the 2,000 image files based on height, select the first group with the same height and move them into a Portrait or Landscape folder. Move the remainder into the other folder.

Now you load an image from a group and record a macro where you make a selection consisting of 1px height at the line below the affected line and extending the full width, then copy this selection up one line, save changes and close the file.

Since it's hard to mark a selection 1px high and copy it exactly 1px higher, I'd then edit the recorded macro so the position co-ordinates of the selection rectangle's corners and the paste location where absolutely correct. Then I'd load each of the remaining images in this orientation group and batch process them, automatically loading & running the same macro.

You would then need to repeat the macro recording/finessing a 2nd time to suit the other orientation group.

Dan.

This has to be done at a RAW level. One column of pixels is dimmer than the rest and could be interpolated. I was the one who suggested that. I understand that this is takes time.

Is there a RAW converter which would take this as a pattern noise and fix it automatically?

Photoshop supports batch operations, where it goes by itself through a folder of files and performs your batch operation to fix the lines.

For the fix, you could look at 'convolve' filter with a matrix like:

0 1 0

0 0 0

0 1 0

in a selection mask for a horizontal line (the 1's should average the pixel below and above to the center).

Probably best asked on a Photoshop forum though.

J A C S said:

This has to be done at a RAW level. One column of pixels is dimmer than the rest and could be interpolated. I was the one who suggested that. I understand that this is takes time.

Is there a RAW converter which would take this as a pattern noise and fix it automatically?

Click to expand...

Logic says that any RAW converter which can use flat field corrections should correct this perfectly .. if a flat field is available (better would be one with no vignette ..)

--
Ilias

TM_Photo said:

It has been suggested that here may be the place to ask for some help.

After posting earlier and discovering that indeed all 2000 odd images from yesterday's wedding contain a dead/hot pixel line, I hope that someone here an answer.

https://www.dropbox.com/s/z6vo4ksjv8qehlx/IMG_0326.CR2?dl=0

The images all contain a single line in two directions which cross at a particular point in every shot. This was a brand new camera body which was sent back today however i still need to do the edit in the wedding.

Can I 'map out' the issue or is this simply a case of closing and trying to create an action in PS? It was suggested that a code could be made to map it out but that is beyond my capabilities! (even if I knew what it was)

Only pixel software I found is Windows only. (mac here)

What are my options? Anything other than PS marathon session?

Click to expand...

You can try dcraw. This is somewhat limiting in terms of raw processing (e.g., you have AHD or PPG or VNG demosaicing, no capture sharpening, limited white balance options), but it can be automated fairly easily.

Here is an example of a potential dcraw command:

dcraw -T -4 -q 3 -P bad_pixels.txt input_file.cr2

The format of the "bad_pixels.txt" file contains three space-separated integers per line:

column row time

where column and row are self explanatory (but see note 1 below), and time is the timestamp after which the bad pixel first appeared. Looking at the code of dcraw, I would say that setting the "time" column to the value 2147483647 for all bad pixels should be safe.

You can probably use a spreadsheet to generate the bad pixel list for your bad row and column. I will leave the batch operation as an exercise to the reader, but it should be fairly straightforward to find an example of a DOS batch file (.BAT) that will apply the dcraw command to all the files in a given directory.

notes:

1. You probably want to run dcraw on one of the files (without a -P) first to see the actual dimensions of the output image. Normally a dcraw output file has a few extra rows and columns compared to what you normally see in a OOC JPEG. In other words, you may have to experiment a little to obtain the correct row / column numbers of the bad row/column.

2. The "-q 3" option selects AHD demosaicing. This is a reasonable choice, given the options available.

3. The "-4" options produces linear 16-bit output, without any white balancing. You could try "-6 -w -g 1 1" in its place to produce linear 16-bit output with camera white balance.

-Frans

This was the only solution within my skillset, and while I had hoped it would be just a single row o pixels it turns out to be one but with the rows either side also affected slightly. I have done a few single image edits and still fairly evident using both clone and content aware tools.

So I am best to leave all files as they are, select out those I need, split into landscape and portrait orientations.

Then....? The code solutions mentioned, I assume are all outside of PS? If I looked at batching them with an cloning type action will PS allow the to remain as CR2 files?

There is also a possibility to treat in post processing but it depends on what exactly the raw processor does behind the scenes.

My Photoshop CC wouldn't open the file so I created a TIF from RawDigger. The damage to the single column spread into the 2 adjacent columns. It was straightforward to fix a 3 row defect using a 3 row mask and a custom filter to interpolate data to fill those damaged rows.

I updated my version of PS CC so it opened the CR2 file. Unfortunately the PS CC raw convertor spread the damage well beyond 3 columns which made repair impractical.

So if you are prepared to use a raw convertor that limits the damage to 3 adjacent rows/columns (or find a way to limit the PS CC damage to 3 rows/cols) I think there is a good chance to fix it in post processing.

Once you have recorded a PS action to perform the repair, I think it can be run as a batch operation.

Mark

I converted his CR2 to a TIFF using dcraw and then applied this solution in Gimp and it worked perfectly. It's scriptable, too. All free software, OP.

Thank you for looking into it. I just wanted to know if there is anything I can do before I start the main edit and then look at a PS action.

Perhaps best to move over to the PS forum and ask what the most effective cloning method will be. I have tried clone tool and content aware but there is still evidence of a line, albeit better than it was.

Did you try the convolve filter mentioned above? It works perfectly and you don't know the line was ever there.

Thanks Paul.

Doing my best to have a go at this but think I am out of my depth. Encouraging to hear it worked on the sample image though. I don't wan't to start edit on the whole wedding yet as I guess the repair needs to be done first?

Does anyone do this sort of thing on a freelance basis?

It's either that or a 'for dummies' guide on working with dcraw or similar.

I have a solution that might be helpful and sent you a Private Message (PM) but have not heard back.