Trying to understand Rorshack OctaveRawTools

[JohnMoyer]

Since the color channels are separated in the OctaveRawTools software, I thought it might be interesting to experiment with the Canon DPRAW files.

But, it has been decades since I have used Mathmatica and I have never written code for DNG files and I have forgotten much.

I began experimenting with a photo which I had previously published at: https://www.rsok.com/~jrm/2025Jan31_birds_and_cats/2025jan25_cardinal_IMG_2713c.html

I used the libraw 4channel program to separate the channels into individual TIFF files. I then used subtract to get the half of the photosite data from the combined file ( although I am not completely convinced that the combined file is always created in camera using an "add" operation. )

I resized each individual file to 200% using graphicsmagick "-filter point".

I then added the 2 expanded half files back together. Here is a crop to the eye of the Cardinal bird from the red channel scaled 200%. The Cardinalis cardinalis is a mostly red bird.


Scaled 2x using DPRAW data ; Northern Cardinal (Cardinalis cardinalis) in Norman, Oklahoma, United States on January 25, 2025

Do any know how I might combine the individual channels after scaling so they might be demosaiced by rawtherapee?

Here is a crop done in ratherapee after normal demosaic:


cropped in rawtherapee and upscaled 2x by graphicsmagick also using "-filter point"

--
https://www.rsok.com/~jrm/
John Moyer

 

[Jack Hogan]

Are you trying to feed your hacked DNG raw file to RawTherapee John?

In that case after having worked on the individual raw channels you have to re-mosaic them, meaning that you assemble a CFA file twice as large linearly with four times as many pixels as in each color plane - and position each pixel in a quartet in its proper place (e.g. RG/GB, not sure for your camera). I am not familiar with the mentioned toolkit but this is easy to do in Matlab or Octave.

Jack

 

[TimoKoo]

Since the color channels are separated in the OctaveRawTools software, I thought it might be interesting to experiment with the Canon DPRAW files.

But, it has been decades since I have used Mathmatica and I have never written code for DNG files and I have forgotten much.

I began experimenting with a photo which I had previously published at: https://www.rsok.com/~jrm/2025Jan31_birds_and_cats/2025jan25_cardinal_IMG_2713c.html

I used the libraw 4channel program to separate the channels into individual TIFF files. I then used subtract to get the half of the photosite data from the combined file ( although I am not completely convinced that the combined file is always created in camera using an "add" operation. )

I resized each individual file to 200% using graphicsmagick "-filter point".

I then added the 2 expanded half files back together. Here is a crop to the eye of the Cardinal bird from the red channel scaled 200%. The Cardinalis cardinalis is a mostly red bird.


Scaled 2x using DPRAW data ; Northern Cardinal (Cardinalis cardinalis) in Norman, Oklahoma, United States on January 25, 2025

Do any know how I might combine the individual channels after scaling so they might be demosaiced by rawtherapee?

Here is a crop done in ratherapee after normal demosaic:


cropped in rawtherapee and upscaled 2x by graphicsmagick also using "-filter point"

You can select from many demosaicing methods in RT. What if you try method 'None' to your synthetic DNG?

 

[JohnMoyer]

Are you trying to feed your hacked DNG raw file to RawTherapee John?

In that case after having worked on the individual raw channels you have to re-mosaic them, meaning that you assemble a CFA file twice as large linearly with four times as many pixels as in each color plane - and position each pixel in a quartet in its proper place (e.g. RG/GB, not sure for your camera). I am not familiar with the mentioned toolkit but this is easy to do in Matlab or Octave.

Jack

Thanks for your help. The re-mosaic step is what I am having difficulty understanding even with the Octave source code example in OctaveRawTools. https://github.com/horshack-dpreview/OctaveRawTools?tab=readme-ov-file

I am a slow learner and I have forgotten much. I have not yet managed to create a raw dng from the individual color channel TIFF files. I have forgotten much and Matlab seems to have changed in the past decades since I last used it. Since the new raw file will have 4 times as many pixels, I am having difficulty understanding how to change an existing dng to include the data.

Another example of writing a dng from Octave would be much appreciated.

I prefer rawtherapee because of the choice of demosaic algorithms.

I was thinking that with DPRAW I have extra data and that it might be interesting to try using it to upscale an image. But so far I have failed to create a raw dng file. Still to figure out is how to shift one image half a pixel to the right before summing the 2 images and what to do with borders.

It sounded like a good puzzle, but has been too difficult for me.

I used libraw and graphicsmagick.

<pre>

/home/jrm/src/LibRaw-0.21.2/bin/4channels -s 0 -A -g ../IMG_2713.CR3

/home/jrm/src/LibRaw-0.21.2/bin/4channels -s 1 -A -g ../IMG_2713.CR3

gm composite -verbose -compose minus ../IMG_2713.CR3-1.R.tiff ../IMG_2713.CR3.R.tiff IMG_2713R.tiff

gm composite -verbose -compose minus ../IMG_2713.CR3-1.B.tiff ../IMG_2713.CR3.B.tiff IMG_2713B.tiff

gm composite -verbose -compose minus ../IMG_2713.CR3-1.G.tiff ../IMG_2713.CR3.G.tiff IMG_2713G.tiff

gm composite -verbose -compose minus ../IMG_2713.CR3-1.G2.tiff ../IMG_2713.CR3.G2.tiff IMG_2713G2.tiff

for i in B G2 G R

do

gm convert -verbose ../image0-1/IMG_2713${i}.tiff -filter point -resize "200%" IMG_2713${i}_x2.tiff

done

for i in B G2 G R

do

gm convert -verbose ../IMG_2713.CR3-1.${i}.tiff -filter point -resize "200%" IMG_2713${i}_x2b.tiff

done

for i in B G2 G R

do

gm composite -verbose -compose add IMG_2713${i}_x2.tiff IMG_2713${i}_x2b.tiff IMG_2713${i}_x2sum.tiff

done

</pre>

 

[JohnMoyer]

Since the color channels are separated in the OctaveRawTools software, I thought it might be interesting to experiment with the Canon DPRAW files.

But, it has been decades since I have used Mathmatica and I have never written code for DNG files and I have forgotten much.

I began experimenting with a photo which I had previously published at: https://www.rsok.com/~jrm/2025Jan31_birds_and_cats/2025jan25_cardinal_IMG_2713c.html

I used the libraw 4channel program to separate the channels into individual TIFF files. I then used subtract to get the half of the photosite data from the combined file ( although I am not completely convinced that the combined file is always created in camera using an "add" operation. )

I resized each individual file to 200% using graphicsmagick "-filter point".

I then added the 2 expanded half files back together. Here is a crop to the eye of the Cardinal bird from the red channel scaled 200%. The Cardinalis cardinalis is a mostly red bird.


Scaled 2x using DPRAW data ; Northern Cardinal (Cardinalis cardinalis) in Norman, Oklahoma, United States on January 25, 2025

Do any know how I might combine the individual channels after scaling so they might be demosaiced by rawtherapee?

Here is a crop done in ratherapee after normal demosaic:


cropped in rawtherapee and upscaled 2x by graphicsmagick also using "-filter point"

You can select from many demosaicing methods in RT. What if you try method 'None' to your synthetic DNG?

Thanks for your response. Sorry for not making myself clear. I included the crop from rawtherapee for comparison to the red channel enlarged by summing the DPRAW data.

I do not remember which demosaic algorithm I used in rawtherapee this time.

--

John Moyer's Home Page

John Moyer home page with prime numbers.

www.rsok.com

John Moyer

 

[Horshack]

Hehe, it's Horshack

My OctaveRawTools provides several examples on how to work with the individual CFA channels and recombine them when done. The swapRedBlueChannelsInDng is the simplest:

%
% split the raw data into its separate RGGB channels. this also converts
% the data from its uint16 packing to double floating point
%
[r, g1, g2, b] = rawBayerToChannels(dng.cfaPatternStr, dng.imgData);

% for fun, swap the R and B channels by rebayering the data but with b/r transposed
imgDataOut = channelsToRawBayer(dng.cfaPatternStr, b, g1, g2, r);

% update the DNG with the new data
success = saveRawDataToDng(dngFilename, dng.stripOffset, imgDataOut);

The rawBayerToChannels() returns four matrices of image data, one per CFA channel. You can then do whatever analysis/manipulation on those channels. When done, you can recombine them into the interlaced CFA pattern by calling channelsToRawBayer(), then call saveRawDataToDng() to save the recombined data into a DNG that you can feed into any raw image processor to develop and see the results of your manipulation.

Make sure to use only uncompressed DNG files - my project's readme provides instructions on how to generate them using Adobe's DNG Converter .

 

[JohnMoyer]

Hehe, it's Horshack

My OctaveRawTools provides several examples on how to work with the individual CFA channels and recombine them when done. The swapRedBlueChannelsInDng is the simplest:

%
% split the raw data into its separate RGGB channels. this also converts
% the data from its uint16 packing to double floating point
%
[r, g1, g2, b] = rawBayerToChannels(dng.cfaPatternStr, dng.imgData);

% for fun, swap the R and B channels by rebayering the data but with b/r transposed
imgDataOut = channelsToRawBayer(dng.cfaPatternStr, b, g1, g2, r);

% update the DNG with the new data
success = saveRawDataToDng(dngFilename, dng.stripOffset, imgDataOut);

The rawBayerToChannels() returns four matrices of image data, one per CFA channel. You can then do whatever analysis/manipulation on those channels. When done, you can recombine them into the interlaced CFA pattern by calling channelsToRawBayer(), then call saveRawDataToDng() to save the recombined data into a DNG that you can feed into any raw image processor to develop and see the results of your manipulation.

Make sure to use only uncompressed DNG files - my project's readme provides instructions on how to generate them using Adobe's DNG Converter .

Thanks. I downloaded your source code and read it. Thanks for publishing the source code. The part I am having difficulty with is creating a higher resolution dng than the original. Maybe I just had a bad idea.

I have been trying to do it without a starting dng because there will be 4 times as many pixels in the end as a dng created by Adobe DNG Converter would create. "% Ovewrites the raw CFA data inside an existing uncompressed DNG."

Thanks again.

 

[TimoKoo]

Since the color channels are separated in the OctaveRawTools software, I thought it might be interesting to experiment with the Canon DPRAW files.

But, it has been decades since I have used Mathmatica and I have never written code for DNG files and I have forgotten much.

I began experimenting with a photo which I had previously published at: https://www.rsok.com/~jrm/2025Jan31_birds_and_cats/2025jan25_cardinal_IMG_2713c.html

I used the libraw 4channel program to separate the channels into individual TIFF files. I then used subtract to get the half of the photosite data from the combined file ( although I am not completely convinced that the combined file is always created in camera using an "add" operation. )

I resized each individual file to 200% using graphicsmagick "-filter point".

I then added the 2 expanded half files back together. Here is a crop to the eye of the Cardinal bird from the red channel scaled 200%. The Cardinalis cardinalis is a mostly red bird.


Scaled 2x using DPRAW data ; Northern Cardinal (Cardinalis cardinalis) in Norman, Oklahoma, United States on January 25, 2025

Do any know how I might combine the individual channels after scaling so they might be demosaiced by rawtherapee?

Here is a crop done in ratherapee after normal demosaic:


cropped in rawtherapee and upscaled 2x by graphicsmagick also using "-filter point"

You can select from many demosaicing methods in RT. What if you try method 'None' to your synthetic DNG?

Thanks for your response. Sorry for not making myself clear. I included the crop from rawtherapee for comparison to the red channel enlarged by summing the DPRAW data.

I do not remember which demosaic algorithm I used in rawtherapee this time.

I think you did not get my point.

RT's demosaicing method 'None' means no demosaicing. Jack suggested remosaicing your dng. I thought what happens if you did open your dng in RT without demosaicing because it already was demosaiced.

 

[Horshack]

Hehe, it's Horshack

My OctaveRawTools provides several examples on how to work with the individual CFA channels and recombine them when done. The swapRedBlueChannelsInDng is the simplest:

%
% split the raw data into its separate RGGB channels. this also converts
% the data from its uint16 packing to double floating point
%
[r, g1, g2, b] = rawBayerToChannels(dng.cfaPatternStr, dng.imgData);

% for fun, swap the R and B channels by rebayering the data but with b/r transposed
imgDataOut = channelsToRawBayer(dng.cfaPatternStr, b, g1, g2, r);

% update the DNG with the new data
success = saveRawDataToDng(dngFilename, dng.stripOffset, imgDataOut);

The rawBayerToChannels() returns four matrices of image data, one per CFA channel. You can then do whatever analysis/manipulation on those channels. When done, you can recombine them into the interlaced CFA pattern by calling channelsToRawBayer(), then call saveRawDataToDng() to save the recombined data into a DNG that you can feed into any raw image processor to develop and see the results of your manipulation.

Make sure to use only uncompressed DNG files - my project's readme provides instructions on how to generate them using Adobe's DNG Converter .

Thanks. I downloaded your source code and read it. Thanks for publishing the source code. The part I am having difficulty with is creating a higher resolution dng than the original. Maybe I just had a bad idea.

I have been trying to do it without a starting dng because there will be 4 times as many pixels in the end as a dng created by Adobe DNG Converter would create. "% Ovewrites the raw CFA data inside an existing uncompressed DNG."

Thanks again.

My logic uses the original DNG EXIF data to calculate the size/offsets of the bayer data contained in the DNG, then overwrites the data in-place, which is to say it assumes the output size will match the input. If you'd like to output a larger DNG then you'll need to add logic to support it. Or a simpler idea would be to find/generate a larger "skeleton" DNG outside of the tools with the output size you want, then use that as the DNG you pass into my tools to write your processed data.

 

[JohnMoyer]

Hehe, it's Horshack

...

Thanks again.

My logic uses the original DNG EXIF data to calculate the size/offsets of the bayer data contained in the DNG, then overwrites the data in-place, which is to say it assumes the output size will match the input. If you'd like to output a larger DNG then you'll need to add logic to support it. Or a simpler idea would be to find/generate a larger "skeleton" DNG outside of the tools with the output size you want, then use that as the DNG you pass into my tools to write your processed data.

Thanks again

 

[JohnMoyer]

...

You can select from many demosaicing methods in RT. What if you try method 'None' to your synthetic DNG?

Thanks for your response. Sorry for not making myself clear. I included the crop from rawtherapee for comparison to the red channel enlarged by summing the DPRAW data.

I do not remember which demosaic algorithm I used in rawtherapee this time.

I think you did not get my point.

RT's demosaicing method 'None' means no demosaicing. Jack suggested remosaicing your dng. I thought what happens if you did open your dng in RT without demosaicing because it already was demosaiced.

Thanks again. I used libraw instead of ratherapee to separate the color channels into individual files. I then modified the individual files by upscaling and addition.

I do not know how to create a bayer array from the individual channels and write it into a format that rawtherapee can process. I do not know how to create a dng with 4 times as many pixels as the original. I am old and I have forgotten much and I might be missing something obvious.

At first glance, the red channel file I posted seems to me to have greater resolution than the crop from the original that I created in rawtherapee because rawtherapee had processed one of the 2 DPRAW images at the original resolution.

Thanks again

 

[Horshack]

Hehe, it's Horshack

My OctaveRawTools provides several examples on how to work with the individual CFA channels and recombine them when done. The swapRedBlueChannelsInDng is the simplest:

%
% split the raw data into its separate RGGB channels. this also converts
% the data from its uint16 packing to double floating point
%
[r, g1, g2, b] = rawBayerToChannels(dng.cfaPatternStr, dng.imgData);

% for fun, swap the R and B channels by rebayering the data but with b/r transposed
imgDataOut = channelsToRawBayer(dng.cfaPatternStr, b, g1, g2, r);

% update the DNG with the new data
success = saveRawDataToDng(dngFilename, dng.stripOffset, imgDataOut);

The rawBayerToChannels() returns four matrices of image data, one per CFA channel. You can then do whatever analysis/manipulation on those channels. When done, you can recombine them into the interlaced CFA pattern by calling channelsToRawBayer(), then call saveRawDataToDng() to save the recombined data into a DNG that you can feed into any raw image processor to develop and see the results of your manipulation.

Make sure to use only uncompressed DNG files - my project's readme provides instructions on how to generate them using Adobe's DNG Converter .

Thanks. I downloaded your source code and read it. Thanks for publishing the source code. The part I am having difficulty with is creating a higher resolution dng than the original. Maybe I just had a bad idea.

I have been trying to do it without a starting dng because there will be 4 times as many pixels in the end as a dng created by Adobe DNG Converter would create. "% Ovewrites the raw CFA data inside an existing uncompressed DNG."

Thanks again.

Another simpler idea would be to write your 4x pixels or whatever # as the original pixel count in the DNG, which will yield a crop of the original image but should be enough data for you to analyze the results of the experimental logic you're working on.

For example, if you start with a 6000x4000 DNG, then whatever you logic does that produces a 12000x8000 matrix of data. Crop that 12000x8000 matrix as 6000x4000, which will write only the upper-left quadrant of your output if you start at 0,0. You can start at whatever coordinates you'd like to move the output window around.

 

[JohnMoyer]

Hehe, it's Horshack

My OctaveRawTools provides several examples on how to work with the individual CFA channels and recombine them when done. The swapRedBlueChannelsInDng is the simplest:

...

Thanks again.

Another simpler idea would be to write your 4x pixels or whatever # as the original pixel count in the DNG, which will yield a crop of the original image but should be enough data for you to analyze the results of the experimental logic you're working on.

For example, if you start with a 6000x4000 DNG, then whatever you logic does that produces a 12000x8000 matrix of data. Crop that 12000x8000 matrix as 6000x4000, which will write only the upper-left quadrant of your output if you start at 0,0. You can start at whatever coordinates you'd like to move the output window around.

Thanks. I will try that.

 

[TimoKoo]

...

You can select from many demosaicing methods in RT. What if you try method 'None' to your synthetic DNG?

Thanks for your response. Sorry for not making myself clear. I included the crop from rawtherapee for comparison to the red channel enlarged by summing the DPRAW data.

I do not remember which demosaic algorithm I used in rawtherapee this time.

I think you did not get my point.

RT's demosaicing method 'None' means no demosaicing. Jack suggested remosaicing your dng. I thought what happens if you did open your dng in RT without demosaicing because it already was demosaiced.

Thanks again. I used libraw instead of ratherapee to separate the color channels into individual files. I then modified the individual files by upscaling and addition.

I do not know how to create a bayer array from the individual channels and write it into a format that rawtherapee can process. I do not know how to create a dng with 4 times as many pixels as the original. I am old and I have forgotten much and I might be missing something obvious.

You asked in your OP "Do any know how I might combine the individual channels after scaling so they might be demosaiced by rawtherapee?"

And "Here is a crop done in ratherapee after normal demosaic:"

My idea was opening it without demosaicing. I'm asking you to open your dng in RT, going to the raw tab, selecting the demosaicing method "None" ,looking at the preview.

At first glance, the red channel file I posted seems to me to have greater resolution than the crop from the original that I created in rawtherapee because rawtherapee had processed one of the 2 DPRAW images at the original resolution.

This goes over my skills.

Thanks again

 

[John Sheehy]

I was thinking that with DPRAW I have extra data and that it might be interesting to try using it to upscale an image.

The two images in a Canon dual-pixel RAW are a full aperture image and a "half" aperture image. It is either the right or left of the aperture, and is more like a gradient than an actual half-circle in the bokeh, and there is a gradient of color tint as well. You can get the other half-aperture image through subtraction of the given half-aperture image from the full-aperture image, but if the full-aperture image is clipped, the clipped areas will be too low in value when you subtract from them.

There is no extra 2-D spatial resolution in the DPRAw file; just two different apertures, which should only be different in OOF areas. You can get more headroom in the half-aperture image, since it only gives about 1/2 the normal values, but again, there is a color tint gradient from left to right in the half-aperture image.

 

[John Sheehy]

...

You can select from many demosaicing methods in RT. What if you try method 'None' to your synthetic DNG?

Thanks for your response. Sorry for not making myself clear. I included the crop from rawtherapee for comparison to the red channel enlarged by summing the DPRAW data.

I do not remember which demosaic algorithm I used in rawtherapee this time.

I think you did not get my point.

RT's demosaicing method 'None' means no demosaicing. Jack suggested remosaicing your dng. I thought what happens if you did open your dng in RT without demosaicing because it already was demosaiced.

Thanks again. I used libraw instead of ratherapee to separate the color channels into individual files. I then modified the individual files by upscaling and addition.

I do not know how to create a bayer array from the individual channels and write it into a format that rawtherapee can process. I do not know how to create a dng with 4 times as many pixels as the original. I am old and I have forgotten much and I might be missing something obvious.

Only a quad-CFA demosaic method can handle the DNG that you seem to want to create. What you might do is double each pixel vertically, and use the two half-aperture pixel values as the left 2 and right 2, but again, as I mentioned in another post, you will have no real increase in spatial resolution of the subject, and you will have checkerboard artifacts due to each half-aperture channel having a different spectral response from left to right in the sub-images.

At first glance, the red channel file I posted seems to me to have greater resolution than the crop from the original that I created in rawtherapee because rawtherapee had processed one of the 2 DPRAW images at the original resolution.

It doesn't have more resolution; it has more aliasing, as a stripped-out red raw channel usually does with sharp optics for the pixel density.

 

[JohnMoyer]

I was thinking that with DPRAW I have extra data and that it might be interesting to try using it to upscale an image.

The two images in a Canon dual-pixel RAW are a full aperture image and a "half" aperture image. It is either the right or left of the aperture, and is more like a gradient than an actual half-circle in the bokeh, and there is a gradient of color tint as well. You can get the other half-aperture image through subtraction of the given half-aperture image from the full-aperture image, but if the full-aperture image is clipped, the clipped areas will be too low in value when you subtract from them.

There is no extra 2-D spatial resolution in the DPRAw file; just two different apertures, which should only be different in OOF areas. You can get more headroom in the half-aperture image, since it only gives about 1/2 the normal values, but again, there is a color tint gradient from left to right in the half-aperture image.

Thanks. I did the subtract before doing the upscale and add.

Since the 2 half images have a slightly different angle of view, it seems like if I were more clever then I should be able to do something like the DPRAW tool in the Canon DPP software and get slightly more horizontal resolution.

I disagree on one minor point, there is slightly more horizontal spacial resolution, but no more vertical resolution.

If one notices the tiny black feathers on the red beak, then it seems to me that there is more resolution in the 2 half images combined by upscaling and adding than in the 2 images combined in the camera and upscaling later. I attribute this to the slightly different angle of view between the 2 images.

I apologize for not labeling the 2 images clearly.

 

[JohnMoyer]

...

...

At first glance, the red channel file I posted seems to me to have greater resolution than the crop from the original that I created in rawtherapee because rawtherapee had processed one of the 2 DPRAW images at the original resolution.

It doesn't have more resolution; it has more aliasing, as a stripped-out red raw channel usually does with sharp optics for the pixel density.

Thanks again.

Does the anti-aliasing filter not prevent the aliasing?

 

[John Sheehy]

At first glance, the red channel file I posted seems to me to have greater resolution than the crop from the original that I created in rawtherapee because rawtherapee had processed one of the 2 DPRAW images at the original resolution.

It doesn't have more resolution; it has more aliasing, as a stripped-out red raw channel usually does with sharp optics for the pixel density.

Thanks again.

Does the anti-aliasing filter not prevent the aliasing?

No; the AA filter is too weak for the red and blue channels. If it were strong enough for them, it would be too strong overall.

 

[JohnMoyer]

At first glance, the red channel file I posted seems to me to have greater resolution than the crop from the original that I created in rawtherapee because rawtherapee had processed one of the 2 DPRAW images at the original resolution.

It doesn't have more resolution; it has more aliasing, as a stripped-out red raw channel usually does with sharp optics for the pixel density.

Thanks again.

Does the anti-aliasing filter not prevent the aliasing?

No; the AA filter is too weak for the red and blue channels. If it were strong enough for them, it would be too strong overall.

Thanks again. I am still learning