sx50 Canon Raw - How does it compare to CHDK raw files?

Started Jan 8, 2013 | Discussions
Coyote_Cody Senior Member • Posts: 1,166
sx50 Canon Raw - How does it compare to CHDK raw files?

(Reason for question - buy the sx40 & use CHDK or sx50 for about $160 more for Canon Raw - I like Raw as an option)

I suspect this may not yet be tested (a usable CHDK for sx50 yet?), but maybe someone with an CHDK sx40 & sx50 could give some insight or test things.

- How do file sizes compare?

- Does the Canon Raw file give you the 'inside of the lens barrel' when lens is at the widest possible setting? CHDK is a raw dump of sensor data to DNG, no cropping.

On CHDK DNG raws, you will actually 'see' the sides/end of the lens barrel & more importantly can have a small but significantly 'wider' lens, maybe on the order of 2-4mm, on less wide no barrel is visible but still wider than jpeg (for me).

My testing of my very small portable Elph 300hs & earlier on sx260, it seems 2-3mm wider than their approx. 24mm, especially on macro shots where lens distortion/flat field is not as important or visible.

- Using LR4.x/PS-CSx, do/can you get similar results from the CHDK vs Canon Raw files (I assume some version of LR or Camera raw will process sx50 Raw by now)?

- How do the raw files 'look', do they contain the same amount of noise & such?

Thanks for any insight/answers!

-- hide signature --

The Light is Right - Take that Shot!!
Check out my photo galleries @
http://picasaweb.google.com/TauPhoto

Don_Campbell Veteran Member • Posts: 3,108
Re: sx50 Canon Raw - How does it compare to CHDK raw files?
1

I see no one is jumping in with answers closer to the specifics of your question so I'll give some answers that may help even if slightly off the mark.

Coyote_Cody wrote:

(Reason for question - buy the sx40 & use CHDK or sx50 for about $160 more for Canon Raw - I like Raw as an option)

I suspect this may not yet be tested (a usable CHDK for sx50 yet?), but maybe someone with an CHDK sx40 & sx50 could give some insight or test things.

- How do file sizes compare?

SX50 HS version of CHDK is still in the pre-alpha phase of porting. I haven't tried CHDK for it yet but will probably use it for some things when it is stable.

I have used CHDK, mostly for producing raw files, with my SX20 IS.

- Does the Canon Raw file give you the 'inside of the lens barrel' when lens is at the widest possible setting? CHDK is a raw dump of sensor data to DNG, no cropping.

Well, one can get a file called raw from CHDK that you assign a suffix file type to, but it is not usually too useful in that form because those raws are not directly supported by raw conversion software. What is useful is the CHDK-produced DNG file type. DNG being the Adobe "Digital Negative" standardized raw format. The DNG files I get from CHDK with the SX20 do not have wild distortion and I cannot "see the sides/end of the lens barrel." They are a few pixels taller than the jpgs OOC 4000x3004 vs 4000x3000.

The native raws produced by the SX50HS can be directly processed by RawTherapee to jpgs that are 4064x3037 pixels. I assume that Adobe's LR and PS updates are now converting SX50 raws. The DNGs created by Adobe's DNGConverter become jpgs that are 4064x3036. I have only just started dealing directly with RT on CR2 files so I don't have a whole lot to say about the difference.

The DNGs from CHDK from my SX20 are about 18MB. The CR2s from SX50 are on the order of 14 MB and the DNGs from Adobe's DNGConverter are about 12.5MB.

The original description of CHDK DNG files on the CHDK site says that they are limited to 10-bits per channel color depth. It is possible that the DNGConverter DNGs are similarly limited. The Canon release about the SX50HS says it produces 12-bit per channel raws. If the CHDK DNGS were still limited to 10 bits of color depth then that would be a strong reason for skipping CHDK-produced DNGs. I have no idea what CHDK would do with camera settings of raw or raw + jpg but other CHDK settings enabled. Some others who have used CHDK with cameras that also produce native raws could maybe tell us.

I don't see distortion as you describe for other CHDK raws.

On CHDK DNG raws, you will actually 'see' the sides/end of the lens barrel & more importantly can have a small but significantly 'wider' lens, maybe on the order of 2-4mm, on less wide no barrel is visible but still wider than jpeg (for me).

My testing of my very small portable Elph 300hs & earlier on sx260, it seems 2-3mm wider than their approx. 24mm, especially on macro shots where lens distortion/flat field is not as important or visible.

- Using LR4.x/PS-CSx, do/can you get similar results from the CHDK vs Canon Raw files (I assume some version of LR or Camera raw will process sx50 Raw by now)?

I get fine results from RawTherapee.

It bears repeating that the OOC jpgs are generally quite good for the SX50 and so the imperative of using raw mode is not as great. I still like it, but I can forgo it when using some of the other special modes of the camera.

- How do the raw files 'look', do they contain the same amount of noise & such?

Raws are raw and they contain all the noise intrinsic to the sensor. Canon's DPP applies NR to them that makes them look a lot like the OOC jpgs which have an impressive tradeoff between detail and NR that makes midrange ISOs useful.

I haven't used DPP much because I generally do work in Linux. However, interestingly when you import a raw into DPP the noise is obvious initially and then after 10-20 seconds (I only "give" WinXP in a virtual machine 2 virtual processors) the NR is applied and the image is quite like the OOC jpg. When imported into other applications the default NR may be less and so a bit of work is required to make them come out as nice as Canon produces both in camera and in DPP.

Thanks for any insight/answers!

I would summarize my thoughts this way: I think it is likely that the DNG files produced by CHDK for the SX40 are limited to 10-bits per channel unless you get an authoritative answer to the contrary. The 12-bits per channel of color depth of the SX50 CR2 raws should intrinsically permit a wider range of adjustment of color, exposure compensation and so on than 10-bits would permit. You will still find that you are butting up against small-sensor noise when processing raws. You might find that the terrific tradeoff between NR and detail produced by DPP or the OOC jpgs is reason enough to go for the SX50 over the SX40. DPP isn't replete with controls but the few controls that it has produce fine results with SX50 raws.

Then there are the other advantages of the SX50 which include the astonishing zoom range with corresponding terrific IS. In camera processing does a great job of getting good images at ISOs up to 800 with not so bad images at 1600. Many of the interesting shooting modes are not available with raw output but that has become less restrictive considering the high quality of the OOC jpgs.

Don

 Don_Campbell's gear list:Don_Campbell's gear list
Canon EOS 7D Mark II Canon EOS R5 Sigma 150-600mm F5-6.3 | C Canon RF 100-500mm F4.5-7.1L IS USM
(unknown member) Veteran Member • Posts: 3,073
Re: sx50 Canon Raw - How does it compare to CHDK raw files?

Don,

Thanks for your observations.  It is very interesting reading for me since I am new to bridge cameras having just purchased the SX50.  I am enjoying both JPGs and RAW files and I'm using DPP to develope the RAW files as I use to with my Canon DSLRs.  I wish there was a C3 and C4 option along with the C1 and C2 options so I could save a couple more range of preset settings :0)  I'm not sure I'll need the hack for this camera??

Murry

-- hide signature --
Don_Campbell Veteran Member • Posts: 3,108
Re: sx50 Canon Raw - How does it compare to CHDK raw files?

MurryG wrote:

Don,

Thanks for your observations. It is very interesting reading for me since I am new to bridge cameras having just purchased the SX50. I am enjoying both JPGs and RAW files and I'm using DPP to develope the RAW files as I use to with my Canon DSLRs. I wish there was a C3 and C4 option along with the C1 and C2 options so I could save a couple more range of preset settings :0) I'm not sure I'll need the hack for this camera??

Murry

I'm not sure I will need CHDK either. As I said above, I have used CHDK for the SX20 for a couple of years and almost entirely for the ability to shoot raw images. CHDK's battery charge indicator has been helpful but Canon has incorporated that.

There are a lot of other things CHDK does, but I've  just never gotten into exploiting those capabilities. Ultrafast shutter speeds, short duration flash, intervalometer, long duration exposures and so on are useful for folks who have the interest but except perhaps for more exposure or focus bracketing choices I'm pretty impressed with what I get from the SX50 out of the box. I doubt I'll end up using half of its bells and whistles and scenes/modes. If CHDK could improve the woeful EVF, I'd be pacing back and forth waiting in line. It can't so I'm not holding my breath.

I really like the SX50. Like so many folks are saying, it's really a fun machine and part of that fun is putting out pretty nice images under challenging conditions.

Don

 Don_Campbell's gear list:Don_Campbell's gear list
Canon EOS 7D Mark II Canon EOS R5 Sigma 150-600mm F5-6.3 | C Canon RF 100-500mm F4.5-7.1L IS USM
I2K4
I2K4 Senior Member • Posts: 1,465
Re: sx50 Canon Raw - How does it compare to CHDK raw files?

I don't know if the new CHDK for SX50 is going to provide RAW/DNG, but a couple of things to "keep an eye on" regarding the relation between RAW and jpeg output:

. Canon's native RAW Powershots disable the func/set setting for jpeg exposures when shooting RAW + jpeg.  By contrast, CHDK does not interfere with the full range of Canon jpeg settings.  So there's a notable difference in OOC jpeg output, advantage to CHDK, for those who like to shoot in dual RAW + jpeg.

. Canon no longer  provides "SuperFine" jpeg compression, but CHDK has restored it on various affected cameras.  This is the case for a little Ixus I recently bought, and the difference is a noticeable improvement.  (It just about doubles the size of the jpeg file,  and even the old "SuperFine" icon appears in the Canon review display so it seems the Canon firmware still recognizes it.)

Of course, CHDK is worth installing for many other reasons.

Don_Campbell Veteran Member • Posts: 3,108
Re: sx50 Canon Raw - How does it compare to CHDK raw files?

I2K4 wrote:

I don't know if the new CHDK for SX50 is going to provide RAW/DNG, but a couple of things to "keep an eye on" regarding the relation between RAW and jpeg output:

. Canon's native RAW Powershots disable the func/set setting for jpeg exposures when shooting RAW + jpeg. By contrast, CHDK does not interfere with the full range of Canon jpeg settings. So there's a notable difference in OOC jpeg output, advantage to CHDK, for those who like to shoot in dual RAW + jpeg.

That's good to know. However, if the DNG produced by CHDK is 10-bits per channel instead of the native raw's 12-bits per channel it might be a less useful tradeoff sometimes.

. Canon no longer provides "SuperFine" jpeg compression, but CHDK has restored it on various affected cameras. This is the case for a little Ixus I recently bought, and the difference is a noticeable improvement. (It just about doubles the size of the jpeg file, and even the old "SuperFine" icon appears in the Canon review display so it seems the Canon firmware still recognizes it.)

The SX50 has a superfine setting for jpg in addition to fine. I've not done the same level of testing with the two settings in the SX50, but I can say that jpegsnoop.exe shows the same degree of compression for each that is seen in virtually all other Canons and my tests suggest that it is the rare image indeed that even a keen observer could choose between the two with a score better than would be produced by chance. That is a long story but it can be resurrected if the argument starts afresh.

Of course, CHDK is worth installing for many other reasons.

If you have the application and interest and CHDK has the function it is a terrific thing to have.

Don

 Don_Campbell's gear list:Don_Campbell's gear list
Canon EOS 7D Mark II Canon EOS R5 Sigma 150-600mm F5-6.3 | C Canon RF 100-500mm F4.5-7.1L IS USM
OP Coyote_Cody Senior Member • Posts: 1,166
Re: sx50 Canon Raw - How does it compare to CHDK raw files?

Don, thanks for the responses.

In the regard to "raw is raw', well obviously that may not be true if CHDK is 10 bit and Canon is 12 bit, Correct?

The CHDK DNG is a dump of sensor data (found in the memory of the camera-part of porting issues) into a DNG formmated file structure (the other CHDK raw file is pure sensor data & generally not useful as you noted), nothing more or less?, thus the bit depth is whatever it is, provided by the sensor hardware/fw in camera, unless CHDK is processing the raw data with DNG software tools of some vintage age that can not do more than 10 bit in the file structure, throwing away the extra data, DNG certainly supports more.

The 'real' bit depth I believe is purely a function of the camera Hardware sensor D/A converter not s/w, although it may be f/w selectable (ie bit depth selectable depending on sensor hardware- have seen this) and if it is s/w then why not offer a bit depth of 16 or 24 or more for all cameras?

No it is a function of the D/A in camera, so why would CHDK reduce it somehow, I suspect the 10bit statement is from earlier years as P&S cams are/were 10bit depth, not 12, 14 or 16 bit. It lessens the file sizes & DSP processing the less the bit depth.

As to the 'lens distortion', IF you do not see lens distortion via Canon raw files, then SURELY Canon is processing the raw file either in-camera or RawT. is auto-magically before you see it, because it is there.

Lens distortion in the sx50 should be seen in Canon raw (I certainly see it in DPP with my DSLR's & Canon lens) & certainly will be seen in CHDK DNG files, thus the questions of the 'barrel' being seen in widest angle shots.

I think the sx50 has more than raws as a benefit over the sx40 but still not sure worth $160 or more, I will test/play with one next week.

Thanks again Don for your insights! Enjoy your fine camera!!

-- hide signature --

The Light is Right - Take that Shot!!
Check out my photo galleries @
http://picasaweb.google.com/TauPhoto

OP Coyote_Cody Senior Member • Posts: 1,166
Re: sx50 Canon Raw - How does it compare to CHDK raw files?

Don, one script I find nice is the 'motion detection' script which uses the AF sensors to detect motion and will then take a shot. This is a great script for animal triggered shots.

The other you mentioned is nice too, the intervalometer one as you say, if one wants that sort of pic.

The lightning script is also cool, when it detects a burst of light, it shoots!

ALL of these could be in Canon cams from the factory, some would be a great selling feature, Canon seems very conservative vs Sony (ex. sweep panoramic) & others, its a real shame! Its just a little rom/ram memory use!!

Don_Campbell wrote:

MurryG wrote:

Don,

Thanks for your observations. It is very interesting reading for me since I am new to bridge cameras having just purchased the SX50. I am enjoying both JPGs and RAW files and I'm using DPP to develope the RAW files as I use to with my Canon DSLRs. I wish there was a C3 and C4 option along with the C1 and C2 options so I could save a couple more range of preset settings :0) I'm not sure I'll need the hack for this camera??

Murry

I'm not sure I will need CHDK either. As I said above, I have used CHDK for the SX20 for a couple of years and almost entirely for the ability to shoot raw images. CHDK's battery charge indicator has been helpful but Canon has incorporated that.

There are a lot of other things CHDK does, but I've just never gotten into exploiting those capabilities. Ultrafast shutter speeds, short duration flash, intervalometer, long duration exposures and so on are useful for folks who have the interest but except perhaps for more exposure or focus bracketing choices I'm pretty impressed with what I get from the SX50 out of the box. I doubt I'll end up using half of its bells and whistles and scenes/modes. If CHDK could improve the woeful EVF, I'd be pacing back and forth waiting in line. It can't so I'm not holding my breath.

I really like the SX50. Like so many folks are saying, it's really a fun machine and part of that fun is putting out pretty nice images under challenging conditions.

Don

-- hide signature --

The Light is Right - Take that Shot!!
Check out my photo galleries @
http://picasaweb.google.com/TauPhoto

OP Coyote_Cody Senior Member • Posts: 1,166
Re: sx50 Canon Raw - How does it compare to CHDK raw files?

I2K4 wrote:

I don't know if the new CHDK for SX50 is going to provide RAW/DNG, but a couple of things to "keep an eye on" regarding the relation between RAW and jpeg output:

. Canon's native RAW Powershots disable the func/set setting for jpeg exposures when shooting RAW + jpeg. By contrast, CHDK does not interfere with the full range of Canon jpeg settings. So there's a notable difference in OOC jpeg output, advantage to CHDK, for those who like to shoot in dual RAW + jpeg.

. Canon no longer provides "SuperFine" jpeg compression, but CHDK has restored it on various affected cameras. This is the case for a little Ixus I recently bought, and the difference is a noticeable improvement. (It just about doubles the size of the jpeg file, and even the old "SuperFine" icon appears in the Canon review display so it seems the Canon firmware still recognizes it.)

Of course, CHDK is worth installing for many other reasons.

I agree, superfine & raw no limits alone is worth it, plus all those scripts!!

-- hide signature --

The Light is Right - Take that Shot!!
Check out my photo galleries @
http://picasaweb.google.com/TauPhoto

Don_Campbell Veteran Member • Posts: 3,108
Re: sx50 Canon Raw - How does it compare to CHDK raw files?

Coyote_Cody wrote:

Don, thanks for the responses.

In the regard to "raw is raw', well obviously that may not be true if CHDK is 10 bit and Canon is 12 bit, Correct?

I said "raw is raw" with respect to noise in the raw image. Real raw files have no noise reduction applied. Most folks only view raw files after importing into an application that will apply some tweaking in making them visible. DPP opens the raw and automatically applies NR--at least in my hands on my machine in a VM of Windows.

The issue of 10-bits vs 12-bits is a real issue if it's true and how do we find that out? I sure don't know. The 12-bits per channel statement I took as most authoritative is from a Canon PR. The 10-bits number is based on a statement in the CHDK forums by the CHDK authorities in 2008. Is it still that? Who knows?

The CHDK DNG is a dump of sensor data (found in the memory of the camera-part of porting issues) into a DNG formmated file structure (the other CHDK raw file is pure sensor data & generally not useful as you noted), nothing more or less?, thus the bit depth is whatever it is, provided by the sensor hardware/fw in camera, unless CHDK is processing the raw data with DNG software tools of some vintage age that can not do more than 10 bit in the file structure, throwing away the extra data, DNG certainly supports more.

"Pure sensor dump" is words without detail or the voice of authority. The only authoritative words I've found suggest that DNGs produced by CHDK are limited to 10 bits of color depth per color channel. Until we get something authoritative that is different about the SX50 it stands as a worry.

The 'real' bit depth I believe is purely a function of the camera Hardware sensor D/A converter not s/w, although it may be f/w selectable (ie bit depth selectable depending on sensor hardware- have seen this) and if it is s/w then why not offer a bit depth of 16 or 24 or more for all cameras?

Whoa. It is the A/D converter that is at issue here. But it is also how that information gets mashed into a DNG format. How they do that may be by some convenient algorithm that they (CHDK developers) have no particular interest in changing for one camera or a few cameras.

No it is a function of the D/A in camera, so why would CHDK reduce it somehow, I suspect the 10bit statement is from earlier years as P&S cams are/were 10bit depth, not 12, 14 or 16 bit. It lessens the file sizes & DSP processing the less the bit depth.

Sorry. You sidestep the idea that stuff in CHDK is in at least two parts: the algorithms and routines that work for all cameras; the special "glue" that links to those routines for a particular camera. I can easily see how they wouldn't create 12-bit per channel data handling software for one or a couple of cameras. I can also see that they might have worked out ways to let the camera do its own thing for raws and just kept CHDK's hands off. I don't know.

What would be nice is if they left the 12-bit per channel paths in the software and let the camera produce its own raws and allowed it to do that for a wider range of camera shooting modes than is allowed in the stock camera. I have no idea if that's what will be done or has been done with other cameras.

As to the 'lens distortion', IF you do not see lens distortion via Canon raw files, then SURELY Canon is processing the raw file either in-camera or RawT. is auto-magically before you see it, because it is there.

I don't "see" the raw file in raw format. The only way I "see" a raw file is as imported and preliminarily processed by a converter software package. I don't know what level of  geometric processing has happened where in the chain of data movement.

Lens distortion in the sx50 should be seen in Canon raw (I certainly see it in DPP with my DSLR's & Canon lens) & certainly will be seen in CHDK DNG files, thus the questions of the 'barrel' being seen in widest angle shots.

I can't say that I remember if any 4.3mm wide angle images were among those I've looked at in DPP. I've used DPP almost not at all. I've used RT almost entirely as a converter of DNGs to standard color images. The Adobe DNG Converter program certainly produces corrections for lens distortions. You may be right that DPP would show that lens distortion for a photo taken at a 4.3 mm focal length.

I think the sx50 has more than raws as a benefit over the sx40 but still not sure worth $160 or more, I will test/play with one next week.

You'll have to look deeply into the various strengths it has. I would never (and didn't) buy an SX40 to replace my SX20. I did buy an SX50 to replace it and did so only after reading user's reactions, reading reviews and seeing photos that the camera produced. I knew that the EVF was woeful (way worse than the SX20's) but that users were having a lot of fun and taking nice photos. When the camera went on sale in my town I bought it. I have been very happy with image quality for a small sensor camera. I accept the woeful EVF which I knew it had.

Don

Thanks again Don for your insights! Enjoy your fine camera!!

-- hide signature --

The Light is Right - Take that Shot!!
Check out my photo galleries @
http://picasaweb.google.com/TauPhoto

 Don_Campbell's gear list:Don_Campbell's gear list
Canon EOS 7D Mark II Canon EOS R5 Sigma 150-600mm F5-6.3 | C Canon RF 100-500mm F4.5-7.1L IS USM
Don_Campbell Veteran Member • Posts: 3,108
Re: sx50 Canon Raw - How does it compare to CHDK raw files?

Coyote_Cody wrote:

I2K4 wrote:

I don't know if the new CHDK for SX50 is going to provide RAW/DNG, but a couple of things to "keep an eye on" regarding the relation between RAW and jpeg output:

. Canon's native RAW Powershots disable the func/set setting for jpeg exposures when shooting RAW + jpeg. By contrast, CHDK does not interfere with the full range of Canon jpeg settings. So there's a notable difference in OOC jpeg output, advantage to CHDK, for those who like to shoot in dual RAW + jpeg.

. Canon no longer provides "SuperFine" jpeg compression, but CHDK has restored it on various affected cameras. This is the case for a little Ixus I recently bought, and the difference is a noticeable improvement. (It just about doubles the size of the jpeg file, and even the old "SuperFine" icon appears in the Canon review display so it seems the Canon firmware still recognizes it.)

Of course, CHDK is worth installing for many other reasons.

I agree, superfine & raw no limits alone is worth it, plus all those scripts!!

In case you missed it in my earlier post: the SX50 has superfine jpg as an option.

Don

 Don_Campbell's gear list:Don_Campbell's gear list
Canon EOS 7D Mark II Canon EOS R5 Sigma 150-600mm F5-6.3 | C Canon RF 100-500mm F4.5-7.1L IS USM
OP Coyote_Cody Senior Member • Posts: 1,166
Re: sx50 Canon Raw - How does it compare to CHDK raw files?

Don, not all cams do, my 300hs does not have SF, so my prior statement stands, was not referring to sx50, don't have one and with Canon raw would not need SF as I am not too lazy or busy to process raw, even from a 1/2.3" sensor cam!

Also, I do not take or process CHDK raw on my 300hs always, just when the shot warrants it or may warrant it, I suspect like most of us CHDK users w/o native raw.

I am jealous of native raw on P&S cams !!

Best regards!

-- hide signature --

The Light is Right - Take that Shot!!
Check out my photo galleries @
http://picasaweb.google.com/TauPhoto

OP Coyote_Cody Senior Member • Posts: 1,166
Re: sx50 Canon Raw - How does it compare to CHDK raw files?

Don_Campbell wrote:

Coyote_Cody wrote:

Don, thanks for the responses.

In the regard to "raw is raw', well obviously that may not be true if CHDK is 10 bit and Canon is 12 bit, Correct?

I said "raw is raw" with respect to noise in the raw image. Real raw files have no noise reduction applied. Most folks only view raw files after importing into an application that will apply some tweaking in making them visible. DPP opens the raw and automatically applies NR--at least in my hands on my machine in a VM of Windows.

I agree, I have LR4 & it seems all my raw files CHDK or Canon are noisier or at least different vs. my DSLR .cr2 raws in DPP for instance.

DPP does not process DNG's nor the other CHDK raw available. The simple reason I read is that the raw files on various P&S are all different sizes, 8mp vs 10mp vs 12mp vs this cam & that cam, and each new/diff P&S would have to have a customized bit readout (nibble/byte order, arrangement,etc - no generic solution) thus making a real '.cr2' file, DPP readable, too cumbersome lots of work. This is Canon's fault, no consistency, offsets for this & that change even for same cam but diff versions of f/w (f/w would not work for Canadian sx260 but worked for US version - diff versions- 100b vs 100c all because of adding french language supposedly)!

The issue of 10-bits vs 12-bits is a real issue if it's true and how do we find that out? I sure don't know. The 12-bits per channel statement I took as most authoritative is from a Canon PR. The 10-bits number is based on a statement in the CHDK forums by the CHDK authorities in 2008. Is it still that? Who knows?

The CHDK DNG is a dump of sensor data (found in the memory of the camera-part of porting issues) into a DNG formmated file structure (the other CHDK raw file is pure sensor data & generally not useful as you noted), nothing more or less?, thus the bit depth is whatever it is, provided by the sensor hardware/fw in camera, unless CHDK is processing the raw data with DNG software tools of some vintage age that can not do more than 10 bit in the file structure, throwing away the extra data, DNG certainly supports more.

"Pure sensor dump" is words without detail or the voice of authority. The only authoritative words I've found suggest that DNGs produced by CHDK are limited to 10 bits of color depth per color channel. Until we get something authoritative that is different about the SX50 it stands as a worry.

"Pure sensor dump" refers to the 'un formatted' raw you can get from CHDK (part of porting is finding the offset in memory of the sensor data, then determining how much data to read, how to read each 10 or 12 bit part, its order, etc, I believe/refer to this as the the non DNG raw data file offered by CHDK), it can be read by some software that knows how to interpret them, but I never saw it work correctly when trying several of the less popular/special s/w apps for this file type, thus I started using DNG raw only.

In case you don't know, If you get a 'chunk' of bits, it can be anything, so you must know what those bits are, 'their format'. You may need to read the bits in 8, or 10 or 12 bit chunks to get the byte or 10, 12 bits for the Red channel, the the green channel..... etc, or these 4 bits need to be the most significant bits of the 12 bit data for the Red channel, etc, you must flip them, so reading and formatting into DNG is not a 'blind' or auto process.

That is partly what CHDK sx50 porters are doing, and if they are getting successful raw photos from a sx50 then they have figured out how to read the sensor data correctly, but if they are chopping off 2 bits per channel because of DNG format limits, that will be VERY disappointing for all. I hope not.

The 'real' bit depth I believe is purely a function of the camera Hardware sensor D/A converter not s/w, although it may be f/w selectable (ie bit depth selectable depending on sensor hardware- have seen this) and if it is s/w then why not offer a bit depth of 16 or 24 or more for all cameras?

Whoa. It is the A/D converter that is at issue here. But it is also how that information gets mashed into a DNG format. How they do that may be by some convenient algorithm that they (CHDK developers) have no particular interest in changing for one camera or a few cameras.

You are absolutely correct, I meant A/D not D/A & true again about a special '12 bit' algorithm, but as mentioned above how do they know what bits to ignore, the sensor data is just a bunch of bits, so they would have to know which bits to ignore??

No it is a function of the D/A in camera, so why would CHDK reduce it somehow, I suspect the 10bit statement is from earlier years as P&S cams are/were 10bit depth, not 12, 14 or 16 bit. It lessens the file sizes & DSP processing the less the bit depth.

Sorry. You sidestep the idea that stuff in CHDK is in at least two parts: the algorithms and routines that work for all cameras; the special "glue" that links to those routines for a particular camera. I can easily see how they wouldn't create 12-bit per channel data handling software for one or a couple of cameras. I can also see that they might have worked out ways to let the camera do its own thing for raws and just kept CHDK's hands off. I don't know.

As above, the sensor data is just memory in ram, they must discover the offset to its beginning and its size and then must read it bit for bit, nibble by nibble and interpret it correctly to get all the GRBG at whatever bit depth correctly, no half way here, it must be correct of it is WRONG, so if they can 'interpret' it correctly, then stuffing it into DNG would be the relative easy part IF DNG is an open standard as implied, then they just indicate in the file it is 12 bits not 10, or is it just a stupid file header value, hope not.

What would be nice is if they left the 12-bit per channel paths in the software and let the camera produce its own raws and allowed it to do that for a wider range of camera shooting modes than is allowed in the stock camera. I have no idea if that's what will be done or has been done with other cameras.

The camera does produce it raws, Canon (as others said) does NOT apply their fancy/special algorithms to raw .cr2 output whereas they do to jpg outputs.

I suspect it is simply Canon has not broken up their 'special software stuff' away from their jpg processing algorithms. Likely it would need a major re-write of their 'Power shot' algorithms/code segments and they were not willing to do it. Maybe next generation?

Does the S95/100/110 get those special processing for their raws (jpeg+raws)?? If so then why not sx50, if not a re-write is in order!!

It might be DSP speed for jpg-raws (extra work for 2 file ouputs) but it would NOT be DSP speed for processsing ONLY raws with these specials.

As mentioned elsewhere, CHDK usually does not put the same limits on things that Canon does, but I have seen issues of Auto mode vs Program mode and ported things NOT working in both modes, we will see!

As to the 'lens distortion', IF you do not see lens distortion via Canon raw files, then SURELY Canon is processing the raw file either in-camera or RawT. is auto-magically before you see it, because it is there.

I don't "see" the raw file in raw format. The only way I "see" a raw file is as imported and preliminarily processed by a converter software package. I don't know what level of geometric processing has happened where in the chain of data movement.

LR4 shows the CHDK DNGs unaltered UNLESS it is one of their 'recognized' cam/lens. Since NO P&S Canon other than a handful (S95/S100...) are 'thought' to have or need raw processing, then it never recognizes them and thus it NEVER applies any processing at all, as it does not have data to do such.

I have added lens correction file for my 300hs in LR4 but I must initially manually enable it in the 'lens correction' section of LR4, whereas for my DSLRs it recognizes most of my lens and you can choose to use it or not, but it does pre-process the photo to a degree, it has pre-set values, a best guess, but can be canceled and overridden.

Once you add a lens correction file for a cam/lens, it will use it automatically but you can disable or manipulate if wanted, also you can 'create' a generic raw setting of all settings and then LR4 will 'auto' apply them, good and bad IMO.

Lens distortion in the sx50 should be seen in Canon raw (I certainly see it in DPP with my DSLR's & Canon lens) & certainly will be seen in CHDK DNG files, thus the questions of the 'barrel' being seen in widest angle shots.

I can't say that I remember if any 4.3mm wide angle images were among those I've looked at in DPP. I've used DPP almost not at all. I've used RT almost entirely as a converter of DNGs to standard color images. The Adobe DNG Converter program certainly produces corrections for lens distortions. You may be right that DPP would show that lens distortion for a photo taken at a 4.3 mm focal length.

Not DPP but LR4, DPP does not process DNGs, only .CRW & .CR2 raws and maybe TIFFs, as well as jpgs.

I think the sx50 has more than raws as a benefit over the sx40 but still not sure worth $160 or more, I will test/play with one next week.

You'll have to look deeply into the various strengths it has. I would never (and didn't) buy an SX40 to replace my SX20. I did buy an SX50 to replace it and did so only after reading user's reactions, reading reviews and seeing photos that the camera produced. I knew that the EVF was woeful (way worse than the SX20's) but that users were having a lot of fun and taking nice photos. When the camera went on sale in my town I bought it. I have been very happy with image quality for a small sensor camera. I accept the woeful EVF which I knew it had.

To me, the photos from sx50 look the same as sx40 which look the same as my 300hs, they all use the same basic (GOOD) sensor, the CMOS BSI 12.1mp (thank Canon for no 16mp sensor crap), my DIGIC 4 processor in 300hs is as fast as DIGIC 4/5 in sx40 or faster than DIGIC 5 in sx260hs and does many of the same special modes as fast.

So it is NOT really the jpg quality, they are the same to me (I tested sx40 even using CHDK last year), but the lens, the 2 buttons on front for framing, etc, the EVF is worthless to me (same as sx40) so I am thinking of the overall package is it worth the extra $$, ??

Thanks for all the input!!  I do look forward to playing with the sx50 next week!

The Light is Right - Take that Shot!!
Check out my photo galleries @
http://picasaweb.google.com/TauPhoto

reyalp Regular Member • Posts: 201
Re: sx50 Canon Raw - How does it compare to CHDK raw files?

A user on the CHDK forum pointed me to this thread. As one of the main CHDK developers for the last few years, I can clarify.

Don_Campbell wrote:

"Pure sensor dump" is words without detail or the voice of authority. The only authoritative words I've found suggest that DNGs produced by CHDK are limited to 10 bits of color depth per color channel. Until we get something authoritative that is different about the SX50 it stands as a worry.

CHDK does not change the bit depth of the data. It uses the same frame buffer that Canon raw and jpeg use. This may be 10, 12 or 14 bits depending on the camera. Pre digic IV cameras are mostly 10 bit, recent cameras are mostly 12 bit, and G1 X is the only known 14 bit. The fact the frame buffer has N bits doesn't necessarily mean there are actually N bits of precision, but I'm quite certain it's the same data whether the Canon firmware or CHDK writes it to disk.

But it is also how that information gets mashed into a DNG format.

DNG was designed as a raw format, so very little needs to be done to the actual data:

1) Interpolate over zero valued pixels, if DNG 1.1 format is selected. If DNG 1.3 is selected, we skip this and write opcodes in the DNG header instead, but these aren't widely supported outside of Adobe tools.

2) Reverse the byte order of the frame buffer, since DNG spec requires big-endian data when BitsPerSample is not 8, 16 or 32. Sigh.

3) Append the framebuffer to the file where we already wrote the header, thumbnail, exif etc.

TOMSDPR Contributing Member • Posts: 935
Re: sx50 Canon Raw - How does it compare to CHDK raw files?

Hi,

CHKD RAWs from the SX 260 sounds very attractive to me. How is your experience.

Is it possible to achieve less fuzzy/cleaner results from ISO 200 upwards with CHKD RAWs?

Which CHKD version have you been using?

Thks.

Tom

 TOMSDPR's gear list:TOMSDPR's gear list
Canon PowerShot S95 Olympus XZ-1 Panasonic Lumix DMC-GM1 Sony a7R III Fujifilm X-T100 +9 more
OP Coyote_Cody Senior Member • Posts: 1,166
Re: sx50 Canon Raw - How does it compare to CHDK raw files?
1

I can not answer as I was using/testing CHDK for sx260 during the porting phase, helped discover a couple issues but I ended up returning the camera deciding to get an even smaller camera.

I do use CHDK for the 300hs, but I almost never expose below ISO 100, so you should go over to the CHDK forum and ask there, I would think someone can answer you ques. fully.

Good Luck!!

-- hide signature --

The Light is Right - Take that Shot!!
Check out my photo galleries @
http://picasaweb.google.com/TauPhoto

Steen Bay Veteran Member • Posts: 7,418
Re: sx50 Canon Raw - How does it compare to CHDK raw files?

Don_Campbell wrote:

Coyote_Cody wrote:

As to the 'lens distortion', IF you do not see lens distortion via Canon raw files, then SURELY Canon is processing the raw file either in-camera or RawT. is auto-magically before you see it, because it is there.

I don't "see" the raw file in raw format. The only way I "see" a raw file is as imported and preliminarily processed by a converter software package. I don't know what level of geometric processing has happened where in the chain of data movement.

Lens distortion in the sx50 should be seen in Canon raw (I certainly see it in DPP with my DSLR's & Canon lens) & certainly will be seen in CHDK DNG files, thus the questions of the 'barrel' being seen in widest angle shots.

I can't say that I remember if any 4.3mm wide angle images were among those I've looked at in DPP. I've used DPP almost not at all. I've used RT almost entirely as a converter of DNGs to standard color images. The Adobe DNG Converter program certainly produces corrections for lens distortions. You may be right that DPP would show that lens distortion for a photo taken at a 4.3 mm focal length.

Don't think that the distortion is corrected in the SX50 raw files, but most converters including LR/PS and DPP will correct it automatically. Below an example showing how heavily distorted an uncorrected 24mm equivalent S100 image is. Guess than an uncorrected 24mm SX50 image will look pretty much the same.

http://forums.dpreview.com/forums/post/39634085

P.S - Here's another example, this time with SX50 images, just posted by AdamT :

http://forums.dpreview.com/forums/post/50626886

TOMSDPR Contributing Member • Posts: 935
Re: sx50 Canon Raw - How does it compare to CHDK raw files?

Coyote_Cody wrote:

I can not answer as I was using/testing CHDK for sx260 during the porting phase, helped discover a couple issues but I ended up returning the camera deciding to get an even smaller camera.

I do use CHDK for the 300hs, but I almost never expose below ISO 100, so you should go over to the CHDK forum and ask there, I would think someone can answer you ques. fully.

Good Luck!!

OK - back to square one. Never mind. THKS.

 TOMSDPR's gear list:TOMSDPR's gear list
Canon PowerShot S95 Olympus XZ-1 Panasonic Lumix DMC-GM1 Sony a7R III Fujifilm X-T100 +9 more
I2K4
I2K4 Senior Member • Posts: 1,465
DNG 1.1 versus 1.3

You guys are way over my head technically, but worth pointing out a recent experience - I'd been updating CHDK on my bridge camera without ever noticing the change to default DNG 1.3, which worked fine and wasn't noticeable at all.

When I picked up a little Ixus 300 (CHDK in beta) I found RAW/DNG produced a lot of colored flecks - it took some digging in the CHDK forum to learn that on some cameras DNG 1.3 doesn't do what DNG 1.1 does with badpixel.bin, and the only fix for that camera is to revert to DNG 1.1 and create badpixel.bin.

It's not a widely documented problem, drove me nuts for a couple of weeks (though I won't be shooting much RAW with that little critter) and I wonder how many other cameras are affected.

Don_Campbell Veteran Member • Posts: 3,108
Re: sx50 Canon Raw - How does it compare to CHDK raw files?

reyalp wrote:

A user on the CHDK forum pointed me to this thread. As one of the main CHDK developers for the last few years, I can clarify.

Don_Campbell wrote:

"Pure sensor dump" is words without detail or the voice of authority. The only authoritative words I've found suggest that DNGs produced by CHDK are limited to 10 bits of color depth per color channel. Until we get something authoritative that is different about the SX50 it stands as a worry.

CHDK does not change the bit depth of the data. It uses the same frame buffer that Canon raw and jpeg use. This may be 10, 12 or 14 bits depending on the camera. Pre digic IV cameras are mostly 10 bit, recent cameras are mostly 12 bit, and G1 X is the only known 14 bit. The fact the frame buffer has N bits doesn't necessarily mean there are actually N bits of precision, but I'm quite certain it's the same data whether the Canon firmware or CHDK writes it to disk.

But it is also how that information gets mashed into a DNG format.

DNG was designed as a raw format, so very little needs to be done to the actual data:

1) Interpolate over zero valued pixels, if DNG 1.1 format is selected. If DNG 1.3 is selected, we skip this and write opcodes in the DNG header instead, but these aren't widely supported outside of Adobe tools.

2) Reverse the byte order of the frame buffer, since DNG spec requires big-endian data when BitsPerSample is not 8, 16 or 32. Sigh.

3) Append the framebuffer to the file where we already wrote the header, thumbnail, exif etc.

Fascinating. SX50 CR2 files are around 14 MB each. The DNG files created by Adobe's DNGConverter are around 12 MB each. Clearly Adobe is mashing them in some way that makes them around 15% smaller than the CR2 files from which they are derived. Both make beautiful JPGs when converted.

As I said above, the DNGs made in camera by CHDK in my SX20 are about 18 MB. Curious.

Don

 Don_Campbell's gear list:Don_Campbell's gear list
Canon EOS 7D Mark II Canon EOS R5 Sigma 150-600mm F5-6.3 | C Canon RF 100-500mm F4.5-7.1L IS USM
Keyboard shortcuts:
FForum MMy threads