LX3 sensor analysis

[ejmartin]

One issue left hanging in the analysis was the issue of the overall normalization of the ISO of the LX3 RAW data. I was able to secure some side-by-side shots comparing the LX3 to two Canon DSLR's. Here there will be differences due to both the ISO normalization of the DSLR compared to, as well as the transmissivity of the lens put on the DSLR. For the 1D3 tested with 24-70/f2.8, the LX3 was 0.51 stop less sensitive, for the 1Ds3 tested with 24-105/f4, the LX3 was 0.86 stop less sensitive. So relative to the DSLR at ISO 100 (assuming the DSLR normalized properly; otherwise, consider these ratios of sensitivities), the LX3 would be at somewhere around ISO 55 to 70 in the RAW data. However the cameras (LX3 and DSLR) appear to be metering the same, so this means that the LX3 is underexposing the RAW by about .5-.86 stop and lifting it back up in post-processing.

I am puzzled as to why they would make such a choice of underexposure, leaving vastly more highlight headroom than the DSLR's do and consequently bringing more shadow noise into a typical exposure than is necessary.

BTW the results seem to agree qualitatively with the ISO calibration done by DxO at their new site DxOMark.com.
--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/

 

[christiak]

Interesting read. Thanks for you input on this.

Looks like the highlight tone priority mode on some dslrs. There's been many complaints by users of not only noise in p&s cameras, but also dynamic range. This appears to be Panny's approach to giving some highlight room and seems to work quite well for what the camera is.
Regards,
Kelly

One issue left hanging in the analysis was the issue of the overall
normalization of the ISO of the LX3 RAW data. I was able to secure
some side-by-side shots comparing the LX3 to two Canon DSLR's. Here
there will be differences due to both the ISO normalization of the
DSLR compared to, as well as the transmissivity of the lens put on
the DSLR. For the 1D3 tested with 24-70/f2.8, the LX3 was 0.51 stop
less sensitive, for the 1Ds3 tested with 24-105/f4, the LX3 was 0.86
stop less sensitive. So relative to the DSLR at ISO 100 (assuming the
DSLR normalized properly; otherwise, consider these ratios of
sensitivities), the LX3 would be at somewhere around ISO 55 to 70 in
the RAW data. However the cameras (LX3 and DSLR) appear to be
metering the same, so this means that the LX3 is underexposing the
RAW by about .5-.86 stop and lifting it back up in post-processing.

I am puzzled as to why they would make such a choice of
underexposure, leaving vastly more highlight headroom than the DSLR's
do and consequently bringing more shadow noise into a typical
exposure than is necessary.

BTW the results seem to agree qualitatively with the ISO calibration
done by DxO at their new site DxOMark.com.
--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/

 

[Leo]

...........

Black RAW level: 16 (but blacks are clipped)
Saturation RAW level: 4095
photosite efficiency: 2.20 e- raw level
Full well capacity (e- at raw saturation): 9000 e-
Read noise: 2.55 raw levels = 5.6 e-
Photosite dynamic range: 10.6 stops

..........

--
emil
--

Hi emil,

I have read the above paragraph and need your help again but now to understand the above numbers from your original post for this thread.

Black RAW level: 16 (but blacks are clipped)
Saturation RAW level: 4095

The level 4095 represents 12 EV (or stops), however the black level is already 16, setting the starting point to 4 EV (or stops). Would not the later start by 4 stops reduce the total dynamic range from 12 EV to 8 EV (stops) and not to 10.6 stops?
Thanks,
Leo

 

[Jeff Charles]

Very interesting post and your site is full of good information. I've learned a lot from it.

Until I read this post, I had convinced myself that the DXOMark.com ISO sensitivity results were incorrect for the LX3. They weren't consistent with the evidence from sample photos, and they were about one stop off from the results that DIWA Labs got with the same equipment. Also, the DIWA results correspond to the sample photo evidence. Apparently, though, Panasonic is doing something behind the scenes.

... so this means that the LX3 is underexposing the
RAW by about .5-.86 stop and lifting it back up in post-processing.
...

I do have a question about this. When you write "post-processing" do you mean processing done by the raw development software on the computer, or do you mean processing done to the raw file in the camera? The reason I ask is that I have found only about a 0.10 EV difference in the exposures that SilkyPix produces from the exposures that Raw Therapee produces. If exposure is being boosted in PP on the computer, I'd expect that SilkyPix, which is customized for the LX3, would boost it a lot more than RT.

Jeff

 

[ejmartin]

Oh how embarrassing, I got the sign wrong in the posted result -- it's the other way around. The 1D3 is .5 stop LESS sensitive, and the 1Ds3 .86 stop LESS sensitive than the LX3.

This is more the norm for digicams, and it should have been a red flag when I was writing the post. Digicam manufacturers do that, because ordinarily they have so little DR that it doesn't make sense to leave up to an extra stop of highlights unused if the highlights are not demanding; better to devote the DR to decrease shadow noise. But the LX3 has almost the DR of a DSLR, so I'm not sure that's necessary and one might do as well by setting a little negative EC to get some highlight latitude. The DSLR's leave about 3 stops between middle gray and saturation in the test shots, the LX3 only a bit over two.

One issue left hanging in the analysis was the issue of the overall
normalization of the ISO of the LX3 RAW data. I was able to secure
some side-by-side shots comparing the LX3 to two Canon DSLR's. Here
there will be differences due to both the ISO normalization of the
DSLR compared to, as well as the transmissivity of the lens put on
the DSLR. For the 1D3 tested with 24-70/f2.8, the LX3 was 0.51 stop
less sensitive, for the 1Ds3 tested with 24-105/f4, the LX3 was 0.86
stop less sensitive. So relative to the DSLR at ISO 100 (assuming the
DSLR normalized properly; otherwise, consider these ratios of
sensitivities), the LX3 would be at somewhere around ISO 55 to 70 in
the RAW data. However the cameras (LX3 and DSLR) appear to be
metering the same, so this means that the LX3 is underexposing the
RAW by about .5-.86 stop and lifting it back up in post-processing.

I am puzzled as to why they would make such a choice of
underexposure, leaving vastly more highlight headroom than the DSLR's
do and consequently bringing more shadow noise into a typical
exposure than is necessary.

BTW the results seem to agree qualitatively with the ISO calibration
done by DxO at their new site DxOMark.com.

With the error corrected, the results contradict the DxO finding. But I'm wondering how they could calibrate the test for the LX3 without taking the camera apart -- the DSLR sensitivity tests are done with direct exposure of the sensor with no lens mounted, whereas unless they disassemble the digicam the test would have to be done with the lens in place. Whatever the transmissivity of the lens, it would reduce the calibrated exposure of the sensor, and this might account for the relatively low value.

--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/

--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/

 

[ejmartin]

Interesting read. Thanks for you input on this.
Looks like the highlight tone priority mode on some dslrs. There's
been many complaints by users of not only noise in p&s cameras, but
also dynamic range. This appears to be Panny's approach to giving
some highlight room and seems to work quite well for what the camera
is.

It would look like that if I had not rushed the analysis and reversed the sign of the effect. I'm embarrassed to say I did do that, and the result is more in line with other digicams I've tested. See the correction to the sensitivity post just made.

--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/

 

[ejmartin]

Very interesting post and your site is full of good information. I've
learned a lot from it.

Until I read this post, I had convinced myself that the DXOMark.com
ISO sensitivity results were incorrect for the LX3. They weren't
consistent with the evidence from sample photos, and they were about
one stop off from the results that DIWA Labs got with the same
equipment. Also, the DIWA results correspond to the sample photo
evidence. Apparently, though, Panasonic is doing something behind the
scenes.

I'm thoroughly embarrassed to say that I royally screwed up the sign of the sensitivity difference... The result should have read that the LX3 is more sensitive than the DSLR's. This is more in line with other digicams I have tested, and contradicts the DxO result. It may have to do with the test methodology, which for DSLR's exposes the sensor directly without a lens mounted. They don't have that option for digicams, unless they are dismantling them to do the testing. The ISO calibration would then be affected by the transmissivity of the lens. I found that this could substantially affect the results when I was testing the 40D against the D300.

Again, profound apologies for the goof-up.

... so this means that the LX3 is underexposing the
RAW by about .5-.86 stop and lifting it back up in post-processing.
...

I do have a question about this. When you write "post-processing" do
you mean processing done by the raw development software on the
computer, or do you mean processing done to the raw file in the
camera? The reason I ask is that I have found only about a 0.10 EV
difference in the exposures that SilkyPix produces from the exposures
that Raw Therapee produces. If exposure is being boosted in PP on the
computer, I'd expect that SilkyPix, which is customized for the LX3,
would boost it a lot more than RT.

Any over- or under-exposure of the raw data can be compensated by applying exposure compensation during raw conversion (or by the jpeg engine). This can totally hide in the jpeg output where middle grey is set in the raw data relative to saturation. There is some freedom to set where middle grey is placed in the raw data relative to raw saturation, and as long as the raw converter knows where that point is, it can set the black, white, and gray points appropriately and the developed jpeg will look correct in terms of exposure; you'd never know that the raw was underexposed. Highlight Tone Priority on Canon DSLR's works this way, as does Active D-Lighting in Nikons, and whatever Sony etc call their variants of it.

A typical DSLR has about 3.5 stops between metered middle gray and RAW saturation, and a typical RAW converter sets the white point a bit lower than this to provide smooth rolloff to white. Digicams typically set the raw exposure a little higher, which is why they are more prone to blow out highlights; they do this because, having less DR overall, they have to carefully parse it out between highlights and shadows -- leave too much highlight headroom, and you don't use it while shadow noise gets brought up at the same time. This is less of a problem with the LX3, since its DR is close to that of DSLR's; so if you are blowing highlights, or want a more DSLR-like exposure, set -1/3 to -2/3 EC on the camera.

--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/

 

[Jeff Charles]

...
I'm thoroughly embarrassed to say that I royally screwed up the sign
of the sensitivity difference... The result should have read that
the LX3 is more sensitive than the DSLR's. This is more in line
with other digicams I have tested, and contradicts the DxO result.
...
Again, profound apologies for the goof-up.

No need to apologize. We all make mistakes. The only way to avoid making mistakes is to not do anything ;-)

Regarding the reason for the DXO results being off, someone in another thread suggested that they had forgotten to include ISO 80, and thereby all the values moved to the left. I compared the DXO results with the DIWA results, and they were in fact shifted by one ISO.

...This is less
of a problem with the LX3, since its DR is close to that of DSLR's;
so if you are blowing highlights, or want a more DSLR-like exposure,
set -1/3 to -2/3 EC on the camera.
...

That's good to know. DXO found pretty good DR for the LX3, but those numbers will get even better once they correct their ISO Sensitivity results.

Jeff

 

[bstring]

Looking forward to following along to see the outcome of the DxO ISO Sensitivity issue will be. Good to hear LX3 is on par with other compacts in this regard and not 'cheating'.

Thanks for sharing the wealth of knowledge.

Brian

--
Phanfare supporter-

 

[Ehrik]

Black RAW level: 16 (but blacks are clipped)
Saturation RAW level: 4095

The level 4095 represents 12 EV (or stops), however the black level
is already 16, setting the starting point to 4 EV (or stops). Would
not the later start by 4 stops reduce the total dynamic range from 12
EV to 8 EV (stops) and not to 10.6 stops?

16 to 4095 is just labelling; you could call it 0 to 4079 so it's not the ratio of those numbers we are interested in. It's the noise rather than the signal resolution that sets the limit. The DR definition Emil works with (the engineering definition) is the ratio between the saturation level and the read noise. (Where signal and noise are equally strong.)

(So you get 4095-16) / 2.55 = 1600
And 2^10.6 = 1600, so DR is 10.6 stops.

If you feel that the engineering definition has too noisy shadows, you can decide that a signal to noise ratio of, say, 4:1 is acceptable to you and then subtract 2
(2^2=4) from 10.6 to get a usable DR of 8.6 stops. See Emil's linked article.

Just my two oere
Erik from Sweden

 

[Tim Homola]

Very cool results emil. How did Pany make such a leap in sensor capability? From darn poor noise with the LX2, to damn good with the LX3?

 

[Ehrik]

Panasonic's digicam noise reputation was always to a big part a result of a JPEG engine (particularly Venus 3) that caused ugly smear without really getting the noise away, rather than the sensors being terribly substandard. (I don't know about the LX2, though, more than that it had pattern noise which could be a problem when deep shadows were brightened.)

E.g. the FZ50 which came at the same time as the LX2, has had its sensor tested thoroughly with the same methods as in this thread, and compares OK with the Sony sensor in the Canon G9.

Still, the LX3 seems to be a step up from that level.

Very cool results emil. How did Pany make such a leap in sensor
capability? From darn poor noise with the LX2, to damn good with the
LX3?

Just my two oere
Erik from Sweden

 

[xvimbi]

Good to see that this discussion is being continued over here...

Regarding the reason for the DXO results being off, someone in
another thread suggested that they had forgotten to include ISO 80,
and thereby all the values moved to the left. I compared the DXO
results with the DIWA results, and they were in fact shifted by one
ISO.

The argument against a shift in the data is that, after applying the shift, the 'Measured ISO sensitivity' vs. 'Manufacturer ISO' would no longer be linear, which is totally uncharacteristic.

More likely is that they forgot to either take the ISO 80 reading or neglected to present it.

Also, as pointed out elsewhere, DIWA and DxO used different methods for determining ISO sensitivities, so these numbers don't resolve the issue (i.e., at least not in a straightforward way; i.e., at least not straightforward for me

Best - MM

 

[Jeff Charles]

Regarding the reason for the DXO results being off, someone in
another thread suggested that they had forgotten to include ISO 80,
and thereby all the values moved to the left. I compared the DXO
results with the DIWA results, and they were in fact shifted by one
ISO.

The argument against a shift in the data is that, after applying the
shift, the 'Measured ISO sensitivity' vs. 'Manufacturer ISO' would no
longer be linear, which is totally uncharacteristic.

More likely is that they forgot to either take the ISO 80 reading or
neglected to present it.

Also, as pointed out elsewhere, DIWA and DxO used different methods
for determining ISO sensitivities, so these numbers don't resolve the
issue (i.e., at least not in a straightforward way; i.e., at least
not straightforward for me
...

The 3 pieces of evidence for the DXO ISO Sensitivity results for the LX3 being understated are

1. Photos taken with the LX3 and other cameras at the same ISO and exposure (f-stop +shutter speed) are roughly of equal brightness.
2. The DIWA results are almost exactly one stop offset from the DXOMark results.

3. Emil's tests, as I understand them, are consistent with 1 and 2 and contradict the DXOMark results.

Maybe no one of these is conclusive, but the all support the same conclusion.

The bottom line, regardless of the DXOMark results, is that in use in the field, the nominal ISOs of the LX3 are very close to the actual ISOs, and may in fact be somewhat understated.

Jeff

 

[xvimbi]

The 3 pieces of evidence for the DXO ISO Sensitivity results for the
LX3 being understated are
1. Photos taken with the LX3 and other cameras at the same ISO and
exposure (f-stop +shutter speed) are roughly of equal brightness.

This, to me, indicates exactly the opposite, namely that the LX3 sensor is indeed less sensitive. Because it has a larger photosite area as well as larger photosites, it should in fact need less light than, say, the G10 to produce an image of equal brightness.

None of what's been presented, on both sides of the argument, makes complete sense. DxO needs to come forward to shed some light on all this.

Best - MM

 

[Ehrik]

This, to me, indicates exactly the opposite, namely that the LX3
sensor is indeed less sensitive. Because it has a larger photosite
area as well as larger photosites, it should in fact need less light
than, say, the G10 to produce an image of equal brightness.

Generally, a bigger photosite can hold more total photons but will also
capture more photons at a specific exposure since it covers a bigger area
of the sensor. This tends to cancel out and both sensors should saturate
at the same exposure.

So white will be the same with both and depending on the tone curves
applied, other tones could also be the same.

Just my two oere
Erik from Sweden

 

[ejmartin]

You need to measure the raw data. A Canon DSLR with Highlight Tone Priority enabled at ISO 2oo, and with HTP disabled, generate equally bright images overall coming out of any raw converter that understands the HTP tag in the metadata (with the HTP image having a smoother rolloff of highlights in the output, since that's what HTP is for). But under the hood, HTP 2oo uses the sensor output amplification of ISO 100, so the image in the raw data looks one stop underexposed. The raw converter that understands that HTP was used, simply boosts the output by a stop (apart from rolling off the highlights) to get the image brightness back to what the metering indicated at the time of exposure for ISO 200.

DxO is testing sensor raw data, so their measurements are of the sensor performance, and not fooled by any of this sort of objuscation of the ISO standard that the manufacturers engage in. My measurements are similar in method, though I arrived at a different result.

But the point is that you can't go by converted raw output to determine the actual sensitivity of the sensor.
--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/

 

[xvimbi]

Generally, a bigger photosite can hold more total photons but will also
capture more photons at a specific exposure since it covers a bigger
area of the sensor. This tends to cancel out and both sensors should saturate
at the same exposure.

But isn't it generally so that a sensor with a larger total photosite area yields a brighter image at a given aperture/shutter speed combination than a sensor with smaller total photosite area (see dSLR vs. compacts)?

Best - MM

 

[xvimbi]

But the point is that you can't go by converted raw output to
determine the actual sensitivity of the sensor.

Thank you, thank you, thank you!

Best - MM

 

[ejmartin]

Generally, a bigger photosite can hold more total photons but will also
capture more photons at a specific exposure since it covers a bigger
area of the sensor. This tends to cancel out and both sensors should saturate
at the same exposure.

But isn't it generally so that a sensor with a larger total photosite
area yields a brighter image at a given aperture/shutter speed
combination than a sensor with smaller total photosite area (see dSLR
vs. compacts)?

A larger photosite area captures more total photons, because it has a larger total area. But brightness is measured by the flux of photons per unit area; the larger number of photons divided by the larger area of a large photosite will see the same flux as a smaller number of photons recorded, divided by a smaller photosite area.

What a larger sensor allows by capturing more total photons is a higher signal/noise ratio in the image data, which is a major reason for the higher image quality of cameras with large sensors.

--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/