LX3 sensor analysis

[ejmartin]

Some people have expressed an interest in the capability of the LX3 sensor, and thanks to a few people willing to take a series of RAW images to my specifications, I was able to infer some of its properties. If you have no interest in this sort of thing, please move quietly on to another thread.

I have to say, I am quite impressed with the LX3's sensor; I couldn't believe the efficiency number when I first got it. Here are the sensor properties at base ISO, ISO 80:

Black RAW level: 16 (but blacks are clipped)
Saturation RAW level: 4095
photosite efficiency: 2.20 e- raw level
Full well capacity (e- at raw saturation): 9000 e-
Read noise: 2.55 raw levels = 5.6 e-
Photosite dynamic range: 10.6 stops

The photosite efficiency is superior; my LX1 only gets about half that (the read noise is also substantially better than the LX1). I have compiled photosite efficiency results for a number of DSLR's at
http://theory.uchicago.edu/~ejm/pix/20d ... #pixelsize

see Table 1 a bit further down the page. Translating the LX3 photosite efficiency to ISO 400 by dividing by 5=400/80, and dividing also by the pixel area (2µ)^2, gives an efficiency figure of merit of .106 electrons per raw level per square micron. Naively this is better than the 1D3/1Ds3, and just a bit short of the D3, on a per area basis. However, the relative normalization of the ISO needs to be measured to correctly compare.

Here is the photosite S/N ratio (vertical axis, in stops) as a function of raw level (horizontal axis, in stops):



The LX3 is the blue curve, the G10 the red curve. The LX3 plot is not far from the 1D3 at ISO 800 (!); see
http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p2.html#SNR-DR

Please note also that these are pixel S/N ratios; image S/N involves the S/N at a fixed spatial scale relative to frame height, which involves scaling the pixel S/N ratio by the square root of the MP count. Translated into stops, the G10 curve should be raised by about .25, bringing it closer (but still short of) the LX3 curve.
--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/

 

[oluv]

very interesting, although i only understand half of it ;-)

but once i also have made a test with LX1 and LX3 and i think it is visible from this crop, that the LX1 sensor has considerably stronger noise, although the LX1 image appears sharper, probably due to a stronger AA-filter in the LX3, but the captured details seem nevertheless to be more or less equal.

(both files were converted in raw therapee with exactly the same settings and the same amount of sharpening.)

 

[0lf]

Very interesting.

It will be interisting to compare sensor efficiency with the Canon SX1 when it will be realesed to compare the efficiency of the latest generation of CCD and CMOS technologies.

 

[oluv]

unfortunately no raw on these. will have to wait until CHDK comes out with a hack for them.

 

[Ehrik]

Some people have expressed an interest in the capability of the LX3
sensor,

And I'm one of those so here's a big "Thank you!" for your effort.

I have to say, I am quite impressed with the LX3's sensor; I couldn't
believe the efficiency number when I first got it.

Nice to hear. I had a hunch that they had made some progress, good
to have it verified.

Here are the
sensor properties at base ISO, ISO 80:

Black RAW level: 16 (but blacks are clipped)
Saturation RAW level: 4095
photosite efficiency: 2.20 e- raw level
Full well capacity (e- at raw saturation): 9000 e-

That's quite a bit more than the 4800 e- at ISO 100 that
John Sheehy reported for the FZ50 here:
http://forums.dpreview.com/forums/read.asp?forum=1018&message=25681682

Read noise: 2.55 raw levels = 5.6 e-

The LX2 exhibited ugly horizontal streaks when the shadows were pushed.
Did you have any chance to check for pattern noise?

Photosite dynamic range: 10.6 stops

The photosite efficiency is superior; my LX1 only gets about half
that (the read noise is also substantially better than the LX1). I
have compiled photosite efficiency results for a number of DSLR's at
http://theory.uchicago.edu/~ejm/pix/20d ... #pixelsize

It seems the forum software broke the link.
Folks, try this:
http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html#pixelsize
Or this and scroll down:
http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html

see Table 1 a bit further down the page. Translating the LX3
photosite efficiency to ISO 400 by dividing by 5=400/80, and dividing
also by the pixel area (2µ)^2, gives an efficiency figure of merit of
.106 electrons per raw level per square micron. Naively this is
better than the 1D3/1Ds3, and just a bit short of the D3, on a per
area basis. However, the relative normalization of the ISO needs to
be measured to correctly compare.

Let us know if you need help with that. Viztyger has the LX3 and the D300
so we might persuade him to do a side by side. And there are many others
with various DSLRs and the LX3.

Here is the photosite S/N ratio (vertical axis, in stops) as a
function of raw level (horizontal axis, in stops):



The LX3 is the blue curve, the G10 the red curve. The LX3 plot is
not far from the 1D3 at ISO 800 (!); see
http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p2.html#SNR-DR
Please note also that these are pixel S/N ratios; image S/N involves
the S/N at a fixed spatial scale relative to frame height, which
involves scaling the pixel S/N ratio by the square root of the MP
count. Translated into stops, the G10 curve should be raised by
about .25, bringing it closer (but still short of) the LX3 curve.

So the LX3 seems to be the one to beat.

Just my two oere
Erik from Sweden

 

[larsbc]

First, thanks for posting this information.

ejmartin wrote:
[snip]

Photosite dynamic range: 10.6 stops

So, using this testing methodology, how do other cameras compare? I ask because I am rather surprised that it has a range of 10.6 stops, so I'd like to see that value in the context of other cameras' performance.

For instance, would that score be directly comparable to the dynamic range score that Dpreview sometimes posts in its reviews?

larsbc

 

[bstring]

quietly waiting to read more, understand bits and pieces.

--
Phanfare supporter-

 

[Ehrik]

ejmartin wrote:
[snip]

Photosite dynamic range: 10.6 stops

So, using this testing methodology, how do other cameras compare? I
ask because I am rather surprised that it has a range of 10.6 stops,
so I'd like to see that value in the context of other cameras'
performance.

The best current DSLRs have a pixel DR just shot of 11.7 which is the
limit when you start to need more than 12 bits to store all the information
(so yes the 14 bit modes are the marketing departments' victory over
science).

This all refers to an engineering definition where the dark end is where you
have a signal:noise ratio of 1:1. That's very noisy, so the range that will give
good photographic results would be a couple of stops less. The document
linked in the OP shows an example of how signal:noise ratio looks in practice.

For instance, would that score be directly comparable to the dynamic
range score that Dpreview sometimes posts in its reviews?

Not so much. Even the raw section there has a number of problems that
makes the results unreliable. The math and physics behind noise and DR
is not trivial so it's perhaps understandable that DPR are a bit lost.

Just my two oere
Erik from Sweden

 

[ciscodk1]

Oluv - how did you convert LX3's raw file?
I have downloaded Raw Therapee but it can't see the raw files from LX3(?)

allan

 

[bstring]

not to jump in line, but I read mine in raw therapee by changing the file extension to .raw.

--
Phanfare supporter-

 

[ejmartin]

The LX2 exhibited ugly horizontal streaks when the shadows were pushed.
Did you have any chance to check for pattern noise?

Yes I did have a look, it is extremely well controlled. Essentially none.

However, the relative normalization of the ISO needs to
be measured to correctly compare.

Let us know if you need help with that. Viztyger has the LX3 and the
D300
so we might persuade him to do a side by side. And there are many others
with various DSLRs and the LX3.

Let me see if I can get one of my contributors to test the same cameras I have for the rest of the tests.

--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/

 

[ejmartin]

Photosite dynamic range: 10.6 stops

So, using this testing methodology, how do other cameras compare? I
ask because I am rather surprised that it has a range of 10.6 stops,
so I'd like to see that value in the context of other cameras'
performance.

Have a look at some other DSLR's analyzed in the same manner at
http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p2.html#SNR-DR

For instance, would that score be directly comparable to the dynamic
range score that Dpreview sometimes posts in its reviews?

DPR's testing methodology leaves a lot to be desired. The jpeg DR reported depends strongly on the choice of tone curve the manufacturer uses in its jpeg engine. The RAW DR methodology is totally lame -- for instance, a raw DR of 12.6 stops is reported for the Sony A900, and yet in a linear recording medium like RAW, the 12 bit encoding of the A900 is mathematically incapable of recording more than 12 stops of DR.

The suggested way to read the S/N graph is as follows. The engineering definition of DR is essentially the range on the horizontal axis between raw saturation and S/N=1 (ie 0 stops). This is a rather liberal definition that sets the noise floor where noise equals signal. A more conservative standard is your choice -- decide what minimum S/N ratio you find acceptable, and count the number of stops from there to saturation to find your personally acceptable DR. You will find a shadow S/N test image at the link above to help you decide what minimum S/N you find acceptable.

--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/

 

[Ehrik]

For instance, would that score be directly comparable to the dynamic
range score that Dpreview sometimes posts in its reviews?

DPR's testing methodology leaves a lot to be desired. The jpeg DR
reported depends strongly on the choice of tone curve the
manufacturer uses in its jpeg engine.

And also on the noise filtering: It's particularly telling to see how their
test program goes crazy and reports extremely high DR for the high
ISOs of some cameras like the A-350.

The RAW DR methodology is
totally lame -- for instance, a raw DR of 12.6 stops is reported for
the Sony A900, and yet in a linear recording medium like RAW, the 12
bit encoding of the A900 is mathematically incapable of recording
more than 12 stops of DR.

With highlight recovery, the different colour channels are often shifted to
cover somewhat different ranges of light intensities, so the total "DR",
with compromised colour accuarcy, can cover more than a single
12 bit channel. But I see a fundamental flaw in trying to measure and
report this highlight recovery. I talked about this in the post linked below
and would appreciate your comment on that if you have time.

http://forums.dpreview.com/forums/read.asp?forum=1000&message=29616791

Just my two oere
Erik from Sweden

 

[Leo]

......

DPR's testing methodology leaves a lot to be desired. The jpeg DR
reported depends strongly on the choice of tone curve the
manufacturer uses in its jpeg engine. The RAW DR methodology is
totally lame -- for instance, a raw DR of 12.6 stops is reported for
the Sony A900, and yet in a linear recording medium like RAW, the 12
bit encoding of the A900 is mathematically incapable of recording
more than 12 stops of DR.

The suggested way to read the S/N graph is as follows. The
engineering definition of DR is essentially the range on the
horizontal axis between raw saturation and S/N=1 (ie 0 stops). This
is a rather liberal definition that sets the noise floor where noise
equals signal. A more conservative standard is your choice -- decide
what minimum S/N ratio you find acceptable, and count the number of
stops from there to saturation to find your personally acceptable DR.
You will find a shadow S/N test image at the link above to help you
decide what minimum S/N you find acceptable.

--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/

emil,

DPR performed testing for different camera based on the same procedure . In relative terms their tests still should be a good guideline for camera comparison. Most of the review readers are looking for comparative cameras data to add basis for their camera selection.
Leo

 

[ejmartin]

emil,
DPR performed testing for different camera based on the same
procedure . In relative terms their tests still should be a good
guideline for camera comparison. Most of the review readers are
looking for comparative cameras data to add basis for their camera
selection.
Leo

I agree that most visitors to sites such as DPR (especially those interested in digicams, and jpeg shooters) are interested in review sites that consistently compare cameras. I also agree that DPR uses a consistent procedure for their camera evaluation. What is missing is an understanding on the part of DPR of what their methodology is testing, and hence some context is missing from the jpeg analysis while the RAW DR testing is deeply flawed.

If no tone curve is applied to the RAW conversion other than gamma compensation, the theoretical limit of DR that can be encoded in an 8-bit jpeg with gamma=2.2 is a little over 10 stops. Any tone curve that adds contrast by deepening the shadows, or a black point set too high in the camera's jpeg engine (usually to mask shadow noise), reduces the DR that the jpeg is able to display; this is independent of whether the noise of the capture further limits the DR or not. For DSLR's, whose RAW DR is well over 10 stops at low ISO, the jpeg DR test that DPR performs reveals more about the tone curve and black point of the jpeg engine than it does about the camera's true DR. For digicams it's more of a moot point, since the DR is relatively low to begin with (perhaps around 6 to 8 stops) and so little affected by these issues.

The RAW DR test that DPR uses employs the highlight and shadow recovery capabilities of ACR to try to pull up the shadows and pull back the highlights and thereby recover "extra" DR. Consider for instance highlights: when a channel is blown in highlights, ACR attempts to infer its value via the values of the other two channels. Depending on the color temperature of the lighting used in the test, and the transmissivity of the color filters in the sensor, the range of EV of the different color channels can be a stop or more. So DPR's test is effectively trying to determine the DR from the range of EV between, say, the shadow sensitivity of the green channel to the highlight saturation level of the blue channel. It's just methodologically unsound.

--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/

 

[tartamillo]

a consistent procedure for their camera evaluation. What is missing
is an understanding on the part of DPR of what their methodology is
testing, and hence some context is missing from the jpeg analysis
while the RAW DR testing is deeply flawed.

I'm lucky when I understand 1/10 of what you write, but the geek in me appreciates nonetheless. However it seems to me that you are interested in evaluating bare sensor performance, while DPR is interested in more practical stuff: how's the whole camera going to behave in the real world?

To be more clear: if I understand correctly you say that the LX3 sensor has almost 1 stop of DR more than the G10. It's what old fashioned people like me call latitude, right? I like that, but it's still unclear to me if the LX3 as a camera is going to show that bigger latitude when faced with some high contrast scene or if it will be limited by something in the image processing path.
--
Maki

 

[ejmartin]

a consistent procedure for their camera evaluation. What is missing
is an understanding on the part of DPR of what their methodology is
testing, and hence some context is missing from the jpeg analysis
while the RAW DR testing is deeply flawed.

I'm lucky when I understand 1/10 of what you write, but the geek in
me appreciates nonetheless. However it seems to me that you are
interested in evaluating bare sensor performance, while DPR is
interested in more practical stuff: how's the whole camera going to
behave in the real world?

To be more clear: if I understand correctly you say that the LX3
sensor has almost 1 stop of DR more than the G10. It's what old
fashioned people like me call latitude, right? I like that, but it's
still unclear to me if the LX3 as a camera is going to show that
bigger latitude when faced with some high contrast scene or if it
will be limited by something in the image processing path.
--

As a practical matter, most users are interested in the bottom line -- what does the camera output? That is especially true of jpeg's, which are closer to the finished product than a RAW file. So DPR's jpeg "Dynamic Range" test (which they perform for DSLR's but not for digicams -- at least I couldn't find it in the LX3 review) is both quantitative and useful. Note however that it is really a measurement of the tone curve of the jpeg engine in the camera; for instance if you look at the review of the 50D

http://www.dpreview.com/reviews/canoneos50d/page19.asp

what is being plotted is the tone curve of the camera -- how it treats the shadow "toe" and how "soft" is the "shoulder" of highlight rolloff. One can learn a lot about the manufacturer's approach to image processing and the "look" of the finished product from this measurement, and so it is quite handy to have. Similarly, colorchecker tests of color accuracy are useful for discerning the manufacturer's approach to color reproduction.

When it comes to RAW imaging, one is separating the role of the camera as recording device, and taking the role of image processing out of its hands. There are then two useful sets of comparisons to be made -- what are the qualities and capabilities of the camera as a recording device, and what are the results that can be achieved from RAW conversion. There is utility to both; tests of the latter sort give the reader a chance to see the end product, setting a lower bound on image quality; while tests of the former sort give an unvarnished measure of the RAW output itself, unfiltered through the conversion process. This can be important, as for instance the 50D review initially used an inferior beta profile for ACR that gave substantially inferior results; more generally, different raw converters do a better job with some cameras than with others. Sony users have been wailing about the poor job ACR does with some of their DSLR's. I'm not convinced that there is a level playing field for camera evaluation based on the output of proprietary raw converters, since the processing the image has undergone is not open to scrutiny. If a test of converted RAW images shows camera A to fare poorly in comparison to camera B using raw converter X, what can we conclude? We cannot necessarily conclude that camera B is the better one, because it may simply be that raw converter X is not properly optimised for camera A.

So, will the LX3 as a camera show a bigger latitude when faced with some high contrast scene or will it be limited by something in the image processing path? Proper sensor testing will show whether the problem lies with the camera's hardware, or with the subsequent image processing. The point of RAW is that you can always change the latter if it doesn't do the job.

--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/

 

[Leo]

emil,
DPR performed testing for different camera based on the same
procedure . In relative terms their tests still should be a good
guideline for camera comparison. Most of the review readers are
looking for comparative cameras data to add basis for their camera
selection.
Leo

I agree that most visitors to sites such as DPR (especially those
interested in digicams, and jpeg shooters) are interested in review
sites that consistently compare cameras. I also agree that DPR uses
a consistent procedure for their camera evaluation. What is missing
is an understanding on the part of DPR of what their methodology is
testing, and hence some context is missing from the jpeg analysis
while the RAW DR testing is deeply flawed.

If no tone curve is applied to the RAW conversion other than gamma
compensation, the theoretical limit of DR that can be encoded in an
8-bit jpeg with gamma=2.2 is a little over 10 stops. Any tone curve
that adds contrast by deepening the shadows, or a black point set too
high in the camera's jpeg engine (usually to mask shadow noise),
reduces the DR that the jpeg is able to display; this is independent
of whether the noise of the capture further limits the DR or not.
For DSLR's, whose RAW DR is well over 10 stops at low ISO, the jpeg
DR test that DPR performs reveals more about the tone curve and black
point of the jpeg engine than it does about the camera's true DR.
For digicams it's more of a moot point, since the DR is relatively
low to begin with (perhaps around 6 to 8 stops) and so little
affected by these issues.

The RAW DR test that DPR uses employs the highlight and shadow
recovery capabilities of ACR to try to pull up the shadows and pull
back the highlights and thereby recover "extra" DR. Consider for
instance highlights: when a channel is blown in highlights, ACR
attempts to infer its value via the values of the other two channels.
Depending on the color temperature of the lighting used in the test,
and the transmissivity of the color filters in the sensor, the range
of EV of the different color channels can be a stop or more. So
DPR's test is effectively trying to determine the DR from the range
of EV between, say, the shadow sensitivity of the green channel to
the highlight saturation level of the blue channel. It's just
methodologically unsound.

--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/

emil,

It seems that you are well involved and better understand the image Dynamic Range subject. My understanding is based on its general definition I have learnt at school year ago, which sounded like: the dynamic range is a ratio of the Max to Min signals where the Min signal based on the noise level selected on the quality requirement at the pause (or for image it probably would be the black level). It may be simple Max Signal to Max Noise or RMS Signal to RMS Noise level (or any other levels).

I have cropped the same image part from the three test images posted by Imaging Resource for G10 and LX3 at ISO levels 400, 800 and 1600 . The Red (or red-purple) label text "New Benissimo Pepper Oil" of the LX3 test images is loosing its color with the ISO increase from ISO 400 to ISO 1600.

For letter "O" in word Oil at ISO 800:
G10: Background (R=176, G=162, B=136) The letter (R=133, G=107, B=94)
LX3 Background (R=183, G=159, B=136) The letter (R=106, G=79, B=58)

I have used Photoshop to measure RGB components. The Background numbers have been read at the center of "O" and the Letter numbers have been read from "O" itself,left side.

The background numbers are almost the same but the "O" character numbers show the loss of red component. At ISO 1600 LX3 displays less color indicating higher R component loss.

My monitor is calibrated. However, as it is a relative test calibration should not matter. Is not it an indication of the lower LX3 lower DR with the increased ISO or lower DR in general?

ISO 400



ISO 800 G10 vs LX3



ISO 1600 G10 vs LX3



Leo

 

[ejmartin]

My understanding is based on its general
definition I have learnt at school year ago, which sounded like: the
dynamic range is a ratio of the Max to Min signals where the Min
signal based on the noise level selected on the quality requirement
at the pause (or for image it probably would be the black level). It
may be simple Max Signal to Max Noise or RMS Signal to RMS Noise
level (or any other levels).

I think the use of RMS signal might be more adapted to audio, where one is naturally dealing with sinusoidal waveforms; in imaging, the signal tends to be more local constant than oscillating, so the upper end is taken as the saturation level of the raw data (4095 for 12-bit, 16383 for 14-bit data); for the noise level it is common to take the rms noise in the absence of a signal, though max noise would be fine too, so long as one is consistent. I use saturation level divided by rms black frame noise.

I have cropped the same image part from the three test images posted
by Imaging Resource for G10 and LX3 at ISO levels 400, 800 and 1600 .
The Red (or red-purple) label text "New Benissimo Pepper Oil" of the
LX3 test images is loosing its color with the ISO increase from ISO
400 to ISO 1600.

For letter "O" in word Oil at ISO 800:
G10: Background (R=176, G=162, B=136) The letter (R=133, G=107, B=94)
LX3 Background (R=183, G=159, B=136) The letter (R=106, G=79, B=58)

I have used Photoshop to measure RGB components. The Background
numbers have been read at the center of "O" and the Letter numbers
have been read from "O" itself,left side.

The background numbers are almost the same but the "O" character
numbers show the loss of red component. At ISO 1600 LX3 displays less
color indicating higher R component loss.

My monitor is calibrated. However, as it is a relative test
calibration should not matter. Is not it an indication of the lower
LX3 lower DR with the increased ISO or lower DR in general?

It looks to me that both cameras are doing a lot of noise reduction which is skewing all sorts of tonal values (look for instance at the color bleeding of reds all around the lettering you are focused on). The lettering is desaturated in the Panasonic image, which is a way of suppressing chroma noise. This is not an indication of lower dynamic range, since the tonal values you measured are in midtones. Lower DR manifests itself in noise that obliterates shadow detail, if the highlight tonalities are kept fixed.

--
emil
--



http://theory.uchicago.edu/~ejm/pix/20d/

 

[Leo]

emil,
Thank you!
Leo