Serguei Palto
Senior Member
- Messages
- 1,378
- Solutions
- 1
- Reaction score
- 1,441
I have decided to make this post after recent results published at “Photons to Photos” (PTP) on measurements of Dynamic Range (DR) of the latest Lumix G9 Mark II camera, which are in strong disagreement not only with my measurements, but also with real life observations of many G9II users. Despite PTP has recognized some evident errors in their first-published data, the corrected data, to my mind, remain not valid. I have also found that the problem is related not only to G9II, but also to many other cameras presented at PTP. As an example, here I am discussing on DR of Lumix G9 in comparison with Canon EOS 6D Mark II.
I own both G9, which is my main working horse, and Canon EOS 6DMII, which I also like very much. Thus, it is, of course, interesting to know how these two cameras compare in the dynamic range. The simplest way, which many use, is to go to the “Photons to Photos” site and just check the plot as shown in #1.
#1. DR comparison for Lumix G9 and Canon 6D M II taken from PTP site
As we can see, according to the PTP-plot, G9 has about 1 stop DR-advantage at the base ISO 200, and more than 1 stop at the extended ISO 100. It is also interesting question why the G9-DR at ISO 100 is higher than at ISO200, while ISO100 is specified by Panasonic as the extended ISO?
But what is surprised, that above ISO 800 the G9 loses its advantage – the G9-DR drops significantly faster than 6DMII-DR, which is a real puzzle for those who familiar with radio-engineering and electronics. As a result, at ISO 6400 the G9-DR is about 0.5 stop less than 6DMII. And this is not the last surprise. At ISO 12600 the G9-DR suddenly jumps up and becomes even higher than 6DMII-DR.
Is that what tells PTP-plot TRUE?
During last few years I work on my hobby-project related to Raw-processing software called iWE (image waves editor), which also has tools for analyzing the statistics of the Raw-data and allow the Dynamic Range measurements. Namely iWE-tools were used to study the DR-performance of these two cameras.
Let us check this with iWE Raw-processing software.
What is Dynamic Range?
Mathematically, it is the ratio of the highest value (V_max), which can be registered by a sensor photodiode (pixel), to the minimal value defined by noise level (V_noise) in electronic circuit with the pixel-photodiode connected to this circuit, when the light is blocked. With this definition (well-known in Physics) the DR tells us on ability of the photodiode to register minimal values of the light energy without the narrowing the spectral bandwidth of the registration. In terms of the sensor performance, it is the ability of the sensor to register the image details in deep shadows or in case of the image is strongly underexposed. The high DR also provides flexibility in saving the room for highlights by shifting the exposure to lower values. It is convenient to express the DR in Exposure Values (EV) as logarithm to base 2 (log (V_max/V_noise)).
How to measure DR?
The simplest way is to analyze the black-frame data taken at rather fast shutter speed with the cap on the lens to minimize the penetration of the light. These data, of course, should be properly preprocessed to be used just in the form of linear data registered by sensor photodiodes with the subtracted black level and well-defined maximum level for the light energy registration. The RAW-converter must be set to regime, when no white balance is applied; the demosaicing should be turned off, or simplest (bilinear) interpolation algorithm should be used. In other words, the data must be proportional to the registered light energy with zero value at zero light energy, so at zero light energy we just have fluctuating read noise data.
The effective noise level is defined as the “standard deviation” calculated by simple well-known math procedure. In case of the non-demosaicing data we deal with the half-size image with two physical green pixels forming single logical green pixel. For the last reason in this case we have enhanced DR (about +0.5 EV) for the green image channel compared for the virtual case when only one green physical pixel is used for the logical green pixel of the image. If one needs to have better correlation with real-life whole-size images, the DR measurements must be done for the demosaiced data. I do recommend using the simplest bilinear interpolation algorithm which introduces minimal amount of the noise into the green channel compared to the half-size image. The positive difference between the DR for the green channel of the non-demosaicing half-size image and bilinear-interpolated full-size image is only about +0.2 EV. Because the bilinear interpolation is based on the averaging of the neighboring pixels of the same color, the resulting DR for R and B channels is about 0.1 EV higher than that for half-size non-interpolated image. In this post I show the results for the bilinear interpolated full-size image. The DR results are also discussed in connection to real-life full-size images.
G9 vs 6DMII according to iWE
The picture #2 shows the dynamic range measured with help of iWE for both G9 and 6DMII. The data inaccuracy is about 0.02EV, which is defined by statistics over about 64000 pixels. At ISO 100 the G9-DR is about 0.1 EV higher than at ISO 200, which points out that ISO 100 is not purely extended one. Comparison of the linear image data from RAWs for similar images taken at ISO 100 and ISO200 at the same exposure (same shutter speed and lens F-number) shows that the analog gain at ISO 200 is higher than at ISO 100. Thus, it is a real question why in the G9 manual the ISO 100 is pointed as the extended?
As one can see the G9-DR is indeed significantly higher than the 6DMII-DR at the base ISO 200 (according to the manual), which is in relative agreement with the PTP measurements. The difference is slightly higher than 1 stop for all the RGB channels. For the green channel the G9-DR is approaching the value very close to 13 stops, which for 12-bit ADC means the perfect performance of the G9 electronics (instead of scientific metrics the PTP uses their own so-called “Photographic” DR, which is also not a good idea, because Photonic DR does not allow judging on how close is DR to the theoretical maximum which is even higher than the bit-depth of the Analog-to-Digital converter if the quantum fluctuations of the lowest bit are taken into account).
However, the most important fact is that G9-DR is higher than 6DMII-DR for all the ISO values, which is in strong disagreement with PTP results shown in #1.
Moreover, at ISO 25600 the G9-DR is the same as at ISO 12800. What it means? It means that ISO 25600 is the extended ISO – no additional analog amplification is applied at ISO 25600 compared to ISO12800. The data in #3 and #4 taken at ISO 12800 and 25600 at the same exposure (same shutter speed and same F-number) illustrate this very well. At the same shutter speed and lens F-number the linear data registered by the sensor are the same for both ISO12800 and ISO25600 (for example, at bottom right side in iWE “Statistics panel” one can read the mean value for the green channel to be equal 2348 and 2353 for ISO12800 and ISO25600, respectively). Thus, the ISO25600 is indeed the extended ISO (actually, it is the same ISO12800, which is reflected by same DR-value on the plot #2), and there is no DR-jump (increase) above ISO 12800 as reported in #1 by “Photons to Photos” data. The second real question to Panasonic - why in the G9 manual the ISO 25600 is not pointed as the extended?
#2. The scientific Dynamic range data for Lumix G9 and Canon 6D M II measured with help of iWE software. Top plot is for Green (G) channel; the bottom plot is for Red(R) and Blue (B) channels
#3. ISO 12800, SS 1/100, F6.3. The statistics data along the yellow-line marker are shown at the bottom right-side panel (Green channel: the Mean value 2348). To discussion on the extended G9 ISO above the ISO12800
#4. ISO 25600, SS 1/100, F6.3. The statistics data along the yellow-line marker are shown at the bottom right-side panel (Green channel: the mean value 2353 – with accounting for noise the same as at ISO 12800; there is no additional analog amplification compared to ISO 12800)
Can we visualize the better DR performance of the G9 compared to the 6DMII?
Yes, but we must provide the conditions for the same exposure of pixels and, as a result, the same photon noise in the both cases (individual pixels of the 6DMII and G9 must receive the same number of photons). To do this, we have to take into account that the physical area of the 6DMII-photodiode (pixel) is 3 times larger than the area of the G9-pixel (not 4 times, because of aspect ratio and pixels count difference; FF 6DMII sensor has 26 Mpx, while m43 G9 has 20 Mpx). The same pixel exposure is achieved at the same shutter speed, but the lens F-number in case of G9 must be about 1.73 times lower than for 6DMII (for example, if the 6DMII-lens has F-number 8 then G9-lens should be set to F-number 4.5).
The photo #5 is taken with Canon 6DMII at ISO 1600, and it is 5 stops underexposed (shutter speed 1/500 s; F-number 8), so the 5 stops were compensated in the post-processing.
#5. Canon 6D M II at ISO 1600; the photo is underexposed -5 stops. +5 stops compensated in iWE
#6. G9 photo taken at ISO 1600 and the same shutter speed as photo #5. The lens F-number is 4.5 instead of 8 at photo #5 to provide the same number of photons per pixel as in case of photo #5
#7. 5DMII (crop from #5)
#8. G9 (crop from #6). The in-shadows wall-paper ornament is significantly better pronounced than in #7
The photo #6 is taken with G9 at the same pixel exposure at ISO 1600 (the same shutter speed 1/500 s, but at F-number 4.5 to get the same quantity of photons per pixel as in case of the Canon’s pixel). In the post processing the same 5 stops were compensated.
If we compare the shadows in photo #6 and photo #5 (see also crops #7 and #8) then it is evident that G9 recovers the information in shadows much better than 6DMII, which is in agreement with the Dynamic Range iWE-measurements (about 0.5 stops advantage of the G9). It is also in strong disagreement with PTP-data pointing the DR-advantage of the 6DMII at ISO above 800. Similar analysis shows that the G9 DR-advantage remains at all ISO values in accordance with the data shown in #2.
Conclusion
According to iWE the G9 has demonstrated excellent DR performance, which at base ISO is more than one stop higher than DR of Canon 6D Mark II. G9-DR for the green channel approaching the value 13 EV, which on account of 12-bit depth of the ADC is already limited by quantum fluctuation of lowest ADC-bit. It is interesting that 25600 ISO of the G9 is found to be extended, while ISO 100 is characterized by lower analog gain than ISO 200 which explains the growth of the G9 DR below ISO 200.
The G9-DR is higher than the DR of the Canon 6DMII for all the ISO values, which was also confirmed by real-life photos. The last fact does not agree with PTP DR measurements. iWE-measurements show no specific jumps in DR at ISO above 12800 as it was reported by PTP.
The PTP data can not be considered as reliable. The authors of “Photon to Photos” site must seriously revise their methodology, because such errors influence not only choice of camera consumers but also result in underestimation the work of highly qualified specialists which make all the best to produce high-performance camera for us.
I own both G9, which is my main working horse, and Canon EOS 6DMII, which I also like very much. Thus, it is, of course, interesting to know how these two cameras compare in the dynamic range. The simplest way, which many use, is to go to the “Photons to Photos” site and just check the plot as shown in #1.
#1. DR comparison for Lumix G9 and Canon 6D M II taken from PTP site
As we can see, according to the PTP-plot, G9 has about 1 stop DR-advantage at the base ISO 200, and more than 1 stop at the extended ISO 100. It is also interesting question why the G9-DR at ISO 100 is higher than at ISO200, while ISO100 is specified by Panasonic as the extended ISO?
But what is surprised, that above ISO 800 the G9 loses its advantage – the G9-DR drops significantly faster than 6DMII-DR, which is a real puzzle for those who familiar with radio-engineering and electronics. As a result, at ISO 6400 the G9-DR is about 0.5 stop less than 6DMII. And this is not the last surprise. At ISO 12600 the G9-DR suddenly jumps up and becomes even higher than 6DMII-DR.
Is that what tells PTP-plot TRUE?
During last few years I work on my hobby-project related to Raw-processing software called iWE (image waves editor), which also has tools for analyzing the statistics of the Raw-data and allow the Dynamic Range measurements. Namely iWE-tools were used to study the DR-performance of these two cameras.
Let us check this with iWE Raw-processing software.
What is Dynamic Range?
Mathematically, it is the ratio of the highest value (V_max), which can be registered by a sensor photodiode (pixel), to the minimal value defined by noise level (V_noise) in electronic circuit with the pixel-photodiode connected to this circuit, when the light is blocked. With this definition (well-known in Physics) the DR tells us on ability of the photodiode to register minimal values of the light energy without the narrowing the spectral bandwidth of the registration. In terms of the sensor performance, it is the ability of the sensor to register the image details in deep shadows or in case of the image is strongly underexposed. The high DR also provides flexibility in saving the room for highlights by shifting the exposure to lower values. It is convenient to express the DR in Exposure Values (EV) as logarithm to base 2 (log (V_max/V_noise)).
How to measure DR?
The simplest way is to analyze the black-frame data taken at rather fast shutter speed with the cap on the lens to minimize the penetration of the light. These data, of course, should be properly preprocessed to be used just in the form of linear data registered by sensor photodiodes with the subtracted black level and well-defined maximum level for the light energy registration. The RAW-converter must be set to regime, when no white balance is applied; the demosaicing should be turned off, or simplest (bilinear) interpolation algorithm should be used. In other words, the data must be proportional to the registered light energy with zero value at zero light energy, so at zero light energy we just have fluctuating read noise data.
The effective noise level is defined as the “standard deviation” calculated by simple well-known math procedure. In case of the non-demosaicing data we deal with the half-size image with two physical green pixels forming single logical green pixel. For the last reason in this case we have enhanced DR (about +0.5 EV) for the green image channel compared for the virtual case when only one green physical pixel is used for the logical green pixel of the image. If one needs to have better correlation with real-life whole-size images, the DR measurements must be done for the demosaiced data. I do recommend using the simplest bilinear interpolation algorithm which introduces minimal amount of the noise into the green channel compared to the half-size image. The positive difference between the DR for the green channel of the non-demosaicing half-size image and bilinear-interpolated full-size image is only about +0.2 EV. Because the bilinear interpolation is based on the averaging of the neighboring pixels of the same color, the resulting DR for R and B channels is about 0.1 EV higher than that for half-size non-interpolated image. In this post I show the results for the bilinear interpolated full-size image. The DR results are also discussed in connection to real-life full-size images.
G9 vs 6DMII according to iWE
The picture #2 shows the dynamic range measured with help of iWE for both G9 and 6DMII. The data inaccuracy is about 0.02EV, which is defined by statistics over about 64000 pixels. At ISO 100 the G9-DR is about 0.1 EV higher than at ISO 200, which points out that ISO 100 is not purely extended one. Comparison of the linear image data from RAWs for similar images taken at ISO 100 and ISO200 at the same exposure (same shutter speed and lens F-number) shows that the analog gain at ISO 200 is higher than at ISO 100. Thus, it is a real question why in the G9 manual the ISO 100 is pointed as the extended?
As one can see the G9-DR is indeed significantly higher than the 6DMII-DR at the base ISO 200 (according to the manual), which is in relative agreement with the PTP measurements. The difference is slightly higher than 1 stop for all the RGB channels. For the green channel the G9-DR is approaching the value very close to 13 stops, which for 12-bit ADC means the perfect performance of the G9 electronics (instead of scientific metrics the PTP uses their own so-called “Photographic” DR, which is also not a good idea, because Photonic DR does not allow judging on how close is DR to the theoretical maximum which is even higher than the bit-depth of the Analog-to-Digital converter if the quantum fluctuations of the lowest bit are taken into account).
However, the most important fact is that G9-DR is higher than 6DMII-DR for all the ISO values, which is in strong disagreement with PTP results shown in #1.
Moreover, at ISO 25600 the G9-DR is the same as at ISO 12800. What it means? It means that ISO 25600 is the extended ISO – no additional analog amplification is applied at ISO 25600 compared to ISO12800. The data in #3 and #4 taken at ISO 12800 and 25600 at the same exposure (same shutter speed and same F-number) illustrate this very well. At the same shutter speed and lens F-number the linear data registered by the sensor are the same for both ISO12800 and ISO25600 (for example, at bottom right side in iWE “Statistics panel” one can read the mean value for the green channel to be equal 2348 and 2353 for ISO12800 and ISO25600, respectively). Thus, the ISO25600 is indeed the extended ISO (actually, it is the same ISO12800, which is reflected by same DR-value on the plot #2), and there is no DR-jump (increase) above ISO 12800 as reported in #1 by “Photons to Photos” data. The second real question to Panasonic - why in the G9 manual the ISO 25600 is not pointed as the extended?
#2. The scientific Dynamic range data for Lumix G9 and Canon 6D M II measured with help of iWE software. Top plot is for Green (G) channel; the bottom plot is for Red(R) and Blue (B) channels
#3. ISO 12800, SS 1/100, F6.3. The statistics data along the yellow-line marker are shown at the bottom right-side panel (Green channel: the Mean value 2348). To discussion on the extended G9 ISO above the ISO12800
#4. ISO 25600, SS 1/100, F6.3. The statistics data along the yellow-line marker are shown at the bottom right-side panel (Green channel: the mean value 2353 – with accounting for noise the same as at ISO 12800; there is no additional analog amplification compared to ISO 12800)
Can we visualize the better DR performance of the G9 compared to the 6DMII?
Yes, but we must provide the conditions for the same exposure of pixels and, as a result, the same photon noise in the both cases (individual pixels of the 6DMII and G9 must receive the same number of photons). To do this, we have to take into account that the physical area of the 6DMII-photodiode (pixel) is 3 times larger than the area of the G9-pixel (not 4 times, because of aspect ratio and pixels count difference; FF 6DMII sensor has 26 Mpx, while m43 G9 has 20 Mpx). The same pixel exposure is achieved at the same shutter speed, but the lens F-number in case of G9 must be about 1.73 times lower than for 6DMII (for example, if the 6DMII-lens has F-number 8 then G9-lens should be set to F-number 4.5).
The photo #5 is taken with Canon 6DMII at ISO 1600, and it is 5 stops underexposed (shutter speed 1/500 s; F-number 8), so the 5 stops were compensated in the post-processing.
#5. Canon 6D M II at ISO 1600; the photo is underexposed -5 stops. +5 stops compensated in iWE
#6. G9 photo taken at ISO 1600 and the same shutter speed as photo #5. The lens F-number is 4.5 instead of 8 at photo #5 to provide the same number of photons per pixel as in case of photo #5
#7. 5DMII (crop from #5)
#8. G9 (crop from #6). The in-shadows wall-paper ornament is significantly better pronounced than in #7
The photo #6 is taken with G9 at the same pixel exposure at ISO 1600 (the same shutter speed 1/500 s, but at F-number 4.5 to get the same quantity of photons per pixel as in case of the Canon’s pixel). In the post processing the same 5 stops were compensated.
If we compare the shadows in photo #6 and photo #5 (see also crops #7 and #8) then it is evident that G9 recovers the information in shadows much better than 6DMII, which is in agreement with the Dynamic Range iWE-measurements (about 0.5 stops advantage of the G9). It is also in strong disagreement with PTP-data pointing the DR-advantage of the 6DMII at ISO above 800. Similar analysis shows that the G9 DR-advantage remains at all ISO values in accordance with the data shown in #2.
Conclusion
According to iWE the G9 has demonstrated excellent DR performance, which at base ISO is more than one stop higher than DR of Canon 6D Mark II. G9-DR for the green channel approaching the value 13 EV, which on account of 12-bit depth of the ADC is already limited by quantum fluctuation of lowest ADC-bit. It is interesting that 25600 ISO of the G9 is found to be extended, while ISO 100 is characterized by lower analog gain than ISO 200 which explains the growth of the G9 DR below ISO 200.
The G9-DR is higher than the DR of the Canon 6DMII for all the ISO values, which was also confirmed by real-life photos. The last fact does not agree with PTP DR measurements. iWE-measurements show no specific jumps in DR at ISO above 12800 as it was reported by PTP.
The PTP data can not be considered as reliable. The authors of “Photon to Photos” site must seriously revise their methodology, because such errors influence not only choice of camera consumers but also result in underestimation the work of highly qualified specialists which make all the best to produce high-performance camera for us.
Last edited:
