K20 Dynamic Range comparison

@vortout

This would make little sense to me too. The tests on this site seem fantastic, but it's all meaningless without knowing whether RAW or JPEG was used for the tests.

I've searched their site for information on this, but come up with nothing. Strange that they would omit this, given how comprehensive these tests seem to be.

Can you say what makes you think the tests were done on JPEGs?

I am also a little puzzled by their tonal range results. In the noise section of the Test Results section, they claim that 8-bits is the theoretical maximum. At the bottom of this page they test D40x RAW output, and it's seems true that 8-bits is the ceiling. But the D40x RAW file can represent sensor data in 12-bits / channel. Surely 12-bits is the theoretical limit for the tonal range of a channel in RAW.

--
A rose by any other name is still a chicken.

Joe Franek said:

GordonBGood said:

I would expect that a DR test on raw data for the K20D using Imatest
will show that the K20D has about a stop less DR than the top cameras
right across the range and somewhat less DR than the K10D at low ISO
sensitivities.

Click to expand...

But that would mean that raw DR is worse than jpeg DR, wouldn't it?
Do you think this would be unique to K20?

Click to expand...

No, Joe, I meant if they were measured in the same way raw K20DR would be less than the others. JPEG Dynamic Range is almost always less than that of raw data due to a higher contrast tone response curve.

Regards, GordonBGood

based on Canon 450d test which has the same DR at all ISO i would say they use jpeg

seachicken2000 said:

@vortout
This would make little sense to me too. The tests on this site seem
fantastic, but it's all meaningless without knowing whether RAW or
JPEG was used for the tests.

I've searched their site for information on this, but come up with
nothing. Strange that they would omit this, given how comprehensive
these tests seem to be.

Can you say what makes you think the tests were done on JPEGs?

I am also a little puzzled by their tonal range results. In the noise
section of the Test Results section, they claim that 8-bits is the
theoretical maximum. At the bottom of this page they test D40x RAW
output, and it's seems true that 8-bits is the ceiling. But the D40x
RAW file can represent sensor data in 12-bits / channel. Surely
12-bits is the theoretical limit for the tonal range of a channel in
RAW.

--
A rose by any other name is still a chicken.

Click to expand...

--
http://www.photomiks.com
Regards, Dmitry

Gordon and anyone who is interested
You can download files from there:
http://idisk.mac.com/dsemeonov-Public/k10vsk20iso1600/?view=web
K10 and k20, both iso 1600, 1 sec
I used silkypix
as i wrote before LR and dcraw are not relevant to k20d yet

i downsized k20 to k10. i think it is fair when we compare noise and details in shadows. I pushed both shots up 2 stops

So. k20d outperforms k10d without any doubts. i cannot judge if there is 1 stop. i need to do more tests but k20d delivers better S/N ratio for sure

What is advantage of having both cameras you always can evaluate on better/worse base

GordonBGood said:

Joe Franek said:

GordonBGood said:

I would expect that a DR test on raw data for the K20D using Imatest
will show that the K20D has about a stop less DR than the top cameras
right across the range and somewhat less DR than the K10D at low ISO
sensitivities.

Click to expand...

But that would mean that raw DR is worse than jpeg DR, wouldn't it?
Do you think this would be unique to K20?

Click to expand...

No, Joe, I meant if they were measured in the same way raw K20DR
would be less than the others. JPEG Dynamic Range is almost always
less than that of raw data due to a higher contrast tone response
curve.

Regards, GordonBGood

Click to expand...

--
http://www.photomiks.com
Regards, Dmitry

Dmitry, a few observations on your tests, as interspersed:

vortout said:

Gordon and anyone who is interested
You can download files from there:
http://idisk.mac.com/dsemeonov-Public/k10vsk20iso1600/?view=web
K10 and k20, both iso 1600, 1 sec
I used silkypix
as i wrote before LR and dcraw are not relevant to k20d yet
i downsized k20 to k10. i think it is fair when we compare noise and
details in shadows. I pushed both shots up 2 stops

Click to expand...

Yes, downsizing is fair.

vortout said:

So. k20d outperforms k10d without any doubts. i cannot judge if there
is 1 stop. i need to do more tests but k20d delivers better S/N ratio
for sure
What is advantage of having both cameras you always can evaluate on
better/worse base

Click to expand...

However, there are some processing differences between the two files even though you used Silkypix for both. Even though the grey exposure levels of these two images are almost identical, the colours in the K20D image are much more muted than for the K10D, which somewhat reduces the colour noise of the K20D image (see the green table edge at across the bottom of the images). Also, you are starting to see the advantage of the noise reduction applied to the K20D image in the raw file which kicks in at ISO 1600 as determined by Oleg_V. Even so, although the noise is slightly lower for this K20D image as compared to the K10D image they are similar enough so that the measured DR won't be much different.

Where you would really see the differences between the two cameras is if you did the same test but underexposed by even more (6 stops instead of 2 stops for ISO 100, and 5 stops for ISO 200), and then boosted them up to ISO 6400 equivalent as you did here. Then you would see the higher floor noise of the K20D at low ISO's as compared to the K10D, although you will likely also see Vertical Pattern Noise (VPN) for the K10D, although these particular scenes don't really have enough dark grey flat textured surfaces to be able to see it.

Don't get me wrong in thinking that I'm trying to "knock" the K20D, as it is likely the camera I would buy if I didn't buy the more "bang for the Baht" K200D and have more to spend on some good lenses. It just has this one weakness that somewhat limits the usable full resolution DR at low ISO's and also slightly limits the high ISO performance against some of its other brand competitors; however this limitation is actually easier to work around than the VPN problem of the K10D, and many users who don't push the performance envelope of either will never have a problem with any of these limitations.

Regards, GordonBGood

seachicken2000 said:

@vortout
This would make little sense to me too. The tests on this site seem
fantastic, but it's all meaningless without knowing whether RAW or
JPEG was used for the tests.

I've searched their site for information on this, but come up with
nothing. Strange that they would omit this, given how comprehensive
these tests seem to be.

Can you say what makes you think the tests were done on JPEGs?

Click to expand...

DXO tests are good, and in fact can be applied to raw data, but that doesn't appear to be what this review site has done. I downloaded all available associated files for these tests and they all refer to 8-bit JPEG RGB data, just as is one of the optios for the DXO Analyser 3.1 example at:

http://info.dxo.com/demokit_analyzer/page1.html

seachicken2000 said:

I am also a little puzzled by their tonal range results. In the noise
section of the Test Results section, they claim that 8-bits is the
theoretical maximum. At the bottom of this page they test D40x RAW
output, and it's seems true that 8-bits is the ceiling. But the D40x
RAW file can represent sensor data in 12-bits / channel. Surely
12-bits is the theoretical limit for the tonal range of a channel in
RAW.

Click to expand...

If you look at the page to which I linked above, you will see that DXO is calculating linear Dynamic Range based on the ratio between the linear range (4095 for 12 bit data, for example) and the measured noise. Thus, for the Canon 30D in the example which they meaure a raw standard deviation of noise of about 2.86 12-bit ADU's in the green channel, they calculate a DR of the range of 4096 divided by 2.86 = a DR of 1431.5:1 or about 10.48 EV stops, which is correct. They must use some kind of standard tone resonse curve to convert this to a 8-bit JPEG range for the number of steps, but this doesn't mean much to me.

The problem is that when they try to apply this analysis to the JPEG images, they are subject to any JPEG processing that has been done, incuding any noise smoothing done by demosiacing, they will need to try to back out the tone response curve, and sharpening and colour treatments will have already been done.

I would have accepted their DR results had they been done on raw data.

Regards, GordonBGood

RPulley said:

So, they are wrong and you are right, or is it that they did not use
the correct software?

Click to expand...

DXO is comparable software to Imatest in measuring DR, but analysis has to be based on raw data in order to get an accurate measurement of DR. For instance, at the DXO sample page at http://info.dxo.com/demokit_analyzer/page1.html , they determine that the Canon 30D at ISO 3200 has a 8.81 EV stop DR for JPEG data when the raw data for the same ISO says the camera has a 8.44 EV stop DR. We know that JPEG's will never have a greater DR than raw unless some processing manipulation has been done; therefore, DXO's JPEG DR determination is not accurate using their own data. According to my own measurements, the DXO raw data DR determinations for the Canon 30D are accurate.

You may also note that the standard deviation noise charts form a smooth curve for raw data and a very rough jagged one as to tendencies for JPEG data, indicating that the results are distorted or inexact.

RPulley said:

I seem to recall that you predicted that all reviewers would affirm
that you were right in declaring the K20D to be lagging in dynamic
range. As i recall, there was no mention of needing to use only
Imatest to confirm your conclusions.

Click to expand...

Imatest as applied to raw data will likely confirm my results, just as would DXO when applied to raw data.

All of the following quotes were made when referring to raw data, which this review did not use.

RPulley said:

Some of your quotes:

"it looks likely that the K20D actually loses a full stop of DR range
in the shadows as compared to the K10D, even on a sensor area basis,
in spite of not having the K10D's problems with Scanline Pattern
Noise (SPN, Vertical Pattern Noise -VPN - in the K10D's case). I'm
afraid that much is going to be made of this in upcoming reviews from
all the sites, although it won't affect most people's use of the
camera."

"No matter what causes the problem, and we will not likely ever know,
it puts a restriction on the deep shadow DR of the K20D at low ISO's
that isn't there for some of its competitors."

You are 0-1 right now, but don't worry, there is still hope that some
reviewers will agree with you about how bad the K20D compared to the
D300 or 40D, or even the K10D. At that point those of us actually
shooting with the K20D will still have to judge for ourselves who is
right.

Click to expand...

I was referring to raw analysis of the data not JPEG. I suspect that the limitations of "backing out" the linear response of the camera as to DR from JPEG readings is too inexact to be able to determine an accurate DR number using DXO calculations.

I have never said that the K20D is unusable or not enjoyable, just that is has this limitation as far as the noise floor somewhat limiting the DR. Many users will never notice this as they never push the usable DR envelope of the camera. Obviously you are one of them.

GordonBGood

@vortout,GordonBGood
Thanks for the very comprehensive replies!

Looking at the graphs, I see there's an "Average color fidelity (while balance accuracy)" graph which makes no sense for a RAW image.

It's a shame they've used JPEG images for these tests, it makes them pretty meaningless for me, as I shoot in RAW. If they'd used RAW they'd be an excellent set of tests.

The tests do seem to contradict a lot of the subjective reports about these cameras coming in on the forums here. But given the results are on JPEG data, I don't think they contribute too much.

--
A rose by any other name is still a chicken.

Gordon,

if you look at the fine details you will see that k20d outperformed k10 in resolution. When i say outperform i actually mean that the distance is huge. Look at the wooden fan details, look at the green box, look at chinese fan. Somewhere you will not be able even to recognize some details clearly visible at k20 shot. try higher magnification, something about 200-400%

The resolution issue also kills your argument about k20d extra noise reduction applied. I mean that extra noise reduction does not bring extra resolution as i believe.

I also disagree about colors. There is a minor difference is some colors, but we cannot speak about mute colors at all. i have calibrated apple display and not that bad color vision

I do not think that the noise difference in my test is minor. it is huge by the fact and k10 has much more noise in very practical iso 1600 2 stops pushing up

(by the way compare files sizes. k10 one is much bigger than k20 because f the noise which affects jpeg compression )

in spite of the fact that pushing iso 100 6 stops up has a very liitle practical value I will make a test with underexposed images by pushing it up by 6 stops. I will use both 100 iso and 1600 iso

Oleg_v does not have k10 or k20 at all. he got DS

Do not take me wrong i am not protecting k20 because i have it. I also have k10 and DS, but i think it is worth to demystify some rumor in this forum.

GordonBGood said:

vortout said:

Gordon and anyone who is interested
You can download files from there:
http://idisk.mac.com/dsemeonov-Public/k10vsk20iso1600/?view=web
K10 and k20, both iso 1600, 1 sec
I used silkypix
as i wrote before LR and dcraw are not relevant to k20d yet
i downsized k20 to k10. i think it is fair when we compare noise and
details in shadows. I pushed both shots up 2 stops

Click to expand...

Yes, downsizing is fair.

GordonBGood said:

So. k20d outperforms k10d without any doubts. i cannot judge if there
is 1 stop. i need to do more tests but k20d delivers better S/N ratio
for sure
What is advantage of having both cameras you always can evaluate on
better/worse base

Click to expand...

However, there are some processing differences between the two files
even though you used Silkypix for both. Even though the grey
exposure levels of these two images are almost identical, the colours
in the K20D image are much more muted than for the K10D, which
somewhat reduces the colour noise of the K20D image (see the green
table edge at across the bottom of the images). Also, you are
starting to see the advantage of the noise reduction applied to the
K20D image in the raw file which kicks in at ISO 1600 as determined
by Oleg_V. Even so, although the noise is slightly lower for this
K20D image as compared to the K10D image they are similar enough so
that the measured DR won't be much different.

Where you would really see the differences between the two cameras is
if you did the same test but underexposed by even more (6 stops
instead of 2 stops for ISO 100, and 5 stops for ISO 200), and then
boosted them up to ISO 6400 equivalent as you did here. Then you
would see the higher floor noise of the K20D at low ISO's as compared
to the K10D, although you will likely also see Vertical Pattern Noise
(VPN) for the K10D, although these particular scenes don't really
have enough dark grey flat textured surfaces to be able to see it.

Don't get me wrong in thinking that I'm trying to "knock" the K20D,
as it is likely the camera I would buy if I didn't buy the more "bang
for the Baht" K200D and have more to spend on some good lenses. It
just has this one weakness that somewhat limits the usable full
resolution DR at low ISO's and also slightly limits the high ISO
performance against some of its other brand competitors; however this
limitation is actually easier to work around than the VPN problem of
the K10D, and many users who don't push the performance envelope of
either will never have a problem with any of these limitations.

Regards, GordonBGood

Click to expand...

--
http://www.photomiks.com
Regards, Dmitry

Gordon, this test definitely confirms all your predictions:
http://www.optyczne.pl/69.8-Test_aparatu-Pentax_K20D_Zakres_tonalny.html
K20D has DR similar to E-3 and noticeably lower than competition.
--
http://jaad75.zenfolio.com

seachicken2000 said:

@vortout
This would make little sense to me too. The tests on this site seem
fantastic, but it's all meaningless without knowing whether RAW or
JPEG was used for the tests.

Click to expand...

From Diwa labs:

"Advanced cameras have a number of adjustable parameters for image quality, such as Noise Reduction. We make three measurements; with all parameters set to minimum or deactivated, with factory default settings and with RAW. The differences are often considerable. To make test results comparable, we now only publish test results with factory default settings, which deliver optimum overall image quality. "

--
Take care
R
http://www.flickr.com/photos/raphaelmabo

NR

lock

If you ever notice the difference in a jpeg....yes, the winner it is. But I really think a bigger difference is needed before you actually see it.

What makes it interesting is that DR in RAW of the k20 is poor compared to 40d and d300. But we all in the end convert our raw to jpegs for printing , do we ? It seams that the 40d and the d300 loses their raw head start more than I expected.

lock

Rupert60 said:

Interesting indeed, but I a really not impressed with the DR on any
of the DSLR's at the current time. I think this will be an area of
major improvement in the not so distant future, after the MP wars die
off.

Click to expand...

Very good point. I think we'll see major advances both in the camera technology itself and in software that will give you the opportunity to automatically shadow or highlight parts of a picture and correct them selectively. But there's still a long way to go before we can match the fantastic DR (and AWB!) capability of our own eyes...
--
Espen

Thanks for clearing this up Gordon.

Ray- give it up. ;-)

Ok,

here you can download new pictures:
http://idisk.mac.com/dsemeonov-Public/k10vsk20%20more/

I did not make 6 stops sorry. Silkypix can give you 3 stops only but the result is obvious anyway. Check the files sizes again.

I did 3 stops underexposed shots at 100iso and 1600iso
i pushed it to get normal exposure and converted with Silkypix.

Under iso 1600 k20d outperforms k10d again very much. As you see you cannot even understand that it is a roоster at a wooden fan
Under iso 100 we have again far lesser noise at k20 picture

I see the prove that k20 has better DR (signal/noise ratio) in shadows .

Please consider once again. I think that dcraw and ACR are NOT GOOD for pentax k20d at all. It is crucial when we speak about resolution and behaviour in noisy areas. If you use in your tests wrong convertor it can put everything up-side-down. It happened to the Polish test.

vortout said:

in spite of the fact that pushing iso 100 6 stops up has a very
liitle practical value I will make a test with underexposed images by
pushing it up by 6 stops. I will use both 100 iso and 1600 iso

Click to expand...

--
http://www.photomiks.com
Regards, Dmitry

Popular Photography also found the Sonys to be lacking significantly in resolution. Quote from the A350 test:

Experienced DSLR shooters know that megapixels don't tell the whole story, and the A350 confirms this. In Pop Photo Lab tests, the A350's APS-sized 14.2MP sensor delivered less detail (average 2150 lines of resolution at ISO 100-800) than the 12.2MP Sony A700 (2280 lines). Resolution was significantly below the 2350 lines of the Pentax K20D at ISO 100, and nearly the same as the Pentax at ISO 6400 with noise reduction on. (Indeed, the Sony A350 captured detail on par with the 10.1MP Canon EOS 40D.)

MightyMike said:

how is it that the A700 and D300 which use the same sensor are so far
off? is this jpeg dynamic range????
--
Mike from Canada

'I like to think so far outside the box that it would require a
telephoto lens just to see the box!' ~ 'My Quote '



http://www.airliners.net/search/photo.search?sort_order=views%20DESC&first_this_page=0&page_limit=121&&emailsearch=mighty_mike88%40hotmail.com&thumbnails=

Click to expand...

--

Chris
http://www.imagineimagery.com

seachicken2000 said:

@vortout
This would make little sense to me too. The tests on this site seem
fantastic, but it's all meaningless without knowing whether RAW or
JPEG was used for the tests.

I've searched their site for information on this, but come up with
nothing. Strange that they would omit this, given how comprehensive
these tests seem to be.

Can you say what makes you think the tests were done on JPEGs?

I am also a little puzzled by their tonal range results. In the noise
section of the Test Results section, they claim that 8-bits is the
theoretical maximum. At the bottom of this page they test D40x RAW
output, and it's seems true that 8-bits is the ceiling. But the D40x
RAW file can represent sensor data in 12-bits / channel. Surely
12-bits is the theoretical limit for the tonal range of a channel in
RAW.

--

Click to expand...

If you bothered to look at the site you would know the answer to this question. The test methodlogy and a smaple for the 40D is explained on the site, and it appears that the test is done in RAW.

Ray

GordonBGood said:

RPulley said:

So, they are wrong and you are right, or is it that they did not use
the correct software?

Click to expand...

DXO is comparable software to Imatest in measuring DR, but analysis
has to be based on raw data in order to get an accurate measurement
of DR. For instance, at the DXO sample page at
http://info.dxo.com/demokit_analyzer/page1.html , they determine that
the Canon 30D at ISO 3200 has a 8.81 EV stop DR for JPEG data when
the raw data for the same ISO says the camera has a 8.44 EV stop DR.
We know that JPEG's will never have a greater DR than raw unless some
processing manipulation has been done; therefore, DXO's JPEG DR
determination is not accurate using their own data. According to my
own measurements, the DXO raw data DR determinations for the Canon
30D are accurate.

You may also note that the standard deviation noise charts form a
smooth curve for raw data and a very rough jagged one as to
tendencies for JPEG data, indicating that the results are distorted
or inexact.

GordonBGood said:

I seem to recall that you predicted that all reviewers would affirm
that you were right in declaring the K20D to be lagging in dynamic
range. As i recall, there was no mention of needing to use only
Imatest to confirm your conclusions.

Click to expand...

Imatest as applied to raw data will likely confirm my results, just
as would DXO when applied to raw data.

All of the following quotes were made when referring to raw data,
which this review did not use.

GordonBGood said:

Some of your quotes:

"it looks likely that the K20D actually loses a full stop of DR range
in the shadows as compared to the K10D, even on a sensor area basis,
in spite of not having the K10D's problems with Scanline Pattern
Noise (SPN, Vertical Pattern Noise -VPN - in the K10D's case). I'm
afraid that much is going to be made of this in upcoming reviews from
all the sites, although it won't affect most people's use of the
camera."

"No matter what causes the problem, and we will not likely ever know,
it puts a restriction on the deep shadow DR of the K20D at low ISO's
that isn't there for some of its competitors."

You are 0-1 right now, but don't worry, there is still hope that some
reviewers will agree with you about how bad the K20D compared to the
D300 or 40D, or even the K10D. At that point those of us actually
shooting with the K20D will still have to judge for ourselves who is
right.

Click to expand...

I was referring to raw analysis of the data not JPEG. I suspect that
the limitations of "backing out" the linear response of the camera as
to DR from JPEG readings is too inexact to be able to determine an
accurate DR number using DXO calculations.

I have never said that the K20D is unusable or not enjoyable, just
that is has this limitation as far as the noise floor somewhat
limiting the DR. Many users will never notice this as they never
push the usable DR envelope of the camera. Obviously you are one of
them.

GordonBGood

Click to expand...

I think you are wrong about how the tests are conducted.

In the test FAQ explaining how they conduct each test they give a sample using a Nikon D40X camera, and in the attached samples, I see references only to a nef file, which I believe is a RAW file, is it not?

Here is the FAQ for the noise test:

http://www.diwa-labs.com/wip4/test_result.epl?cat=11127

Scroll to the bottom for links to pdf files with the RAW test file link in them.

Of course, nothing is said about how they control for other variables in the tests, so there is room to wonder about that causing incorrect results.

Ray

I'm convinced they used JPEGs.

On the FAQ for the noise test, there are files for tests using RAW and JPEG.

However, if you go to the lab results page for the Sony A350 (for example), scroll down to the Noise section and click "Download PDF", in the PDF file it says:

Image name D:\testbilder\sony\sony_a350\DSC00116.JPG

--
A rose by any other name is still a chicken.