Actual photos hint at flawed theoretical measurements by bclaff

Status
Not open for further replies.
It is mandatory, if you want to properly investigate and get the correct conclusions.
The correct conclusion will always be correct. May or may not require proper investigate to come to it. Some sites do that for you. Some better than others. The correct time is the correct time...clock doesn't have to run to get it right once in awhile :)
You'd be surprised how many correct things are debatable,
Doesn't mean they still aren't correct
That's precisely what it means. Correctness must be properly established.

If you start with the premise that some claim is correct - without knowing what correct is - then what you're doing is... wait for it... it was on the first page of the review... jump to conclusion.
...and how many time people fail to even define the problem.
Sometimes what one sees as a problem really isn't for another individual
Besides, the idea was to reach the correct conclusion.
You may have indeed wanted to reach....some just want to know the correct conclusion with out doing all the reaching. That's why I like Bills data....nice of him to do the reaching for me.
So you're saying that people should read Bill's findings without understanding what they mean, then "jump to conclusion"?

Or are you just playing word games and discuss something completely unrelated to what I wrote?

Alex
 
It is mandatory, if you want to properly investigate and get the correct conclusions.
The correct conclusion will always be correct. May or may not require proper investigate to come to it. Some sites do that for you. Some better than others. The correct time is the correct time...clock doesn't have to run to get it right once in awhile :)
You'd be surprised how many correct things are debatable,
Doesn't mean they still aren't correct
That's precisely what it means. Correctness must be properly established.
Not necessarily for the end user.
You may have indeed wanted to reach....some just want to know the correct conclusion with out doing all the reaching. That's why I like Bills data....nice of him to do the reaching for me.
So you're saying that people should read Bill's findings without understanding what they mean, then "jump to conclusion"?
If they want....his data is correct. His conclusions are pretty specific. PDR of camera X at ISO Y is Z. No "need" to understand what they mean in all cases. I can indeed conclude that Camera Y has a higher PDR at ISO X than camera Z. Like MP for some....all that really matters is which has more. They don't care what that means. That's perfectly valid for some. And may certainly not be valid reasoning or others. Conclusion was still correct...knowing didn't change that
 
It is mandatory, if you want to properly investigate and get the correct conclusions.
The correct conclusion will always be correct. May or may not require proper investigate to come to it. Some sites do that for you. Some better than others. The correct time is the correct time...clock doesn't have to run to get it right once in awhile :)
You'd be surprised how many correct things are debatable,
Doesn't mean they still aren't correct
That's precisely what it means. Correctness must be properly established.
Not necessarily for the end user.
Necessary for the end user, as the end user is trying to decide to upgrade/buy the K-1 II or not.
You may have indeed wanted to reach....some just want to know the correct conclusion with out doing all the reaching. That's why I like Bills data....nice of him to do the reaching for me.
So you're saying that people should read Bill's findings without understanding what they mean, then "jump to conclusion"?
If they want....his data is correct. His conclusions are pretty specific. PDR of camera X at ISO Y is Z. No "need" to understand what they mean in all cases.
Not even what PDR means? I don't think you're promoting a mindless acceptance of numbers.

How about more complicated stuff like what it means for your images?

Alex
 
It is mandatory, if you want to properly investigate and get the correct conclusions.
The correct conclusion will always be correct. May or may not require proper investigate to come to it. Some sites do that for you. Some better than others. The correct time is the correct time...clock doesn't have to run to get it right once in awhile :)
You'd be surprised how many correct things are debatable,
Doesn't mean they still aren't correct
That's precisely what it means. Correctness must be properly established.
Not necessarily for the end user.
Necessary for the end user, as the end user is trying to decide to upgrade/buy the K-1 II or not.
For the one you have in mind maybe. Lots of users out there that just don't fit into the box you or someone else may want to put them in. Doesn't mean the conclusions are not still correct
You may have indeed wanted to reach....some just want to know the correct conclusion with out doing all the reaching. That's why I like Bills data....nice of him to do the reaching for me.
So you're saying that people should read Bill's findings without understanding what they mean, then "jump to conclusion"?
If they want....his data is correct. His conclusions are pretty specific. PDR of camera X at ISO Y is Z. No "need" to understand what they mean in all cases.
Not even what PDR means? I don't think you're promoting a mindless acceptance of numbers.
Not promoting....but certainly pointing out that mindless acceptance of numbers often drives sales. Lots of folks point to PDR as the end all be all without really understanding what it meas. And Bills charts are still correct and fine/valid conclusions can be drawn from them
How about more complicated stuff like what it means for your images?
I already pointed out what it means for my images. That doesn't invalidate someone else's conclusions.

--
My opinions are my own and not those of DPR or its administration. They carry no 'special' value (except to me and Lacie of course)
 
Last edited:
According to the "measurements" and results graphics:
  1. a Sony A7R3 should provide the exact same dynamic range at ISO 100 as a K-1 II.
No, it should provide the same 'photographic' dynamic range (PDR), which it does. PDR uses a higher threshold for 'acceptable noise', and if you look at the less dark patches (remember, this is already after a 6 EV push), like the third grey patch from the right, the cameras are essentially even in performance. Any advantage to the K-1 II in this patch is either because of additional 'baseline' NR performed by ACR (ACR performs varying levels of NR from camera to camera, even when turned off), or additional Raw filtering done by Pentax.

(If you're wondering why I might even suspect / suggest the K-1 II might be doing some low level Raw filtering at ISO 100, it's because if you compare it to the K-1, the K-1 has a more random noise pattern, while the K-1 II has vertical / horizontal 'rice grain' patterns reminiscent of noise reduction).
  1. there is mentioning of loss of details for the K-1 II but not for the Sony A7R3
You're pointing to a section of the studio scene we know is rendered softly due to the FE 85/1.8 lens. Look more centrally, and the detail in the a7R III is comparable and/or higher contrast.
But then we are not primarily theoreticians, but photographers, so the proof lies in the image:

https://www.dpreview.com/reviews/im...8&x=-0.6873051525279211&y=0.11229493274677838

Raising shadows is the main usage for a lot of sensor dynamic range.

Maybe I am blind, but the actual photo directly contradicts the bclaff's theory curves:
  1. the Sony A7R3 shows substantial (!) extra noise compared to a Pentax K-1II, so there is no way it has even a comparable dynamic range.
Some actual SNR measurements of our ISO 100 +6 EV Raw files using RawDigger (a7R III normalized to 36MP):

3rd from right grey patch: 9.2 (K-1 II) | 9.3 (a7R III)

2nd from right grey patch: 6.7 (K-1 II) | 5.1 (a7R III)

Darkest grey patch: 4.3 (K-1 II) | 2.9 (a7R III)

The Pentax and Sony are similar (9.2 vs 9.3 SNR) for 'reasonable' threshold cut-offs. That's why Bill's PDR values show the cameras to be the same. By the time you get to SNR = 5 or so (which I'd prefer not to include in my images, but YMMV), the K-1 II is a bit cleaner, and at SNR of 3 the K-1 II fares even better. That's why Bill's 'Engineering Dynamic Range' (EDR, which uses a threshold of SNR=1 on the lower end) is better for the K-1 II (13.8 vs 13.6 EV).

But by the time you're at SNR = 5 or below, for an ISO 100 image, you're not likely to want to include those tones in your image, and would probably clip them to black. That's why EDR has limited utility and PDR tends to be more reflective of 'usable' dynamic range (Bill can chime in here on what the 36MP pixel-level SNR threshold would be for his PDR metric).
  1. the Sony A7R3 has massive loss of detail when using the dynamic range to a large extent, much more than a Pentax K-1II, but somehow I only read the narrative of how the Pentax has some detail rendering issues.
Again, that's not correct - you're conflating lens issues with detail retention due to dynamic range.

-Rishi
 
It is mandatory, if you want to properly investigate and get the correct conclusions.
The correct conclusion will always be correct. May or may not require proper investigate to come to it. Some sites do that for you. Some better than others. The correct time is the correct time...clock doesn't have to run to get it right once in awhile :)
You'd be surprised how many correct things are debatable,
Doesn't mean they still aren't correct
That's precisely what it means. Correctness must be properly established.
Not necessarily for the end user.
Necessary for the end user, as the end user is trying to decide to upgrade/buy the K-1 II or not.
For the one you have in mind maybe. Lots of users out there that just don't fit into the box you or someone else may want to put them in. Doesn't mean the conclusions are not still correct
You may have indeed wanted to reach....some just want to know the correct conclusion with out doing all the reaching. That's why I like Bills data....nice of him to do the reaching for me.
So you're saying that people should read Bill's findings without understanding what they mean, then "jump to conclusion"?
If they want....his data is correct. His conclusions are pretty specific. PDR of camera X at ISO Y is Z. No "need" to understand what they mean in all cases.
Not even what PDR means? I don't think you're promoting a mindless acceptance of numbers.
Not promoting....but certainly pointing out that mindless acceptance of numbers often drives sales. Lots of folks point to PDR as the end all be all without really understanding what it meas. And Bills charts are still correct and fine/valid conclusions can be drawn from them
How about more complicated stuff like what it means for your images?
I already pointed out what it means for my images. That doesn't invalidate someone else's conclusions.
Mako, we're deviating way off my point - no offense, but this somehow tends to happen in our discussions.

My point, let's keep in mind, is that people should be aware about what Bill's result means. If they care about the subject.

Alex
 
Last edited:
My point, let's keep in mind, is that people should be aware about what Bill's result means. If they care about the subject.
My point is...nothing out there indicates that's a requirement. Your "should:" isn't a universal truth. Nothing wrong with that and it dopesn't change the correctness of Bill's conclusions or the usefulness of the data . Lots of valid reasons why folks don't need to be fully aware. Bill did the work for them. That's OK/valid for some

--
My opinions are my own and not those of DPR or its administration. They carry no 'special' value (except to me and Lacie of course)
 
Last edited:
...

https://www.dpreview.com/reviews/im...8&x=-0.6873051525279211&y=0.11229493274677838

...

Maybe I am blind, but the actual photo directly contradicts the bclaff's theory curves:
  1. the Sony A7R3 shows substantial (!) extra noise compared to a Pentax K-1II, so there is no way it has even a comparable dynamic range.
  2. the Sony A7R3 has massive loss of detail when using the dynamic range to a large extent, much more than a Pentax K-1II, but somehow I only read the narrative of how the Pentax has some detail rendering issues.
Same comparison, different part of the photo, chosen to allow better comparison of detail loss:

https://www.dpreview.com/reviews/im...08&x=1.1435551419264773&y=0.06650006941552133

It looks to me like the K1-II is missing a lot more detail than the a7III, but the a7III Image looks noiser.

This is the sort of thing you see when one camera has baked-in NR, but I wouldntl expt that at such a low ISO.

The question is, how much of the apparent noise advantage of the Pentax is acutually the effect of the loss of detail?

While scrolling around the comparison image, I find that I mostly prefer the Sony's high contrast but noisier image over the Pentax's blurry image.
 
Last edited:
Dear Rishi,

I admire your and other DPR staff patience in dealing with the Pentax crowd. You promptly review the basically unchanged camera, wade through the muck of limited choices and Pentax lens sample variations and try to select the best lens out of 5, mainly trying to help Pentax users whether to upgrade or not (mostly for them is the review, the rest of the market has probably irreversibly moved on), and this is what you get: flying accusations of sloppiness, bias, incompetence, almost as if all your reviewing protocols are suddenly called into question. I, also a K-1 user, fully support you, believe all your intentions were good, testing done in the most professional manner considering the small staff on hand and breath of tests made in the review, and I will fully understand if you decide it is not worth the trouble (considering the small and shrinking Pentax market share, and the forum reaction) to continue testing Pentax cameras. I love my K-1, but now this is mostly a camera for the Pentax die-hards, who grew to love and appreciate unique Pentax features and eco-system, but in broader market context it is sadly losing its footing, with the strange situation of releasing pro like bulky and heavy lenses, some pretty expensive, and at the same time offering static shooting oriented inexpensive body, and cancelling most forms of pro support outside Japan. And losing all (AF) third party lens support.

All the best

DS21
 
...

https://www.dpreview.com/reviews/im...8&x=-0.6873051525279211&y=0.11229493274677838

...

Maybe I am blind, but the actual photo directly contradicts the bclaff's theory curves:
  1. the Sony A7R3 shows substantial (!) extra noise compared to a Pentax K-1II, so there is no way it has even a comparable dynamic range.
  2. the Sony A7R3 has massive loss of detail when using the dynamic range to a large extent, much more than a Pentax K-1II, but somehow I only read the narrative of how the Pentax has some detail rendering issues.
Same comparison, different part of the photo, chosen to allow better comparison of detail loss:

https://www.dpreview.com/reviews/im...08&x=1.1435551419264773&y=0.06650006941552133

It looks to me like the K1-II is missing a lot more detail than the a7III, but the a7III Image looks noiser.

This is the sort of thing you see when one camera has baked-in NR, but I wouldntl expt that at such a low ISO.

The question is, how much of the apparent noise advantage of the Pentax is acutually the effect of the loss of detail?

While scrolling around the comparison image, I find that I mostly prefer the Sony's high contrast but noisier image over the Pentax's blurry image.
For now, compare region near the center please. The lens used by DPR to shoot K-1ii's studio scene is defective/decentered. This issue has been raised many times in criticisms of DPR's K1-ii review, that's why DPR is reshooting the studio scene with another lens, hopefully a good copy.
 
Dear Rishi,

I admire your and other DPR staff patience in dealing with the Pentax crowd. You promptly review the basically unchanged camera, wade through the muck of limited choices and Pentax lens sample variations and try to select the best lens out of 5, mainly trying to help Pentax users whether to upgrade or not (mostly for them is the review, the rest of the market has probably irreversibly moved on), and this is what you get: flying accusations of sloppiness, bias, incompetence, almost as if all your reviewing protocols are suddenly called into question.
The test was sloppy, and frankly, their reviewing protocols are questionable.

Sorry if you don't like the facts, you can come up with alternate facts if you like.
I, also a K-1 user, fully support you, believe all your intentions were good, testing done in the most professional manner considering the small staff on hand and breath of tests made in the review, and I will fully understand if you decide it is not worth the trouble (considering the small and shrinking Pentax market share, and the forum reaction) to continue testing Pentax cameras.
Right, like that would be productive.

What would be more productive is publishing tests that use consistent methodology.
I love my K-1, but now this is mostly a camera for the Pentax die-hards, who grew to love and appreciate unique Pentax features and eco-system, but in broader market context it is sadly losing its footing, with the strange situation of releasing pro like bulky and heavy lenses, some pretty expensive, and at the same time offering static shooting oriented inexpensive body, and cancelling most forms of pro support outside Japan. And losing all (AF) third party lens support.

All the best

DS21
 
...

https://www.dpreview.com/reviews/im...8&x=-0.6873051525279211&y=0.11229493274677838

...

Maybe I am blind, but the actual photo directly contradicts the bclaff's theory curves:
  1. the Sony A7R3 shows substantial (!) extra noise compared to a Pentax K-1II, so there is no way it has even a comparable dynamic range.
  2. the Sony A7R3 has massive loss of detail when using the dynamic range to a large extent, much more than a Pentax K-1II, but somehow I only read the narrative of how the Pentax has some detail rendering issues.
Same comparison, different part of the photo, chosen to allow better comparison of detail loss:

https://www.dpreview.com/reviews/im...08&x=1.1435551419264773&y=0.06650006941552133

It looks to me like the K1-II is missing a lot more detail than the a7III, but the a7III Image looks noiser.

This is the sort of thing you see when one camera has baked-in NR, but I wouldntl expt that at such a low ISO.

The question is, how much of the apparent noise advantage of the Pentax is acutually the effect of the loss of detail?

While scrolling around the comparison image, I find that I mostly prefer the Sony's high contrast but noisier image over the Pentax's blurry image.
For now, compare region near the center please. The lens used by DPR to shoot K-1ii's studio scene is defective/decentered. This issue has been raised many times in criticisms of DPR's K1-ii review, that's why DPR is reshooting the studio scene with another lens, hopefully a good copy.
OK. When I look at the text above the central square or the rectangular/triangular patterns just to the left of the central square,I see the same differences though not to the same degree: Pentax image less sharp and less obvious noise.
 
...

https://www.dpreview.com/reviews/im...8&x=-0.6873051525279211&y=0.11229493274677838

...

Maybe I am blind, but the actual photo directly contradicts the bclaff's theory curves:
  1. the Sony A7R3 shows substantial (!) extra noise compared to a Pentax K-1II, so there is no way it has even a comparable dynamic range.
  2. the Sony A7R3 has massive loss of detail when using the dynamic range to a large extent, much more than a Pentax K-1II, but somehow I only read the narrative of how the Pentax has some detail rendering issues.
Same comparison, different part of the photo, chosen to allow better comparison of detail loss:

https://www.dpreview.com/reviews/im...08&x=1.1435551419264773&y=0.06650006941552133

It looks to me like the K1-II is missing a lot more detail than the a7III, but the a7III Image looks noiser.

This is the sort of thing you see when one camera has baked-in NR, but I wouldntl expt that at such a low ISO.

The question is, how much of the apparent noise advantage of the Pentax is acutually the effect of the loss of detail?

While scrolling around the comparison image, I find that I mostly prefer the Sony's high contrast but noisier image over the Pentax's blurry image.
For now, compare region near the center please. The lens used by DPR to shoot K-1ii's studio scene is defective/decentered. This issue has been raised many times in criticisms of DPR's K1-ii review, that's why DPR is reshooting the studio scene with another lens, hopefully a good copy.
OK. When I look at the text above the central square or the rectangular/triangular patterns just to the left of the central square,I see the same differences though not to the same degree: Pentax image less sharp and less obvious noise.
Maybe that’s due to different resolutions of the two sensors, but I can’t say for sure without seeing crop of the exact region you’re taking about.
 
Last edited:
... When I look at the text above the central square or the rectangular/triangular patterns just to the left of the central square,I see the same differences though not to the same degree: Pentax image less sharp and less obvious noise.
Maybe that’s due to different resolutions of the two sensors
42MP vs. 36MP? Perhaps, but unlikely. I wouldn't expect to see any effect from such a small (8%) difference in linear resolution.
but I can’t say for sure without seeing crop of the exact region you’re taking about.
Just use the link in the previous post. Wasn't my message clear enough about where to look?
 
... When I look at the text above the central square or the rectangular/triangular patterns just to the left of the central square,I see the same differences though not to the same degree: Pentax image less sharp and less obvious noise.
Maybe that’s due to different resolutions of the two sensors
42MP vs. 36MP? Perhaps, but unlikely. I wouldn't expect to see any effect from such a small (8%) difference in linear resolution.
8% is actually quite a bit of difference. I would think if you can't see a difference from that sort of resolution increase you aren't looking very hard.
but I can’t say for sure without seeing crop of the exact region you’re taking about.
Just use the link in the previous post. Wasn't my message clear enough about where to look?
 
...

https://www.dpreview.com/reviews/im...8&x=-0.6873051525279211&y=0.11229493274677838

...

Maybe I am blind, but the actual photo directly contradicts the bclaff's theory curves:
  1. the Sony A7R3 shows substantial (!) extra noise compared to a Pentax K-1II, so there is no way it has even a comparable dynamic range.
  2. the Sony A7R3 has massive loss of detail when using the dynamic range to a large extent, much more than a Pentax K-1II, but somehow I only read the narrative of how the Pentax has some detail rendering issues.
Same comparison, different part of the photo, chosen to allow better comparison of detail loss:

https://www.dpreview.com/reviews/im...08&x=1.1435551419264773&y=0.06650006941552133

It looks to me like the K1-II is missing a lot more detail than the a7III, but the a7III Image looks noiser.

This is the sort of thing you see when one camera has baked-in NR, but I wouldntl expt that at such a low ISO.

The question is, how much of the apparent noise advantage of the Pentax is acutually the effect of the loss of detail?

While scrolling around the comparison image, I find that I mostly prefer the Sony's high contrast but noisier image over the Pentax's blurry image.
For now, compare region near the center please. The lens used by DPR to shoot K-1ii's studio scene is defective/decentered. This issue has been raised many times in criticisms of DPR's K1-ii review, that's why DPR is reshooting the studio scene with another lens, hopefully a good copy.
OK. When I look at the text above the central square or the rectangular/triangular patterns just to the left of the central square,I see the same differences though not to the same degree: Pentax image less sharp and less obvious noise.
Maybe that’s due to different resolutions of the two sensors, but I can’t say for sure without seeing crop of the exact region you’re taking about.
Maybe it's due to the D FA 24-70 being slightly less sharp than the Sony 85mm? Maybe there's more sharpening applied to the Sony images?

Choosing areas with the largest difference is wrong. You have to choose areas where no other factors are involved. That's part of the understanding what you're seeing I was talking about above.

In these tests, by the way, there should be no NR applied - not the kind detected by Bill Claff in any case. And the sensors are the same. In other words, whatever you're seeing is not NR.

Alex
 
My point, let's keep in mind, is that people should be aware about what Bill's result means. If they care about the subject.
My point is...nothing out there indicates that's a requirement.
Riiiight. In this thread, people are "seeing" signs of NR at ISO 100, while Bill detected NR only from ISO 640.

Alex
 
Maybe that’s due to different resolutions of the two sensors
42MP vs. 36MP? Perhaps, but unlikely. I wouldn't expect to see any effect from such a small (8%) difference in linear resolution.
8% is actually quite a bit of difference. I would think if you can't see a difference from that sort of resolution increase you aren't looking very hard.
Maybe average eyesight is better in Saskatchewan than everywhere else in the world. IME most people cannot reliably detect an 8% difference in linear resolution. Thom Hogan says:

"As I've written before, studies tend to say that you really need at least 15% change in resolution before most people can even detect a difference."
 
Hello DS21
Dear Rishi,

I admire your and other DPR staff patience in dealing with the Pentax crowd. You promptly review the basically unchanged camera, wade through the muck of limited choices and Pentax lens sample variations and try to select the best lens out of 5, mainly trying to help Pentax users whether to upgrade or not (mostly for them is the review, the rest of the market has probably irreversibly moved on), and this is what you get: flying accusations of sloppiness, bias, incompetence, almost as if all your reviewing protocols are suddenly called into question. I, also a K-1 user, fully support you, believe all your intentions were good, testing done in the most professional manner considering the small staff on hand and breath of tests made in the review, and I will fully understand if you decide it is not worth the trouble (considering the small and shrinking Pentax market share, and the forum reaction) to continue testing Pentax cameras. I love my K-1, but now this is mostly a camera for the Pentax die-hards, who grew to love and appreciate unique Pentax features and eco-system, but in broader market context it is sadly losing its footing, with the strange situation of releasing pro like bulky and heavy lenses, some pretty expensive, and at the same time offering static shooting oriented inexpensive body, and cancelling most forms of pro support outside Japan. And losing all (AF) third party lens support.

All the best

DS21
Sad post, and a little unfair to the Pentax users, who did replied to the review.

I am a developing engineer - for measurement instruments - and am named in some patens.

They did a good work ?

- when I test a JPG for a comparison : both cameras should have the same JPG setting.....but this team used bright and natural : LOL

- when I test the AF-C with a lens @ 200mm and compare later the results at 100%, I use focus priority for to let the camera get it's best results: but the team used released priority. This you can do when you have a big DOF, or with a wide angle lens, but not with a 200mm ! This team let deliberately allow the camera do take a picture, even when the camera's AF say: I'm not spot on. Nothing else is this release priority.

- The team did post dull and less sharp files for the comparison tool

You can call me a fanboy, but sorry, such idiotic mistakes I can't do in my work ...else...;-)

But you turn it around ....the fanboys complain as usual.

best regards KPM2
 
Riiiight. In this thread, people are "seeing" signs of NR at ISO 100, while Bill detected NR only from ISO 640.
Not sure how that relates to.... People must/need/should/are required to pay attention, and it's mandatory they do, to what the numbers mean. I suspect, haven't looked yet, you could "see" NR at ISO 100 using RawDigger that might not be "seen" in Bill's graph data. Even if there....I doubt a practical concern for most though. But in a gear forum...it may certainly be a big/valid concern. Not so much in one of the Photography forums, IMO. Maybe for an astro photographer only. Was helping a bunch of folks with that last week. The degree they went to to have full control over every aspect was amazing. Mandatory NR would have drove them crazy....regardless of the practical impact

--
My opinions are my own and not those of DPR or its administration. They carry no 'special' value (except to me and Lacie of course)
 
Last edited:
Status
Not open for further replies.

Keyboard shortcuts

Back
Top