Rishi and Everyone, Sorry Not A Cyclist, Large Images

Started Jul 7, 2016 | Discussions thread
Rishi Sanyal
Rishi Sanyal dpreview Admin • Posts: 850
Re: Posted this 10 hours ago

MightyMike wrote:

Rishi Sanyal wrote:

I have to say it never gets easier. For my first 1.5 years at DPReview, I worked 14-18 hour days, every day. That's not even an exaggeration. I don't know if you've noticed but our testing has changed dramatically in the past 2 years, and being new to the industry while trying to effect these changes consumed my life. Thankfully, I had/have a great team to work with. Back then, as we were introducing principles like ISO-invariance, equivalent focal plane exposure, the role of shot and read noise and quantization noise in understanding sensor performance, dynamic range, and subject tracking (particularly, differentiating subject tracking abilities, as they differ vastly between cameras), we had vehement nay-sayers, as much of this stuff was new, and not talked about yet by us or other sites. Or when on-sensor PDAF surpassed DSLR PDAF in certain respects. I suppose any time there are new things/ideas that aren't canonically accepted, there's just a lot of skepticism and knee-jerk reactions.

And though I've tried to scale back my hours to something more reasonable, addressing these knee-jerk reactions never gets easier. Because the entire reason for those long hours (my colleagues put in long hours as well) is our passion to be right and inform our readers with the right information.

Thanks for posting a bit of background, i doubt many on the forums know much about what goes on behind the scenes, I know i sure don't.

So when a whole bunch of people start off with the complete opposite understanding - that we're intentionally trying to deceive, paid off, incompetent, etc. - it's extremely saddening and disheartening particularly because of the delta between that perception and the actual truth.

Much better would be to engage in constructive dialogue about how we could work together, put heads together to come up with new ideas, to test cameras better.

Tough to do when people start off with a knee-jerk, sorry for making it hard.

But when the conversation is stuck at 'you're intentionally deceiving us!' or 'you're obviously incompetent and don't even know how to place an AF point over your subject!' - the conversation unfortunately doesn't move forward.

I think you appreciate as well as I do that even when the center point was over the bike (there was a miscommunication between Chris - who did the test - and I, who only reviewed the end results), the results were exactly the same, because the bike handles and other features (or Richard's leg) that fell under the center AF point were all, most of the time, still out-of-focus, with focus falling well behind the bike in even those sets.

I don't think i got involved in that line of questioning, i read it with interest but wanted to wait till something was definite an explanation or a whoops.

We knew we'd already correctly assessed the overall performance, because I myself had done the test previously with Sam - the rollover we'd initially published which was focused on Sam's face using the center point the entire time (but where AF Hold was set to 'Low'). That showed the same exact hit rate (did you see that run? It's what was initially published for the first few hours the review was up).

Now, the latest run, which is essentially exactly like what we published initially on Tuesday save for AF Hold is now 'off', shows exactly the same hit-rate - about 4 sharp images. And yet we have forum members here who were claiming we were incompetent and didn't know how to keep an AF point on our subject (as if placing the AF point on a different part of the subject changed the fact that the camera still couldn't keep up in terms of focus) area now claiming 'results are much better - 4 shots are in focus!'

I hate to say it but it is hard to know exactly what the photographer did unless we (a forum and perhaps Pentax expert) those who are questioning things were right there with you, but i do get it, I really get bothered when people don't take me at my word, its not in my nature (nor most i suppose) to lie or mislead, if i truly believe something and someone can't except that it hurts because i truly believe my point of view. I would expect the same applies to you and everyone at DPR.

Fair enough. We're taking this as impetus to write an in-depth about how we test AF and why. A FAQ of sorts. We realize the need for some boiler plate text on our AF pages that outline the painstaking efforts we go through to vet our results, and the cross-correlation we do between 'lab' (bike, mannequin, etc.) tests and real-world tests/experience. We always make sure the two agree.

What's odd, though, is when a number of such critics will point to some random video that doesn't actually allow for any real conclusion, but at least says what they want to hear ('AF is pretty good'). This is classic confirmation bias, and indicates the lack of any desire to actually understand what we did. Particularly funny is when the references video/test that supposedly disproves us shows the exact opposite: in that particular video, Chris follows up Nick's 'pretty good' comment with 'It's definitely not in the same ballpark as Canon or Nikon', and even the 0.8MP downsized images they show, you can see many out of focus shots - meaning, if anything, that their video suggests as bad or even worse performance than what we showed in our single point examples (did they use single point? I don't know, they didn't even elaborate on their test). Choosing to believe a test that doesn't even explain itself over a test (ours) that at least tries to be as standardized as possible for what it is, done in triplicate over 20+ times with all data provided - well, that's plain and simple confirmation bias.

The other thing is this: if there's any confusion over what we did, how we tested, how thorough we tested, there's a way to ask. A polite way to ask - the way you might ask me had you met me in person. I'm fairly confident that 90% of these people, had they asked me in person, wouldn't have been a jerk about it. And yet on the internet, being a jerk and engaging in knee-jerk reactions is apparently OK.

But here's why I have such a hard time with it: I don't go barging at other websites with guns blazing accusing the reviewers of clearly being incompetent, stupid, biased, paid-off, etc. just because I don't know exactly what they did.

If I see something odd or that needs clarification, I ask.

But maybe I'm just civil.

Thank you for being civil, Mike.

Cheers, Rishi

-- hide signature --

Rishi Sanyal, Ph.D
Technical Editor | Digital Photography Review
dpreview.com (work) | rishi.photography (personal)

 Rishi Sanyal's gear list:Rishi Sanyal's gear list
Sony RX100 IV Nikon D810 Canon MP-E 65mm f/2.5 1-5x Macro Nikon 85mm F1.8G Sigma 35mm F1.4 DG HSM Art +3 more
Post (hide subjects) Posted by
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark MMy threads
Color scheme? Blue / Yellow