Micro Four Thirds Focal Reducer Shootout

Started Feb 20, 2014 | Discussions thread
dougjgreen1 Veteran Member • Posts: 3,547
Re: Given your clear vested interest in the results......

tomtom50 wrote:

dougjgreen1 wrote:

How can we know that you tested equally representative samples of each of these items? Other than the Light Cannon, which is clearly an inferior product, the differences - especially between the Speed Booster and the Lens Turbo, are quite subtle and could easily be explained by sample variations.

How do we know you didn't test a cherry-picked sample of the Speed Booster? It's fairly common practice to use an internally screened sample of one's own product for benchmarking purposes.

How would a tester know they had equally representative copies of two products? In theory he could buy a large number, test them until some sort of bell curve emerged, and compare using median samples.

I think we can safely assume that wasn't done as it would be cost-prohibitive merely to back up a forum post!

On the other hand he disclosed himself as the designer, and Lens Turbo performed pretty decently in the test. He has earned the benefit of my doubt.

I don't think that you can assume at all that this was done strictly as a forum post. What leads you to believe that this was not done as a guerrilla marketing exercise by Metabones the company, funded by Metabones?

Don't get me wrong - I have no issue with the tests themselves, they seem to be perfectly appropriate. But it's naive to think that the poster here just did this out of his own intellectual curiosity and with products he just happened to acquire as a private individual as you or I might. His disclosure about who he is almost certainly makes it clear that this is NOT the case. Which is why I asked a simple question - how were the sample units for each brand obtained? And note that this question has not, as yet been answered. I wonder why, if in fact the OP simply purchased individual production-rev. products for ALL 4 items on the open market, and tested them straight out of the box, why they did not in fact simply say that? Because I'm betting that's NOT how all 4 test subjects were obtained. I simply want to know how these 4 tested units were acquired, in order to draw my own conclusion about the legitimacy of the comparison and whether there was any underlying sampling bias prior to the tests being carried out.

Also, in the interest of full disclosure - in a prior professional engagement, I was in charge of marketing of certain bits of computer and broadband telecommunications technology, and I have a pretty in depth understanding of exactly how benchmark tests can and are used to market the supposed superiority of one piece of technology relative to it's competition, and I am well schooled at the various ways that the tests and the test subjects can be either manipulated or pre-selected to bias the results in favor of the devices that the tester might have an agenda in promoting.

As I said, I think that it's probably pretty apparent that the test itself that the OP described is a clean comparison metric.   But what is far less apparent is whether there was any sample pre-selection at all that was done prior to the tests - either with respect to the metabones unit, or the competitor units that it was compared to.

 dougjgreen1's gear list:dougjgreen1's gear list
Olympus Stylus XZ-10 Nikon 1 V2 Olympus PEN E-P5 Olympus E-M1 Olympus E-PL7 +16 more
Post (hide subjects) Posted by
(unknown member)
Keyboard shortcuts:
FForum PPrevious NNext WNext unread UUpvote SSubscribe RReply QQuote BBookmark MMy threads
Color scheme? Blue / Yellow