DPR Screwed-up the Review of the 50D!

DPR mere showed that Adobe's raw converter converts files from the 40D better than it converts files from the 50D.

Since DPR didn't test with DPP, please tell me why I'm wrong. Certainly based on the DPR results my assertion can't be disproved by their tests. That's really a sorry state of affairs since DPP is provided by Canon along with the 50D.

Seems like "best possible results" should be reported. Not some arbitrary test procedure that favors some cameras over others depending on Adobe!
--
Fred

You are mostly right Fred. The fact that they were happy to publish results with a half-baked version of ACR (4.6 beta) means they weren't too concerned about getting the best results out of this camera.

A digital camera is only as good as the software used to process its files (which includes the software camera makers user to create the in-camera jpegs). Most 50D owners have known since day one that ACR needs some more work before it can match DPP with the 50D, especially the high ISO noise reduction that seems to be the main criticism of the review. Why DPR went ahead and used ACR anyway - and then not at least made a comment that the developing software could be a limitation of the current analysis, is beyond me.

Given the 50D was compared against cameras hat have been out in the market for more than a year, in which time Adobe has optimised support for them in ACR, skews the results even more. Some common sense from DPR would have averted the mess - but part of me says the fuss (and eyeballs on the site) is exactly what they wanted!

It was extremely sloppy of DPR to be so negligent. Shame on them.
--
Fred

I agree totally!!

As i said ultimately the reviews (from the xti) are always negative.

Later the same review talk of camras like the xti as excellent.

I suspect at least a personal love toward Nikon products

That love of Nikon really has no place in a review. Particularly, if DPR expects to remain the premier reviewer of digicams.
--
Fred

fwampler said:

DPR mere showed that Adobe's raw converter converts files from the
40D better than it converts files from the 50D.

Since DPR didn't test with DPP, please tell me why I'm wrong.
Certainly based on the DPR results my assertion can't be disproved by
their tests. That's really a sorry state of affairs since DPP is
provided by Canon along with the 50D.

Seems like "best possible results" should be reported. Not some
arbitrary test procedure that favors some cameras over others
depending on Adobe!

Click to expand...

What about the "best possible results" with the other cameras in the comparisons and their individual reviews? Do you really think that those cameras were shown with their "best possible results"? Can you prove that the 50D is the only camera that wasn't shown with the "best possible results"?

fwampler said:

--
Fred

Click to expand...

You're deflecting the point here. It's clear the 50D was not tested with the best possible software. Whether or not that's the case with other cameras is not the point. Your argument goes to show that there's at least some doubt that the other cameras were tested with optimal software, which casts a shadow over the entire review process on this site. I would be equally appalled if it were shown the Nikons or Sonys were tested with less-than-optimal software and lenses. It's not a defense of Canon, but rather a question mark on the validity of the review.

Guy Lerner said:

You are mostly right Fred. The fact that they were happy to publish
results with a half-baked version of ACR (4.6 beta) means they
weren't too concerned about getting the best results out of this
camera.

Click to expand...

That has been changed and you know it.

Guy Lerner said:

A digital camera is only as good as the software used to process its
files (which includes the software camera makers user to create the
in-camera jpegs).

Click to expand...

Yep, and that goes for ALL the cameras reviewed here.

Guy Lerner said:

Most 50D owners have known since day one that ACR
needs some more work before it can match DPP with the 50D, especially
the high ISO noise reduction that seems to be the main criticism of
the review. Why DPR went ahead and used ACR anyway - and then not at
least made a comment that the developing software could be a
limitation of the current analysis, is beyond me.

Click to expand...

Is ACR (or the way it's used) a limiting factor with all the other DSLRs too?

Guy Lerner said:

Given the 50D was compared against cameras hat have been out in the
market for more than a year, in which time Adobe has optimised
support for them in ACR, skews the results even more.

Click to expand...

Can you prove that ACR is "optimized" for every other DSLR (or any other cameras that shoot raw) and that the use of ACR positively favored all the cameras that were compared with the 50D?

Guy Lerner said:

Some common
sense from DPR would have averted the mess - but part of me says the
fuss (and eyeballs on the site) is exactly what they wanted!
--

Click to expand...

Common sense? That's a good one.

Why don't you fanboys give it a rest? The reviews here aren't perfect, but you guys are only whining because your precious 50D didn't get a Super Duper Best In Class Ultimate Camera Canon Is God Extremely Highly Recommened Don't Bother With Any Other Cameras Everything Else Is Junk We Had An Orgasm By Just Looking At The 50D!! score.

Guy Lerner said:

You're deflecting the point here. It's clear the 50D was not tested
with the best possible software. Whether or not that's the case with
other cameras is not the point. Your argument goes to show that
there's at least some doubt that the other cameras were tested with
optimal software, which casts a shadow over the entire review process
on this site. I would be equally appalled if it were shown the Nikons
or Sonys were tested with less-than-optimal software and lenses. It's
not a defense of Canon, but rather a question mark on the validity of
the review.
--

Click to expand...

Yeah, right. You're only complaining about the way the 50D was tested.

In this case yes, but even you have to agree the testing environment (4.6 beta/final, whatever) was not optimal for the 50D. And if you can see that, you could argue that it's very possible some previous reviews were likewise skewed by non-optimal software (or testing methodologies).

Whatever the case, it shakes the previously rock-solid reputation of DPR reviews.

natureman said:

Guy Lerner said:

You are mostly right Fred. The fact that they were happy to publish
results with a half-baked version of ACR (4.6 beta) means they
weren't too concerned about getting the best results out of this
camera.

Click to expand...

That has been changed and you know it.

Click to expand...

Yes I do. But 4.6 final is still not optimised for the 50D.

natureman said:

natureman said:

A digital camera is only as good as the software used to process its
files (which includes the software camera makers user to create the
in-camera jpegs).

Click to expand...

Yep, and that goes for ALL the cameras reviewed here.

Click to expand...

Absolutely! I wasn't claiming otherwise. I'd argue that ACR is more optmised now for older cameras than it is for newer cameras (given how much better it handles 40D files for example), but that's to be expected.

natureman said:

natureman said:

Most 50D owners have known since day one that ACR
needs some more work before it can match DPP with the 50D, especially
the high ISO noise reduction that seems to be the main criticism of
the review. Why DPR went ahead and used ACR anyway - and then not at
least made a comment that the developing software could be a
limitation of the current analysis, is beyond me.

Click to expand...

Is ACR (or the way it's used) a limiting factor with all the other
DSLRs too?

Click to expand...

Could well be! I don't know, since I don;t own the other cameras to have tested them myself. If it were shown to be a limiting factor for other cameras, then it should be stated as much in the reviews. even a disclaimer to the effect of "results could vary depending on the processing software you use" would go a long way here.

natureman said:

natureman said:

Given the 50D was compared against cameras hat have been out in the
market for more than a year, in which time Adobe has optimised
support for them in ACR, skews the results even more.

Click to expand...

Can you prove that ACR is "optimized" for every other DSLR (or any
other cameras that shoot raw) and that the use of ACR positively
favored all the cameras that were compared with the 50D?

Click to expand...

No I can't (see above). My point is it was not optimised for the 50D (even the final version of ACR is still lacking IMO), and so the review should have made that point clear. I have no issue if DPR wants to claim it is not optimised for other cameras either, but then that begs the question: why the heck are you using the software in the first place?

natureman said:

natureman said:

Some common
sense from DPR would have averted the mess - but part of me says the
fuss (and eyeballs on the site) is exactly what they wanted!
--

Click to expand...

Common sense? That's a good one.

Click to expand...

Ok then, common courtesy.

natureman said:

Why don't you fanboys give it a rest? The reviews here aren't
perfect, but you guys are only whining because your precious 50D
didn't get a Super Duper Best In Class Ultimate Camera Canon Is God
Extremely Highly Recommened Don't Bother With Any Other Cameras
Everything Else Is Junk We Had An Orgasm By Just Looking At The
50D!! score.

Click to expand...

I don't buy cameras based on DPR reviews. I've had my 50D for more than a month now, and bought it for reasons other than what DPR claims to be its 'cons'. If I'm a fanboy of anything it's Canon lenses, not cameras. If Nikon were anywhere near competitive with the lenses that I use from Canon, I'd give them serious consideration. I'm not a big fan of Nikon ergonomics though, but I'd overlook that if the lenses they made were on par with the canon lenses I use (and potentially want to buy in future).

I have exactly the same opinion. Giving DPR the benefit of doubt that they were trying to do an objective test the result is just an amatuerish sloppy job.

Here is what I said in another post.

http://forums.dpreview.com/forums/read.asp?forum=1019&message=29880272

fwampler said:

DPR mere showed that Adobe's raw converter converts files from the
40D better than it converts files from the 50D.

Since DPR didn't test with DPP, please tell me why I'm wrong.
Certainly based on the DPR results my assertion can't be disproved by
their tests. That's really a sorry state of affairs since DPP is
provided by Canon along with the 50D.

Seems like "best possible results" should be reported. Not some
arbitrary test procedure that favors some cameras over others
depending on Adobe!
--
Fred

Click to expand...

Guy Lerner said:

You're deflecting the point here. It's clear the 50D was not tested
with the best possible software. Whether or not that's the case with
other cameras is not the point. Your argument goes to show that
there's at least some doubt that the other cameras were tested with
optimal software, which casts a shadow over the entire review process
on this site. I would be equally appalled if it were shown the Nikons
or Sonys were tested with less-than-optimal software and lenses. It's
not a defense of Canon, but rather a question mark on the validity of
the review.

Click to expand...

I can tell you that there is a huge difference between ACR and Nikon CaptureNX for Nikon DSLR NEFs, both regarding noise and details, not forgetting the colours before the new profiles were made. But of course in manufacturer applications like DPP and CNX, there surely are some NR applied which you can't control, so comparisons of RAWs are difficult and the value of comparisons will be more like JPGs.

Another disadvantage with Nikon DSLR tests here at DPreview is that they never use MLU but only a timer, because they refuse to buy a $10 10pin remote to trigger the shutter using MLU. Every serious photographer knows what this can mean for fine details.

That said, have you seen another review of the 50D, which is totally in line with DPReview's conclusions about the 50D.

http://www.digitalcamerainfo.com/content/Canon-EOS-50D-Digital-Camera-Review-19359.htm

Regards
Ole Thorsen
http://www.pbase.com/ole_thorsen

• OMNISCIENCE

Knowing what
thou knowest not
is in a sense
omniscience.
(Grook by Piet Hein)

I sit with amazement at the claims on the forum.

The Lens is not high enough resolution
The RAW converter was the wrong one
The lens was not focused
It's an anti-canon bias.

Enough, its not like they are the only review pointing out the 50D performance. Have a look at the detail armature photographer review. Very similar.

The JPEG files show the same results as the RAW ones. Now I know the Canon JPEG engine is not the sharpest in the box, but given the 20% extra resolution going into it, if the 50D had a fundamental resolution advantage, some of it should shine through.

And as for the lens, if the 50mm f1.4 is not good enough, few lenses will be. God help you with the new 18-200, or the 17-85 or even the very good 17-55.

Is it not time to say there is enough evidence that its not a big step on in resolution. The has been plenty of reasoned debate as to why going past 12mp on a crop sensor is entering the land of little return for extra work. And we all have seen the more pixels more noise changes.

And finally, in the compact camera arena we have see manufacturers dumbly sticking even more pixels in those compacts when most of them struggle to equal the performance of the older 6 or 8mp cameras that came before.

So why would the same not be possible here?

The samples shown have Photoshop Elements 6 (Mac) in the EXIF - which would also suggest ACR was used. And they cane to the same conclusions? What does that say?

OleThorsen said:

Guy Lerner said:

You're deflecting the point here. It's clear the 50D was not tested
with the best possible software. Whether or not that's the case with
other cameras is not the point. Your argument goes to show that
there's at least some doubt that the other cameras were tested with
optimal software, which casts a shadow over the entire review process
on this site. I would be equally appalled if it were shown the Nikons
or Sonys were tested with less-than-optimal software and lenses. It's
not a defense of Canon, but rather a question mark on the validity of
the review.

Click to expand...

I can tell you that there is a huge difference between ACR and Nikon
CaptureNX for Nikon DSLR NEFs, both regarding noise and details, not
forgetting the colours before the new profiles were made. But of
course in manufacturer applications like DPP and CNX, there surely
are some NR applied which you can't control, so comparisons of RAWs
are difficult and the value of comparisons will be more like JPGs.

Another disadvantage with Nikon DSLR tests here at DPreview is that
they never use MLU but only a timer, because they refuse to buy a $10
10pin remote to trigger the shutter using MLU. Every serious
photographer knows what this can mean for fine details.

That said, have you seen another review of the 50D, which is totally
in line with DPReview's conclusions about the 50D.

http://www.digitalcamerainfo.com/content/Canon-EOS-50D-Digital-Camera-Review-19359.htm

Regards
Ole Thorsen
http://www.pbase.com/ole_thorsen

• OMNISCIENCE

Knowing what
thou knowest not
is in a sense
omniscience.
(Grook by Piet Hein)

Click to expand...

Cant comment about ACR and Nikon, but if that's the case, then DPR should be more upfront about the findings, or at least make the point that "better results can be obtained with better processing software".

My wife and I own both Nikon and Canon cameras (I have a Nikon D300 and my wife the Canon 40D) – both are fine cameras and I have used both interchangeably and am one of the few users that can speak at length about how these two camera compare. Just so no one thinks I’m a troll or looking for flames, I prefer to be a voice of reason living in a photographically integrated family. I began my SLR life as a Canon fan; my 35mm camera was a Canon FTb (which I still have) and I like many Canon offerings for their own strengths – so call me a dual fan of both Canon and Nikon product lines for over three decades now.

I’ve been a member of DPR for years; I trust their testing methodology and DPR has been the most transparent and methodical of testers I’ve seen in my 35 years of reading reviews from many sources. I am sure the testing was credible and I am equally sure the 50D is a decent camera overall; some folks on these forms are simply fanatical. If folks want good reviews of sub-standard produts and not the brutal truth they had better find another site - its as if they are saying "lie to me to make me feel good". What is disappointing is that in the rush for megapixels, this camera took some steps forward at the expense of other that are critical depending on the application.

This review will certainly take a heavy toll on sales no doubt; my wife had been considering upgrading her Canon 40D to the 50D (which I fund), but this review stopped those thoughts in their tracks; instead she has decided to wait for the review of the 5D MKII. This is the true and actual impact of these reviews - like it or not and I don’t think DPR takes their testing lightly as they understand this kind of impact. She will definitely stay with Canon – but her reaction to the review was a big sigh of relief that she doesn’t need to feel as compelled to upgrade now; the 50D seemed to come out so very quickly on the heels of the 40D – too quickly as is now evidenced by the results. She, the Canon fan (not me, the Nikon guy) – was actually a bit gleeful over the 50D review in that she now feels she made the right choice to upgrade to the 40D and glad she didn’t wait for the 50D which she could have done. We all like to feel like we have gotten good value when we risk our hard earned money so I can understand if some 50D buyers are deeply disturbed.

People can’t praise DPR for their often glowing reviews of Canon products in the past and now indict them when the results don’t come out as one would have preferred. If anyone buys a camera before getting some solid feedback from multiple independent reviews and actual users – then they must accept the risk they are taking.

Take heart; Canon will take it on the balance sheet for this one and will have to be more careful in the future. If DPR’s testing was wrong, don’t fret, wait until a few other reviews come out and if they all essentially agree, well, then, you have to call a dud a dud no matter your subjective feelings. I have sense the forthcoming reviews from other major sources will validate DPR’s findings.

Let’s sincerely hope the 5D MKII review doesn’t follow suit; we all benefit when these producers successfully advance the state of the art, drive quality up and prices down through competition. I am disappointed because now Canon has given Nikon reason to maintain a price premium in this class a bit longer.

Mike

--

Polaroid Swinger; Kodak Instamatic 126 Ricoh 500G; Canon FTb; Nikon F2AS; Nikon F3HP; Hasselblad 501CM; Pentax 67II, Nikon 990; Nikon D1x; Nikon D300; PhaseOne P65+ (in my dreams ;-)

I have to agree with Firstlight. As a 39-year Canon man,* I've never seen the "Nikon bias" in DPR that some people refer to, and I have to say there seems to be an element of, "My kid didn't get straight-A's, so I'm suing the school," here.

I'll soon be in position to upgrade, and was looking forward to a 40D when the 50D was released. At first I thought I was lucky not to have gotten a 40D, but at the same time, I had some reservations about 15MP in a 1.6 crop camera. (I've long felt that we'd all be better off if the Megapixel wars had been stopped at, say, 10MP, with additional innovations enabling cameras to produce higher quality results in lower amounts of light, instead of offering no worse IQ at [often pointlessly] higher resolutions). After reading DPR and Digital Camera Info's reviews, I'll more likely get a 40D after all (which is too bad, because I was looking forward to having fine-focus adjustment and the higher-res screen on the back).

I'm sorry to say that it looks like Canon dropped the ball on this one. After owning the high ground for most of this century, they seem to have forgotten how to compete on a level playing field. Sacrificing IQ for MP's is not an improvement.

I hope Canon comes to its senses soon, and concentrates its efforts on improving IQ instead of the "artificial" pixel-count wars.

I'd be delighted to be wrong, if in fact DPP produces significantly better results than ACR, but physics is physics, and there's a limit to how much you can squeeze out of a microscopically-small sensor.

Fortunately, there's no reason to believe the 5DII will be a disappointment (since it uses the already-proven 1Ds III sensor, with considerably larger pixels than any of the compared cameras in the 50D review). Hopefully, the 5DII will keep Canon afloat until the company rights its course in the crop-sensor realm, where it truly seems to have gone astray.

Canon FT, 1969, with FL 28 3.5 and Fl 135 3.5; FTb, 1972; still own my first-generation F-1, purchased new in 1973, along with FD 50 1.4, FD 35 f2, FD 24 2.8, and FD 85 1.8

Currently own Rebel XT (350D) w non-IS kit lens (ugh!) and EF-S 17-55 2.8 IS (swoon...)

natureman said:

fwampler said:

DPR mere showed that Adobe's raw converter converts files from the
40D better than it converts files from the 50D.

Since DPR didn't test with DPP, please tell me why I'm wrong.
Certainly based on the DPR results my assertion can't be disproved by
their tests. That's really a sorry state of affairs since DPP is
provided by Canon along with the 50D.

Seems like "best possible results" should be reported. Not some
arbitrary test procedure that favors some cameras over others
depending on Adobe!

Click to expand...

What about the "best possible results" with the other cameras in the
comparisons and their individual reviews? Do you really think that
those cameras were shown with their "best possible results"? Can you
prove that the 50D is the only camera that wasn't shown with the
"best possible results"?

Click to expand...

Fred, I agree with you about using the "best possible" result, unfortunately DPR has never tested in that method in any review. DPR only tests the camera's in their "default" settings, and the have never converted a single Nikon RAW file using Nikon Capture which many have claimed does a better job than ACR. I would love to see DPR customize each new camera they text til the obtain the "best possible" result for each camera, then compare the images.
http://nickmjr.smugmug.com/
Nick M

Guy Lerner said:

Cant comment about ACR and Nikon, but if that's the case, then DPR
should be more upfront about the findings, or at least make the point
that "better results can be obtained with better processing software".
--

Click to expand...

Maybe they figure that anyone who does raw conversions should already know that?

I just took a look at some of the wording in the reviews and here's what I found in most of the newer reviews:

"For a (more) level playing field for comparison we also shot our studio scene in RAW mode with each camera and converted it using Adobe Camera RAW."

This is from the older Nikon D2x review:

"To equalize the processing of these images we decided to use Adobe Camera RAW (Photoshop CS2, ACR 3.1), while it does not guarantee the exact same processing it's as close as you can get."

Now, maybe I'm mistaken but I thought the wording was different in the newer reviews last time I looked and that the word "equal" or "equalize" was used. Does anyone know whether that's the case or not and whether dpreview has recently changed the wording in the newer reviews?