Difference between a review and - garbage

Started Mar 9, 2006 | Discussions
Wolfeye
Senior MemberPosts: 1,018
Like?
Difference between a review and - garbage
Mar 9, 2006

I have to say I was mildly amused at the recent announcement of "guidelines on what is a review versus a preview" that showed up here at dpreview. All reviews are arbitrary, and thus subject to debate, but it would sure be nice if the reviewers bothered to show bad results from the camera they review. I mean, ANY camera can take nice, sharp, crisp pictures on a sunny day. But when the review states "poor performance with flash" and the sample gallery contains no flash pictures, the review tends to fall more into the disreputable bin. Likewise, with DSLRs where the review states "poor performance with the kit lens" and all the gallery pics are taken with a better lens, how can that be considered useful? Maybe it's just me but when the review states "lots of noise at ISO 800" and there's only one gallery picture taken at ISO 800, I'm left wondering what good the review was.

BJN
BJN
Veteran MemberPosts: 4,890Gear list
Like?
Start your own review site
In reply to Wolfeye, Mar 9, 2006

The amount of work that goes into the reviews at this site and the others in the "triumvirate" is huge. If you don't have any trust in the reviewer to spend time showing the meaningful results you're free to start your own site and include whatever you think is important. Best luck.

But I doubt that your review comments would have much credibility. You've called the best reviews in the industry "garbage" and I think that says everything we need to know about your objectivity, perspective and civility.

-- hide signature --

BJN

 BJN's gear list:BJN's gear list
Panasonic Lumix DMC-GM5
Reply   Reply with quote   Complain
superbouncykid
Contributing MemberPosts: 816
Like?
I agree with your points
In reply to Wolfeye, Mar 9, 2006

same problem here, always taking about "noises", but sample pictures contains mostly daylight pictures.

To the first replier:

The OP is not trying to bash reviewers, just making some valid points on what we miss in reviews.

ignore the "make your own review" comments, some people just don't like to be challenged.
--
http://www.watchingtheclouds.org

Reply   Reply with quote   Complain
Wolfeye
Senior MemberPosts: 1,018
Like?
Re: Start your own review site
In reply to BJN, Mar 9, 2006

BJN wrote:

The amount of work that goes into the reviews at this site and the
others in the "triumvirate" is huge. If you don't have any trust in
the reviewer to spend time showing the meaningful results you're
free to start your own site and include whatever you think is
important. Best luck.

Yes, that is my goal, to start my own review site. I have always wanted to start a review site. It's my life's goal. Right....

But I doubt that your review comments would have much credibility.

Really? If I wrote a review that said hey, this camera is noisy at ISO 800 and had ten pictures, all taken at ISO 800, on my review site, and all ten were noisy, you'd call that - um, incredible?

You've called the best reviews in the industry "garbage" and I
think that says everything we need to know about your objectivity,
perspective and civility.

Civility isn't what's under your skin here, is it? You like what dpreview provides and open your mouth wide when they want to feed it to you, with nary a question. I notice you didn't bother to question whether I had examples of my complaints, but really, you're not interested in that either are you? You just wanted to make an anonymous, feel-good, yes-I-love-dpreview and if you don't, you suck kind of post.

-- hide signature --

BJN

Reply   Reply with quote   Complain
istonic
Forum MemberPosts: 78
Like?
Re: Difference between a review and - garbage
In reply to Wolfeye, Mar 9, 2006

I was amused too but from a slightly different perspective: on my favorite site (this one, of course!), who has the courage to tell things by their names, it is announced and promoted a quantitative approach to reviewing: reviews are just bigger then the previews.

Reply   Reply with quote   Complain
rennie12
Senior MemberPosts: 2,844
Like?
The reviews by all three sites are good. The only two
In reply to istonic, Mar 10, 2006

things I wish for are: If noise is a problem there should be at a minimum several shots demonstrating the problem, not just one ISO 400 shot (or wherever the problem speed is).

Secondly - what do most people photograph most ? People, that's what. It stands to reason shots of people are most informative - everybody know pretty much what a shot of a person should look like. Yet sometimes there are only one or two shots with people. (I realize there may be a release problem here, but one could used the same person over and over). Actually that is a pretty good idea since the viewer builds an image of what this target person looks like. Mr Askey used, I believe, his wife a number of times; and Dave uses his wife regularly (don't know their last name).

-- hide signature --

Best wishes, rennie12
'Do not wish for things to be as you would like them to be -
Wish to like things as they are'

Reply   Reply with quote   Complain
Phil Askey
Veteran MemberPosts: 9,820Gear list
Like?
Re: I agree with your points
In reply to superbouncykid, Mar 10, 2006

I'm sorry but that's simply not fair. The majority of pictures taken by people are in daylight situations yet we go out of our way to include high ISO shots in our galleries.

http://www.dpreview.com/gallery/canoneos5d_samples/
(9 high ISO shots)

http://www.dpreview.com/gallery/sonydscr1_samples/
(6 high ISO shots)

The irony in the original posters complaint is that NOBODY talked about noise or tried to quantify it until we introduced our noise tests.

superbouncykid wrote:

same problem here, always taking about "noises", but sample
pictures contains mostly daylight pictures.

To the first replier:
The OP is not trying to bash reviewers, just making some valid
points on what we miss in reviews.

ignore the "make your own review" comments, some people just don't
like to be challenged.
--
http://www.watchingtheclouds.org

-- hide signature --

Phil Askey
Editor / Owner, dpreview.com

 Phil Askey's gear list:Phil Askey's gear list
Leica M9 Canon EOS 5D Mark III Olympus OM-D E-M5 Leica M8 Panasonic Lumix DMC-GH4 +11 more
Reply   Reply with quote   Complain
steve1984
Senior MemberPosts: 1,257
Like?
What on earth is a 'quick preview' - please help me!
In reply to Wolfeye, Mar 10, 2006
Reply   Reply with quote   Complain
Phil Askey
Veteran MemberPosts: 9,820Gear list
Like?
Re: What on earth is a 'quick preview' - please help me!
In reply to steve1984, Mar 10, 2006

Wow you had to go quite a long way back to find something which doesn't fit our standards huh?

steve1984 wrote:

I though there was only 2 types of 'previews' - now I'm even more
confused

http://www.dpreview.com/news/0302/03021803cp3100preview.asp

http://www.dpreview.com/news/0302/03021805cpsqpreview.asp

-- hide signature --

Phil Askey
Editor / Owner, dpreview.com

 Phil Askey's gear list:Phil Askey's gear list
Leica M9 Canon EOS 5D Mark III Olympus OM-D E-M5 Leica M8 Panasonic Lumix DMC-GH4 +11 more
Reply   Reply with quote   Complain
David Martin
Forum ProPosts: 17,056
Like?
The reviews would never be finished
In reply to Wolfeye, Mar 10, 2006

You are trying to set impossible standards. I don't care what is reviewed, cars, computers, anything - not every word will have total cross-referencing.

Every additional image and page of review takes one heck of a lot of work - the reviews here are the most thorough on the web - although Dave at IR also does an excellent job.

I'm sorry, but to me your critique is not in touch with reality at all - just do your own homework a bit, instead of expecting Phil and Simon to cover every eventuality.

If you would like samples, say with flash, that is not an uncommon request to make to early adopters, and they are usually good enough to help, although they also tend to get a lot of unfair criticism for anything which anyone feels is lless than perfect in their test set-ups.

Wolfeye wrote:

I have to say I was mildly amused at the recent announcement of
"guidelines on what is a review versus a preview" that showed up
here at dpreview. All reviews are arbitrary, and thus subject to
debate, but it would sure be nice if the reviewers bothered to show
bad results from the camera they review. I mean, ANY camera can
take nice, sharp, crisp pictures on a sunny day. But when the
review states "poor performance with flash" and the sample gallery
contains no flash pictures, the review tends to fall more into the
disreputable bin. Likewise, with DSLRs where the review states
"poor performance with the kit lens" and all the gallery pics are
taken with a better lens, how can that be considered useful? Maybe
it's just me but when the review states "lots of noise at ISO 800"
and there's only one gallery picture taken at ISO 800, I'm left
wondering what good the review was.

-- hide signature --

Regards,
DaveMart

Please see profile for equipment

Reply   Reply with quote   Complain
steve1984
Senior MemberPosts: 1,257
Like?
Re: What on earth is a 'quick preview' - please help me!
In reply to Phil Askey, Mar 10, 2006

Actually I just typed "preview" into your excellent search facility and came across those examples straight away (you did actually say you'd been following these standards since the beginning. Next time I won't take what you say so literally!)

Also, it was actually meant as a (obviously bad) joke - sorry if I offended you.

Phil Askey wrote:
Wow you had to go quite a long way back to find something which
doesn't fit our standards huh?

steve1984 wrote:

I though there was only 2 types of 'previews' - now I'm even more
confused

http://www.dpreview.com/news/0302/03021803cp3100preview.asp

http://www.dpreview.com/news/0302/03021805cpsqpreview.asp

-- hide signature --

Phil Askey
Editor / Owner, dpreview.com

Reply   Reply with quote   Complain
Joe0Bloggs
Veteran MemberPosts: 5,391
Like?
Now try again
In reply to Phil Askey, Mar 10, 2006

with reviews of small sensor cameras... I sure had a hard time trying to find a meaningful low light shot from the Sony H1 or the Canon S2... there's one really low light shot for the S2 in a cathedral but it appears to have been taken with abnormally steady hands (or a tripod?) at ISO 50.....

Phil Askey wrote:
I'm sorry but that's simply not fair. The majority of pictures
taken by people are in daylight situations yet we go out of our way
to include high ISO shots in our galleries.

http://www.dpreview.com/gallery/canoneos5d_samples/
(9 high ISO shots)

http://www.dpreview.com/gallery/sonydscr1_samples/
(6 high ISO shots)

The irony in the original posters complaint is that NOBODY talked
about noise or tried to quantify it until we introduced our noise
tests.

superbouncykid wrote:

same problem here, always taking about "noises", but sample
pictures contains mostly daylight pictures.

To the first replier:
The OP is not trying to bash reviewers, just making some valid
points on what we miss in reviews.

ignore the "make your own review" comments, some people just don't
like to be challenged.
--
http://www.watchingtheclouds.org

-- hide signature --

Phil Askey
Editor / Owner, dpreview.com

Reply   Reply with quote   Complain
gerard boulanger
Senior MemberPosts: 2,262Gear list
Like?
Well...
In reply to Wolfeye, Mar 10, 2006

... I think dpreview is doing a decent work even if sometimes I wish to see less of this or more of that in a review or a preview or a full test.

What I like in the dpreview work is their consistency of doing their work, so we can get a better idea of what is what.

Gerard

-- hide signature --

'Better than a good equipment: a good eye and heart'

 gerard boulanger's gear list:gerard boulanger's gear list
Fujifilm X-Pro1 Fujifilm XF 18mm F2 R Fujifilm XF 35mm F1.4 R Fujifilm XF 60mm F2.4 R Macro Fujifilm XF 27mm F2.8
Reply   Reply with quote   Complain
Joe0Bloggs
Veteran MemberPosts: 5,391
Like?
Then again
In reply to Joe0Bloggs, Mar 10, 2006

For the digicams you have the excellent ISO400 studio test shots, which usually quantifies the noise levels of the digicams very well. I just wish you could dim the lights for the high ISO test shots to simulate a real low light shooting situation. As it is, the high shutter speeds make the ISO400 shots look better than they are in real life (dark current noise?)

Now what I really don't understand is why you don't do the same high ISO test shots for DSLRs as you do for digicams. Or actually you do, but you show us only one crop from the studio shot, and a very different crop from any crop you could find on the digicam reviews... and the whole test shot can be downloaded for digicams but not DSLRs...

Reply   Reply with quote   Complain
Wolfeye
Senior MemberPosts: 1,018
Like?
Re: I agree with your points
In reply to Phil Askey, Mar 10, 2006

Those are two recent reviews, and if you're using them as reference points, I stand corrected, and apologize. The ISO thing wasn't my only point - in the Canon 350D review the conclusion states "New kit lens disappointing at telephoto with smaller apertures" and yet not a single gallery picture is taken with the kit lens. ISO problems are still not shown very often - the recent review of the Panasonic FZ30 had only a single gallery shot at high ISO. Very often there is a mention of "poor flash performance" with small digicams but no pictures to back it up.

Ultimately, all I'm asking, is that dpreview show us the bad shots as well as the good. It's hard to fairly appraise a camera when something is said about it and no examples exist.

Phil Askey wrote:
I'm sorry but that's simply not fair. The majority of pictures
taken by people are in daylight situations yet we go out of our way
to include high ISO shots in our galleries.

http://www.dpreview.com/gallery/canoneos5d_samples/
(9 high ISO shots)

http://www.dpreview.com/gallery/sonydscr1_samples/
(6 high ISO shots)

The irony in the original posters complaint is that NOBODY talked
about noise or tried to quantify it until we introduced our noise
tests.

superbouncykid wrote:

same problem here, always taking about "noises", but sample
pictures contains mostly daylight pictures.

To the first replier:
The OP is not trying to bash reviewers, just making some valid
points on what we miss in reviews.

ignore the "make your own review" comments, some people just don't
like to be challenged.
--
http://www.watchingtheclouds.org

-- hide signature --

Phil Askey
Editor / Owner, dpreview.com

Reply   Reply with quote   Complain
futureassassin
Contributing MemberPosts: 775
Like?
There are reviews on this site?
In reply to Wolfeye, Mar 10, 2006

Hmm didnt notice till you mentioned it. I guess I'll have to venture out of the forums.

-- hide signature --

  1. ####################

http://www.aperture.ca
(My Gallery)

http://www.discoverbbc.com
(Get 20MB free photo storage for photos of British Columbia)

Reply   Reply with quote   Complain
Simon Joinson
dpreview AdminPosts: 4,464Gear list
Like?
Re: Difference between a review and - garbage
In reply to Wolfeye, Mar 10, 2006

Wolfeye wrote:

I have to say I was mildly amused at the recent announcement of
"guidelines on what is a review versus a preview" that showed up
here at dpreview. All reviews are arbitrary, and thus subject to
debate, but it would sure be nice if the reviewers bothered to show
bad results from the camera they review. I mean, ANY camera can
take nice, sharp, crisp pictures on a sunny day. But when the
review states "poor performance with flash" and the sample gallery
contains no flash pictures, the review tends to fall more into the
disreputable bin. Likewise, with DSLRs where the review states
"poor performance with the kit lens" and all the gallery pics are
taken with a better lens, how can that be considered useful? Maybe
it's just me but when the review states "lots of noise at ISO 800"
and there's only one gallery picture taken at ISO 800, I'm left
wondering what good the review was.

In the case of compact images there is a very good reason for this.

The majority of images will not be taken by average users at ISO 400 (or whatever the top setting is) because to do so requires manual setting of the ISO (auto ISO VERY rarely goes that high), and once you know how useless it is (and lets be honest they all are) you tend to avoid it.

In the course of producing a compact review I will typically take around 1000 shots, using every ISO setting, and will choose one or two from the higher settings to illustrate the point. If we filled the gallery with noisy ISO 400 shots it wouldn't reflect the actual quality of the camera in general use, and would give the impression that these were the only type of pictures the camera can take. In fact we try in the gallery to show the best the camera is capable of. There's a section called 'specific image quality issues' where examples of problems are shown, and the studio shot and noise sections clearly show the limitations of the higher ISO settings.

As for the comment later on about people shots, I agree that this is a major part of most people's photography, which is why we always include a standard skintone flash shot in the tests. Again I will always take lots of shots of friends and family during the course of producing a review, but as they've all asked not to be published on a site as a condition of being photographed continually I can only include one or two in the gallery. i always try to ensure there is a flash shot and a non-flash portrait in the gallery but I accept that this isn't always the case, and it's something i'm working on!
Simon
--
Simon Joinson, dpreview.com

 Simon Joinson's gear list:Simon Joinson's gear list
Fujifilm FinePix X100 Nikon D750 Canon EOS 20D Panasonic Lumix DMC-GM1 Canon EF 100-400mm f/4.5-5.6L IS USM +16 more
Reply   Reply with quote   Complain
David Martin
Forum ProPosts: 17,056
Like?
So why did you phrase it so offensively?
In reply to Wolfeye, Mar 10, 2006

You are now switching to constructive criticism - of course there are aspects where reviews on this site or anywhere else could be improved - there always are.

Comparing the reviews to garbage though is hardly the way to start a constructive dialogue - do you imagine that the people who run this site are robots, and just there for you to abuse?

This and a couple of other sites provide a wonderful free resource for camera enthusiasts.

Your apology would be reasonable if you had phrased the intial critique moderately, but hardly covers the case for the tone you adopted.

Wolfeye wrote:
Those are two recent reviews, and if you're using them as reference
points, I stand corrected, and apologize. The ISO thing wasn't my
only point - in the Canon 350D review the conclusion states "New
kit lens disappointing at telephoto with smaller apertures" and yet
not a single gallery picture is taken with the kit lens. ISO
problems are still not shown very often - the recent review of the
Panasonic FZ30 had only a single gallery shot at high ISO. Very
often there is a mention of "poor flash performance" with small
digicams but no pictures to back it up.

Ultimately, all I'm asking, is that dpreview show us the bad shots
as well as the good. It's hard to fairly appraise a camera when
something is said about it and no examples exist.

Phil Askey wrote:
I'm sorry but that's simply not fair. The majority of pictures
taken by people are in daylight situations yet we go out of our way
to include high ISO shots in our galleries.

http://www.dpreview.com/gallery/canoneos5d_samples/
(9 high ISO shots)

http://www.dpreview.com/gallery/sonydscr1_samples/
(6 high ISO shots)

The irony in the original posters complaint is that NOBODY talked
about noise or tried to quantify it until we introduced our noise
tests.

superbouncykid wrote:

same problem here, always taking about "noises", but sample
pictures contains mostly daylight pictures.

To the first replier:
The OP is not trying to bash reviewers, just making some valid
points on what we miss in reviews.

ignore the "make your own review" comments, some people just don't
like to be challenged.
--
http://www.watchingtheclouds.org

-- hide signature --

Phil Askey
Editor / Owner, dpreview.com

-- hide signature --

Regards,
DaveMart

Please see profile for equipment

Reply   Reply with quote   Complain
Chromatic Aberration
Contributing MemberPosts: 908
Like?
He NEVER singled anyone out...
In reply to BJN, Mar 10, 2006

BJN wrote:

You've called the best reviews in the industry "garbage" and I
think that says everything we need to know about your objectivity,
perspective and civility.

I personally did not see the OP single out any particular review site. One may imply that this site was the center of attention, but it is not certain.

I also agree that the bad images should also be shared provided that user error is ruled out. If the shot look terrible because the reviewwer did it wrong, then there is no need to share an image that would harm the reputation of any camera...but if good technique is used and the image still looks horrible...give us a sample.

-- hide signature --

Life is like a roll of TOILET PAPER; the closer you get to the end, the faster it goes.

Reply   Reply with quote   Complain
Chromatic Aberration
Contributing MemberPosts: 908
Like?
Not exactly out of your way Phil...
In reply to Phil Askey, Mar 10, 2006

Phil Askey wrote:

I'm sorry but that's simply not fair. The majority of pictures
taken by people are in daylight situations yet we go out of our way
to include high ISO shots in our galleries.

Phil,

Thank you for the website...and the unbiased reviews...

However, posting high ISO samples cannot truely be considered "out of your way" can it? At least no more "out of your way" than posting ISO100 shots...

At this point...the number one criteria for judging a camera on this website is its high ISO performance. I am sure it was not like this in the past, and in 5 years, something else will be the hot topic...but right now, High ISO noise is the number one factor for determining the performance of a camera.

-- hide signature --

Life is like a roll of TOILET PAPER; the closer you get to the end, the faster it goes.

Reply   Reply with quote   Complain
Keyboard shortcuts:
FForum MMy threads