What Are Your Voting Criteria?

Started 3 months ago | Discussions
kaphinga
kaphinga Veteran Member • Posts: 3,632
What Are Your Voting Criteria?

Just curious, what are your criteria for scoring challenges?

Me? I pretend that I am putting together a photo book about whatever the challenge topic is, rated as follows:

1 star - Needs to be disqualified, usually because it totally doesn't follow the rules (e.g. -- submitting a picture of a dog in a cat challenge). On very rare occasions I have rated something a  "1" because the content is offensive.

1.5 stars - The photographer has done something egregiously bad to the image.  Most often, it's extremely overcooked HDR that has destroyed every ounce of tonality.  Occasionally, it can be images that are severely underexposed, overexposed, or out of focus. Fortunately, I don't see too many of these images, although it really depends on the challenge.

2 stars - Snapshot.  Focus and exposure are generally okay, but there are some serious problems with composition and/or light. No thanks. It won't make it into my book.

3 - 3.5 stars -  Decent image.  It's a candidate but not a shoe-in for my hypothetical book.

4 - 4.5 stars - Definitely would want to include this image in my hypothetical book.

5 stars - Wow!  Just wow!  Cream of the crop.

Occasionally, I will bump a snapshot up a level when the content is compelling or the content makes me think about the topic in a different way.  For example, I boosted a couple of entries in the recent airport challenge because some of the really tiny airports were so unlike my preconceived notion of an airport.  Sometimes, there's a snapshot that provides context or gives an additional dimension to the set.

Sometimes, too, I will dock an image a bit, usually half a star, for distracting but easily fixable flaws.  Couldn't you have just cloned out that little _____ (piece of trash, finger, power line, or whatever)? Or, maybe a minor crop would make a big improvement. I usually try to stay out of color judgements because it's so subjective, but sometimes I will dock an image when there's an obvious color issue that could be resolved easily.

Then after the initial voting, I will do a quick second pass over my votes.

*  If I get to the end of the voting, and I don't have any 4.5 or 5 star images, I will usually boost a few of my favorites from the 4 star group.

* Also, if I see a lot of images of the same thing shot the same way, I might try to distribute my ratings so that they aren't all rated the same.

Oh yeah, and I don't vote on topics that I don't know much about. I don't know enough about bird and airplane photography, for example, to make a decent judgement.

So, I shared my method of voting.  What's yours?   There are no right or wrong answers here.  I'm just curious about what everybody else does.

(I apologize if this has been asked before.  I went through about a year's worth of previous discussions and didn't see this as a topic.)

-- hide signature --

Marie

 kaphinga's gear list:kaphinga's gear list
Nikon D750 Fujifilm X100V Olympus E-M5 III Olympus E-M1 III Apple iPhone 11 Pro
Barry Twycross Veteran Member • Posts: 4,676
Re: What Are Your Voting Criteria?
1

If it's off topic, it scores 1/2*.

If it's on topic it gets another 1/2 a star straight off.

If it's reasonable technical quality it gets another 1/2* straight off.

So a reasonable on topic image gets 1.5* before I look at it.

Then I look at it an go with a gut feel:

2* good

3* very good

4* exceptionally good.

5* Wow!!

I think I've only ever given 3 x 5*s, out of 22696 votes (0.001%).

I also give half stars between those. I'll scrutinize the technical quality, and may knock off stars for flaws. Ridiculously large files get scrutinized extra hard because they annoy me.

If I end up without any 4.5* images, I'll look at my highest rated and promote one or two, so I have a distribution from at least 4.5-1.5*.

 Barry Twycross's gear list:Barry Twycross's gear list
Panasonic GX850 Fujifilm GFX 50R Panasonic Lumix G Vario HD 12-32mm F3.5-5.6 Mega OIS Leica Nocticron 42.5mm Fujifilm GF 110mm F2 +9 more
LauraJ Junior Member • Posts: 27
Re: What Are Your Voting Criteria?

If the picture does not come close to the rules either a .5 or no vote

If the picture simply does nothing for me for the challenge title either .5 or no vote

If the picture is interesting but there is no title to the picture, just image number etc, sorry, I will down vote you for not using your brain to come up with a good title/description.

If the picture is great and the title description really fits the bill, I  will give you a great vote for capturing my interest.

I always look at every image in full screen, can I  picture this blown up and hanging on a wall?  Is this striking looking, something, that makes me go Oh Wow!  If so, full marks for it.

If I have seen the same entry consistently in different challenges without any changes (for example, Noah and his boat, I  am so tired of seeing that one, I  won't even bother giving is a vote any more. The same thing goes for the leaf on the ground, boring!!! Please show some imagination) I cannot even give you credit for trying or trying to be consistent.

Some categories simply do not interest me, so cannot vote. Other categories just thrill me to pieces, the entries are fabulous, and I will give many 4 to 4.5 to 5 stars.

I always check the full rules before voting and decide from that.

If the picture is blurry and I cannot understand what the subject is even after reading the title description, I  will down vote you.

If the challenge says macro and the entries are not macro or even good or excellent close ups, I  will down vote you.

If I have commented on your challenge or on your entry, you have touched me and I want to let you know.

The challenges lately have been fantastic and I thank each and every one of you hosts for this and making the voting so enjoyable  for me.

REHS
REHS Regular Member • Posts: 152
Re: What Are Your Voting Criteria?
2

Hello kaphinga,

So glad to see that you're interested in voting!  I think I've learned more about photography from voting than any other single task, including utube videos.  If you enjoy voting, and want to do it fairly, a voting system is very helpful.  I like what you have developed, and that you are asking for additional example methods is awesome!  I  Here's mine:

3 points if it meets the challenge criteria.

Then I look at look at subject, composition, light, color, detail, and post processing effectiveness.  Depending on how the image meets that evaluation, anywhere from .5 to 1.5 points are added. I save the last .5 for the one or two images that nailed the challenge criteria, mastered my skill criteria, and then have a great "WOW" factor.

For those that didn't meet the challenge criteria, they start of at 2.5, and then get .5 points taken away based on image quality and by how much they missed the challenge criteria.

Please remember, that part of the overall criteria is that all images are supposed to be anonymous, so watermarks and similar identifying marks get dinged a point,  Advertising, and self promotions in the image description are also dinged.

Also, please understand that how you view each image can make a big difference in the effectiveness of your evaluation.  Many great challenges may consist of over 100 entries.  In those challenges, I usually do three separate screenings.  The first is done in slideshow mode, and  a "2, 3, or 4" determination is made.  The next screening is where skill and image quality are analyzed and rated, the last screening is used to assure all are ranked in a 'best to worst' order. These last two are done in full screen mode.  I have discovered that DPR's downsizing program removes data.  I mean that an image viewed as a thumbnail may be soft, the standard view is a little better,  slideshow image may have soft corners and edges, but the full screen mode will be tact sharp.  However, using the full screen mode can be time consuming, since you can't move from image to image in that mode, and you can't go from slideshow mode to full screen either.  That's why I use slideshow for initial screening and then standard and full screen mode for the real detailed evaluation.

And finally, after the challenge is over, compare your rankings to the final results, look to see why your ranking is different.  Get to know some of the real skillful members here and discuss your results with them.  I use voting more as a learning tool, and rely on trusted members for personal feedback on the images that I've submitted.

I hope this long-winded description is helpful, and please feel free to PM anytime you would like to discuss anything regarding this site and/or photography.

Mike(REHS)

 REHS's gear list:REHS's gear list
Sony RX100 VII Nikon D850 Nikon D500 Tamron 15-30mm F2.8 Nikon AF-S Nikkor 24-70mm f/2.8G ED +3 more
JahnG
JahnG Veteran Member • Posts: 3,340
Re: What Are Your Voting Criteria?

REHS wrote:

Hello kaphinga,

So glad to see that you're interested in voting! I think I've learned more about photography from voting than any other single task, including utube videos. If you enjoy voting, and want to do it fairly, a voting system is very helpful. I like what you have developed, and that you are asking for additional example methods is awesome! I Here's mine:

3 points if it meets the challenge criteria.

Then I look at look at subject, composition, light, color, detail, and post processing effectiveness. Depending on how the image meets that evaluation, anywhere from .5 to 1.5 points are added. I save the last .5 for the one or two images that nailed the challenge criteria, mastered my skill criteria, and then have a great "WOW" factor.

For those that didn't meet the challenge criteria, they start of at 2.5, and then get .5 points taken away based on image quality and by how much they missed the challenge criteria.

Please remember, that part of the overall criteria is that all images are supposed to be anonymous, so watermarks and similar identifying marks get dinged a point, Advertising, and self promotions in the image description are also dinged.

Also, please understand that how you view each image can make a big difference in the effectiveness of your evaluation. Many great challenges may consist of over 100 entries. In those challenges, I usually do three separate screenings. The first is done in slideshow mode, and a "2, 3, or 4" determination is made. The next screening is where skill and image quality are analyzed and rated, the last screening is used to assure all are ranked in a 'best to worst' order. These last two are done in full screen mode. I have discovered that DPR's downsizing program removes data. I mean that an image viewed as a thumbnail may be soft, the standard view is a little better, slideshow image may have soft corners and edges, but the full screen mode will be tact sharp.

Thank you very much. Finally somebody else conforms similar observatios I have made, but which nobody else has really commented on or confirmed.

BTW, some of the thumbnail pictures are sharp, whereas the full format is ether sharp or sometimes unsharp.

Jahn

However, using the full screen mode can be time consuming, since you can't move from image to image in that mode, and you can't go from slideshow mode to full screen either. That's why I use slideshow for initial screening and then standard and full screen mode for the real detailed evaluation.

And finally, after the challenge is over, compare your rankings to the final results, look to see why your ranking is different. Get to know some of the real skillful members here and discuss your results with them. I use voting more as a learning tool, and rely on trusted members for personal feedback on the images that I've submitted.

I hope this long-winded description is helpful, and please feel free to PM anytime you would like to discuss anything regarding this site and/or photography.

Mike(REHS)

 JahnG's gear list:JahnG's gear list
Canon PowerShot A650 IS Panasonic ZS100 Pentax K-r Pentax K100D Super Pentax K-S1
Cherry_F
Cherry_F Regular Member • Posts: 399
Re: What Are Your Voting Criteria?

I use more or less the same system as Mike(REHS) above. Starting at 2,5 is the picture meets the host rules, then add or remove points depending on composition, use of light, etc.

In addition I give extra points for pictures that show an intent. When the photographer has obviously made an effort to avoid clichés. In other words, when there is some originality. I also give extra points when I can immediately grasp the intent of the photographer and, at the same time, the picture gives me lots to explore.

I know this can be subjective. Your idea of a book is fine, but I would object that a book has a purpose. I imagine a book about farming for instance, if one wants to publish aesthetic pictures, or really want to show the daily chores, the voting would probably be very different.

Oh, and BTW I have two extra 'rules':
- never vote in a challenge I have entered
- vote for all the pictures of a same challenge, and not only the ones I like/dislike.

Trevor Carpenter
Trevor Carpenter Forum Pro • Posts: 18,058
Re: What Are Your Voting Criteria?

Cherry_F wrote:

I use more or less the same system as Mike(REHS) above. Starting at 2,5 is the picture meets the host rules, then add or remove points depending on composition, use of light, etc.

I do the same starting at 2.5.  To get a 2.5, the picture must meet all criteria, must be pleasing to eye, have a thoughtful composition, be in focus.  From there its is a fairly simple process to go lower or higher.  Having voted I play with the top 10 or so to ensure that the order reflects the quality.

 Trevor Carpenter's gear list:Trevor Carpenter's gear list
Panasonic G85 Panasonic Lumix DC-G9 Panasonic Lumix G Vario 14-140mm F3.5-5.6 O.I.S Olympus M.Zuiko Digital ED 60mm F2.8 Macro Panasonic Leica 100-400mm F4.0-6.3 ASPH
hampshirejohn Regular Member • Posts: 457
Re: What Are Your Voting Criteria?

I can't add much to the discussion on voting methods, other than to note I agree with the concensus of replies so far. But it prompts 2 supplementary questions.

First, do you find your votes spread reasonably widely across the whole range, or do you end up clustered in the middle with only the occasional 0.5/1 and 4.5/5? Do you award similar numbers of 2s, 3s, 4s or do 2.5/3s predominate?

Second, and partly related to this, do you vote in an absolute or relative sense? in other words, if you feel a challenge has an above average standard of entries, do you find you have awarded more marks towards the top end? And conversely if you think the overall standard is lower, that you have awarded more lower marks.

 hampshirejohn's gear list:hampshirejohn's gear list
Panasonic Lumix DMC-GF1 Panasonic Lumix DC-G9 Panasonic Lumix G 20mm F1.7 ASPH Panasonic Lumix G Vario 7-14mm F4 ASPH Panasonic Lumix G Vario 100-300mm F4-5.6 OIS +4 more
kaphinga
OP kaphinga Veteran Member • Posts: 3,632
Re: What Are Your Voting Criteria?

hampshirejohn wrote:

I can't add much to the discussion on voting methods, other than to note I agree with the concensus of replies so far. But it prompts 2 supplementary questions.

First, do you find your votes spread reasonably widely across the whole range, or do you end up clustered in the middle with only the occasional 0.5/1 and 4.5/5? Do you award similar numbers of 2s, 3s, 4s or do 2.5/3s predominate?

It seems to follow a bell curve for me -- 3's predominate with very few 1's and 5's.

Second, and partly related to this, do you vote in an absolute or relative sense? in other words, if you feel a challenge has an above average standard of entries, do you find you have awarded more marks towards the top end? And conversely if you think the overall standard is lower, that you have awarded more lower marks.

I tend to vote relative to the entries, more or less. Usually, I will do a quick pass through all the entries before I start voting, just to see what the range is.  Some challenges may skew a little higher and some a little lower.

If I voted on an absolute level, the marks would probably be much lower. I judge painting shows sometimes, and you really have to work within the range of what is submitted.

-- hide signature --

Marie

 kaphinga's gear list:kaphinga's gear list
Nikon D750 Fujifilm X100V Olympus E-M5 III Olympus E-M1 III Apple iPhone 11 Pro
LauraJ Junior Member • Posts: 27
Re: What Are Your Voting Criteria?
1

I start with a base of 3. If the category is an excellent one and the entries are fantastic, the voting ranges are between 3.5 to 5. There are some that I  wish I  could give a 10 to, they are just that good!

If the category is a good one but the entries are good to mediocre to so-so, I will do any where from 0 to 3. There are many times that I wish we had a thumbs down icon to use on certain entries that just don't fit the category. Then I will give it a .5.

If the category simply does not interest me, I  don't vote.

I am very critical of what I am looking at, and my husband gets critiqued on his photographs by me. He has his definite opinions of what is right and wrong (He has been doing this for 60 years) but I also have my opinions as I know what zings me and others. He has learned to listen to me on some of this.

Bill Thoo Contributing Member • Posts: 532
Re: What Are Your Voting Criteria?
1

hampshirejohn wrote:

I can't add much to the discussion on voting methods, other than to note I agree with the concensus of replies so far. But it prompts 2 supplementary questions.

First, do you find your votes spread reasonably widely across the whole range, or do you end up clustered in the middle with only the occasional 0.5/1 and 4.5/5? Do you award similar numbers of 2s, 3s, 4s or do 2.5/3s predominate?

Second, and partly related to this, do you vote in an absolute or relative sense? in other words, if you feel a challenge has an above average standard of entries, do you find you have awarded more marks towards the top end? And conversely if you think the overall standard is lower, that you have awarded more lower marks.

I'm glad you asked this.  I started off trying to comment about this, then got distracted.

Like others I start at with an assumption the entry scores a 2.5 (or 3 in my case) then add or subtract stars according to the aesthetic or technical merit, or lack of.

I don't aim to score a spread from 1-5 stars for each challenge I vote in.  I will only score a 1 or 5 if the image merits that, rather than for the sake of having a 1-5 spread.  In other words I don't change my criteria for scoring relative to the challenge.  If a challenge is full of amazing images it will therefore score high stars on average, or if full of terrible images it will score low stars on average.  Does that make sense?

 Bill Thoo's gear list:Bill Thoo's gear list
Sony a7R II Canon PowerShot D10 Sony Alpha DSLR-A900 Sony FE 35mm F2.8 Samyang 14mm F2.8 ED AS IF UMC +2 more
Keyboard shortcuts:
FForum MMy threads