I dont understand this voting system

Started 10 months ago | Discussions
Xs Vyper
Junior MemberPosts: 47
Like?
I dont understand this voting system
10 months ago

If a pic has a vote of 1 star, would it win against a pic that has no vote?

If a pic that has the majority of the voter, voted 2.5 star or higher and some sprinkle of voters, voted 2 star and lower would it lose.... to a pic that has fewer votes of 2.5 and higher and also lower votes of 2 star and below?

If so....what's the point of 1 star and 2 star or vice versa...what's the point of higher star? Everything is averaged anyway without some form of weighing. Which lead me to my first question above.....If a pic has a vote of 1 star, would it win against a pic that has no vote?

Likewise....i can ruin a good candidate by just voting .5 star just to bring down the pic average vote.

Doesnt make sense to me...unless im missing something.

Xs Vyper
Junior MemberPosts: 47
Like?
Re: I dont understand this voting system
In reply to Xs Vyper, 9 months ago

Also, another thing...how can people cast a vote or determine the appropriate vote without even viewing the photograph? Unless they have a zoom eye, its kinda strange that they can tell what vote a pic deserve.

Reply   Reply with quote   Complain
Steve Throndson
Senior MemberPosts: 1,328
Like?
Re: I dont understand this voting system
In reply to Xs Vyper, 9 months ago

You are not the only member to not understand the voting system.  It's the subject of much talk on this forum. There's some information here.

http://www.dpreview.com/articles/8593414750/how-to-use-dpreview-challenges

Some members vote on the thumbnail images, others by clicking once on the photo to see it in medium size, or by clicking once more to see the 'full size'.  Some view and vote in slide show mode.

Slide show mode is best, in my opinion.  After clicking 'play' to start the slide show, you can advance the photos with your right arrow key.  To vote, use the numbers on your keyboard (not the number pad).  The up and down arrow keys will add or subtract half a point.

Reply   Reply with quote   Complain
Xs Vyper
Junior MemberPosts: 47
Like?
Re: I dont understand this voting system
In reply to Steve Throndson, 9 months ago

Yeah saw that voting tip before but wouldnt it be better if DP uses statistical mean instead of average? I think its more representative of the real score.

Reply   Reply with quote   Complain
Joe Pineapples
Senior MemberPosts: 2,007Gear list
Like?
Re: I dont understand this voting system
In reply to Xs Vyper, 9 months ago

Xs Vyper wrote:

Yeah saw that voting tip before but wouldnt it be better if DP uses statistical mean instead of average? I think its more representative of the real score.

Dpreview state that they use a Bayesian rating algorithm that is intended to cope with varying numbers of votes per entry, partial voting for some but not all entries, etc. They don't specify the details. Clearly it is frequently manipulated by deliberate high and low voting, especially given the low average number of votes per entry.

Many people have stopped entering the challenges, and many hosts have stopped hosting them, because of these issues. IMO this has a much more significant effect on the challenges than the details of the ranking algorithm used.

Joe

 Joe Pineapples's gear list:Joe Pineapples's gear list
Ricoh GR
Reply   Reply with quote   Complain
Joe Pineapples
Senior MemberPosts: 2,007Gear list
Like?
Why a simple average is bad...
In reply to Xs Vyper, 9 months ago

Xs Vyper wrote:

Yeah saw that voting tip before but wouldnt it be better if DP uses statistical mean instead of average? I think its more representative of the real score.

With relatively few votes per entry, you have very little confidence in the average ranking. For example, if photo 'A' has one vote of 5 stars, but photo 'B' has nine votes of 5 stars and one vote of 4 stars, 'A' ranks higher than 'B' on a simple average.

Perhaps you could elaborate on what you mean by "statistical mean" vs. "average". "Mean" can have different meanings (no pun intended), but generally the mean is the arithmetic average of a set of values...

Joe

 Joe Pineapples's gear list:Joe Pineapples's gear list
Ricoh GR
Reply   Reply with quote   Complain
Xs Vyper
Junior MemberPosts: 47
Like?
Re: Why a simple average is bad...
In reply to Joe Pineapples, 9 months ago

Joe Pineapples wrote:

Xs Vyper wrote:

Yeah saw that voting tip before but wouldnt it be better if DP uses statistical mean instead of average? I think its more representative of the real score.

With relatively few votes per entry, you have very little confidence in the average ranking. For example, if photo 'A' has one vote of 5 stars, but photo 'B' has nine votes of 5 stars and one vote of 4 stars, 'A' ranks higher than 'B' on a simple average.

Perhaps you could elaborate on what you mean by "statistical mean" vs. "average". "Mean" can have different meanings (no pun intended), but generally the mean is the arithmetic average of a set of values...

Joe

Yeah that's exactly what i'm noticing on the effect of votes to the ranking of pics, a few low votes can effectively pull down a good contender from the race. One or a group can simply low vote any decent looking competition out of the challenge.

Sorry was referring to Median instead of Mean

Reply   Reply with quote   Complain
RuthC
Regular MemberPosts: 390
Like?
Re: Why a simple average is bad...
In reply to Xs Vyper, 9 months ago

Xs Vyper wrote:

Yeah that's exactly what i'm noticing on the effect of votes to the ranking of pics,

a few low votes can effectively pull down a good contender from the race. One or a group can simply low vote any decent looking competition out of the challenge.

And that is the dilemma we face with the 'Sandbaggers' who take great delight in voting #.5s for most of the entries, and #5 for their own, usually through multiple accounts, or friends, or family. Some of the hosts keep a close check on unusual voting patterns and entrants whom they know have more than one account, but there are other hosts who just don't care enough, and offer  excuses for the multiple accounts, use of the same cameras, same photos in multiple galleries etc. Ruth

Reply   Reply with quote   Complain
Xs Vyper
Junior MemberPosts: 47
Like?
Re: Why a simple average is bad...
In reply to RuthC, 9 months ago

So this is an ongoing problem with the Challenge. Strange that DP wouldn't do a thing about it, it makes the challenge more of a farce rather than a way for photographers to improve their skill.

Reply   Reply with quote   Complain
Joe Pineapples
Senior MemberPosts: 2,007Gear list
Like?
Re: Why a simple average is bad...
In reply to Xs Vyper, 9 months ago

Xs Vyper wrote:

Yeah that's exactly what i'm noticing on the effect of votes to the ranking of pics, a few low votes can effectively pull down a good contender from the race. One or a group can simply low vote any decent looking competition out of the challenge.

Sorry was referring to Median instead of Mean

I see what you mean. I think in the end the number of votes per entry is so low and varies such a lot between entries, that anything you come up with is easily influenced by a few low (or high) votes. It's frustrating, because the challenges used to be great, people put a lot of effort into them, and doing well or winning was very satisfying. Now it's a bit meaningless. I got totally p***ed off with the whole thing a while ago and withdrew / deleted all of my entries. Then I chilled a bit, and now I just enter for a bit of fun and don't worry about it.

Joe

 Joe Pineapples's gear list:Joe Pineapples's gear list
Ricoh GR
Reply   Reply with quote   Complain
RuthC
Regular MemberPosts: 390
Like?
Re: Why a simple average is bad...
In reply to Xs Vyper, 9 months ago

Xs Vyper wrote:

So this is an ongoing problem with the Challenge. Strange that DP wouldn't do a thing about it, it makes the challenge more of a farce rather than a way for photographers to improve their skill.

You're spot on there!

Reply   Reply with quote   Complain
billythek
Senior MemberPosts: 3,798
Like?
Re: Why a simple average is bad...
In reply to Xs Vyper, 9 months ago

Xs Vyper wrote:

Joe Pineapples wrote:

Xs Vyper wrote:

Yeah saw that voting tip before but wouldnt it be better if DP uses statistical mean instead of average? I think its more representative of the real score.

With relatively few votes per entry, you have very little confidence in the average ranking. For example, if photo 'A' has one vote of 5 stars, but photo 'B' has nine votes of 5 stars and one vote of 4 stars, 'A' ranks higher than 'B' on a simple average.

Perhaps you could elaborate on what you mean by "statistical mean" vs. "average". "Mean" can have different meanings (no pun intended), but generally the mean is the arithmetic average of a set of values...

Joe

Yeah that's exactly what i'm noticing on the effect of votes to the ranking of pics, a few low votes can effectively pull down a good contender from the race. One or a group can simply low vote any decent looking competition out of the challenge.

I guess you are referring to your Dog challenge picture.  Not sure why you got 4 half-star votes.  Your picture clearly met the requirements of the challenge.  It wasn't a great picture, but not terrible, either.  The other outlier was the one five-star vote.  Perhaps the half-star votes were in response to that.

That brings up the question, who can see voting results before voting ends?  I've seen it implied that challenge hosts can.  Are there others?

-- hide signature --

- Bill

Reply   Reply with quote   Complain
RuthC
Regular MemberPosts: 390
Like?
Re: Why a simple average is bad...
In reply to billythek, 9 months ago

No, only the host of that particular challenge can see the votes, and DPR bods can also, if they ever took the time. Even when a host can see the voting pattern, he/she cannot tell the identity of the voters.

Reply   Reply with quote   Complain
Xs Vyper
Junior MemberPosts: 47
Like?
Re: Why a simple average is bad...
In reply to billythek, 9 months ago

billythek wrote:

Xs Vyper wrote:

Joe Pineapples wrote:

Xs Vyper wrote:

Yeah saw that voting tip before but wouldnt it be better if DP uses statistical mean instead of average? I think its more representative of the real score.

With relatively few votes per entry, you have very little confidence in the average ranking. For example, if photo 'A' has one vote of 5 stars, but photo 'B' has nine votes of 5 stars and one vote of 4 stars, 'A' ranks higher than 'B' on a simple average.

Perhaps you could elaborate on what you mean by "statistical mean" vs. "average". "Mean" can have different meanings (no pun intended), but generally the mean is the arithmetic average of a set of values...

Joe

Yeah that's exactly what i'm noticing on the effect of votes to the ranking of pics, a few low votes can effectively pull down a good contender from the race. One or a group can simply low vote any decent looking competition out of the challenge.

I guess you are referring to your Dog challenge picture. Not sure why you got 4 half-star votes. Your picture clearly met the requirements of the challenge. It wasn't a great picture, but not terrible, either. The other outlier was the one five-star vote. Perhaps the half-star votes were in response to that.

That brings up the question, who can see voting results before voting ends? I've seen it implied that challenge hosts can. Are there others?

-- hide signature --

- Bill

Not in particular to my dog entry but its a good case anyway, just noticing that so many equally decent entries in different challenges that are ranking lower and so many mediocre/terrible entries ranking better....more like a pattern for every challenge.

I enjoy a real competition/challenge, its not about winning (in photography anyway), its more of a good feeling how's your photograph stack up with the other photos. It's more of a motivation factor, maybe a better subject, maybe a better composition,, maybe a better lighting, maybe a better gear..next time...IMO, it moves a photographer to do better.

You also brought up the half star point(I didnt even notice that on my dog entry), which according to DP means "Poor image that doesn't fit theme or rules", one vote of half star could be just a misguided vote but four? Looks to me a clear example of sabotage.

Nope, not joining any challenge from now on

Just for the record...i have no clue who voted 5 star for my entry

Reply   Reply with quote   Complain
tasad
Regular MemberPosts: 344
Like?
Re: Why a simple average is bad...
In reply to RuthC, 9 months ago

RuthC wrote:

No, only the host of that particular challenge can see the votes, and DPR bods can also, if they ever took the time.

They took the time ... but it was a few years ago  and  DPR was under different  management :  http://www.dpreview.com/forums/post/35306772

Even when a host can see the voting pattern, he/she cannot tell the identity of the voters.

Reply   Reply with quote   Complain
barb_s
Contributing MemberPosts: 629Gear list
Like?
Re: Why a simple average is bad...
In reply to Xs Vyper, 9 months ago

Xs Vyper wrote:

Just for the record...i have no clue who voted 5 star for my entry

As a host, I've noticed there are people who are very generous with 5 star votes in some categories and I think someone who likes dogs in general voted 5 stars on a whole bunch of the pictures and liked yours also.

-- hide signature --

Barb

 barb_s's gear list:barb_s's gear list
Panasonic Lumix DMC-ZS3 RX100 III Sony SLT-A55 Sony Alpha NEX-6 Sony E 30mm F3.5 Macro +2 more
Reply   Reply with quote   Complain
merops
Senior MemberPosts: 1,090
Like?
Re: Why a simple average is bad...
In reply to Xs Vyper, 9 months ago

Xs Vyper wrote:

...

Not in particular to my dog entry but its a good case anyway, just noticing that so many equally decent entries in different challenges that are ranking lower and so many mediocre/terrible entries ranking better....more like a pattern for every challenge.

I enjoy a real competition/challenge, its not about winning (in photography anyway), its more of a good feeling how's your photograph stack up with the other photos. It's more of a motivation factor, maybe a better subject, maybe a better composition,, maybe a better lighting, maybe a better gear..next time...IMO, it moves a photographer to do better.

You also brought up the half star point(I didnt even notice that on my dog entry), which according to DP means "Poor image that doesn't fit theme or rules", one vote of half star could be just a misguided vote but four? Looks to me a clear example of sabotage.

Nope, not joining any challenge from now on

Just for the record...i have no clue who voted 5 star for my entry

I just wanted to say that hosts vary enormously in quality - they are volunteers: some are diligent and otherscouldn't care less; some can diplomatically manage a challenge in such a way that the competition is reasonably fair. Keep you eyes open and you will notice who are the good hosts, and who not.

Reply   Reply with quote   Complain
Brian in Montana
Regular MemberPosts: 348Gear list
Like?
Re: I dont understand this voting system
In reply to Xs Vyper, 9 months ago

I just stumbled onto this thread after viewing the results of the first DPR challenge I entered and noticing the "outlying" ratings for the top images. I thought I would add my two cents worth.

Note that I was not unhappy about how my own entry was rated - actually a little better than I figured - but I was curious about why the distribution was so "abnormal" (still don't know).

Normal or slightly skewed distributions - aka bell/asymmetrical bell curves - might be predicted when asking a population to "score" some parameter (like the quality of an image) if that parameter has generally known and common expectations (e.g. challenge rules and photographic aspects such as focus, sharpness, composition etc. to name a few). An example: when people are asked to rate the taste of comparable food items, the results are often nearly symmetrical distributions. In such a case, comparing the average (mean) ratings of the items tells you with reasonable accuracy which are better or worse tasting.

When rating distributions are expected to be "normal" but are not, then "fixes" can be applied to the data in order to eliminate bias and derive comparable results (such as winners in a contest). Bayesian theory encompasses a rather elaborate family of equations which are sometimes used to this end. You can look up the gory details, but I don't recommend it - lots of heavy math and specialized statistics vocabulary. In essence, data "fixing" is accomplished by determining an "expected" rating for a parameter, which you might think of as the average rating, and also determining the variability of the rating (how much scatter there is). The fancy equations are used to essentially compress the data for a rating distribution, taking into account "outlying" data points by diminishing their "importance". The average rating ends up being shifted upwards if a statistical test shows that low ratings are outside an expected range (which is based on the bulk of the votes cast). Think of it like dropping the high and low scores for a competition, but more accurately de-weighting points that fall further from the average. This is a simplistic interpretation - DPR's method must use an actual mathematical treatment - but in then end, I believe the results are basically fair, and the images that most people like best end up in the higher spots. Can attempted manipulation knock an entry a few spots out of its rightful place? Probably. Can it turn a well liked entry into a bottom dweller or garbage into a winner? No.

What I really don't get is why anyone would try to influence the results. Regardless of that, the best way to negate the effect of manipulation is to input more ratings. Assuming that most people are honest, the bulk of their "true" ratings will squelch the squirrely ones.

Apologies for blathering. Regards, Brian

 Brian in Montana's gear list:Brian in Montana's gear list
Nikon D90 Nikon D7000 Nikon AF-S Nikkor 50mm f/1.4G Nikon AF Nikkor 28mm f/2.8D Nikon AF-S Micro-Nikkor 105mm f/2.8G IF-ED VR +19 more
Reply   Reply with quote   Complain
also el
Regular MemberPosts: 304
Like?
Re: I dont understand this voting system
In reply to Brian in Montana, 9 months ago

brian,  blather on!!   finally, someone who knows probability and statistics as well as handling a camera.   you were lucky to take part in a challenge that had a good number of members taking time to vote.    it is good that more hosts are asking for participation in voting by the entrants.   fine shot of the bird.

great utah shots in your gallery!

Reply   Reply with quote   Complain
Brian in Montana
Regular MemberPosts: 348Gear list
Like?
Re: I dont understand this voting system
In reply to also el, 9 months ago

Thank you for the compliments, also el. I'm no expert in statistics (but used it a lot in my career) or with a camera (but trying to get better) - I just hope to understand things I am involved in as best I can.

I'm not qualified to host a DPR Challenge at this point, and don't know if I would anyway because I think the onus is on the host to "police" it in a way that might eliminate biases where they exist (read lots of time and effort). That could mean reviewing the ratings in detail and perhaps using PMs to prod "outlying raters" for an explanation of their ratings. If there are attempted manipulations going on, it shouldn't be too hard to identify the source(s). Anyway, let's not lose sleep over it. If people become too disillusioned with the Challenges, then Challenges will be discontinued sooner or later due to lack of participation. Until then, the methodology for determining final rankings seems capable of damping out most of the unfortunate noise.

Regards, Brian

 Brian in Montana's gear list:Brian in Montana's gear list
Nikon D90 Nikon D7000 Nikon AF-S Nikkor 50mm f/1.4G Nikon AF Nikkor 28mm f/2.8D Nikon AF-S Micro-Nikkor 105mm f/2.8G IF-ED VR +19 more
Reply   Reply with quote   Complain
Keyboard shortcuts:
FForum MMy threads