Proposed Changes to Challenges: Community Discussion

Started Mar 15, 2012 | Discussions
CG33
Regular MemberPosts: 121Gear list
Like?
Re: Simple and effective fix - all that is needed.
In reply to RaptorUK, Apr 1, 2012

That's true.

Reply   Reply with quote   Complain
Safelight
New MemberPosts: 12
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to Scott Everett, Apr 3, 2012

Reasonable changes and enhancements. Here are some thoughts on a handful of the items:

Enable challenge entrants to vote on the challenges they enter, but not on their own entry

Why allow challenge entrants to vote at all? This invites bias.

Introduction of an additional challenge type that the host chooses when creating a new challenge: the host can choose the challenge winner (removing the public vote)

I would suggest then, allowing the host to pick the top 3 or so winners as in many cases several interpretations of a challenge topic produce equally outstanding entries.

Ability for users to fill in EXIF data only when it is not automatically identified by DPR, and only before voting

I propose that EXIF data should be unavailable till the voting is done. The date taken is fine but lens, aperture, shutter speed, ISO, etc. are irrelevant when judging an image's impact (unless the challenge is all about these technical aspects of course). Awarding a greater vote to an image because it was taken at f2 vs. f4 is meaningless. Once the voting is completed, then it is very interesting to learn these details - and any more the winner may wish to share.

A voting style option I'd like to see: allow the host to specify a "vote for favorite"-only vote. Users get one vote (no stars). The top 3 photos voted on, are the winners.

Reply   Reply with quote   Complain
BrianA2
Regular MemberPosts: 114Gear list
Like?
Focal length as seen during voting
In reply to Safelight, Apr 4, 2012

I think that the focal length displayed during voting should be 35mm equivalent. There is a temptation currently, when seeing a focal length of say 5.5mm to dismiss that in the basis it must have been taken with a P&S, and therefore is somehow inferior.

Given the current display of winners on the challenge page, there should never be more than six challenges finishing on any given day. The seventh and subsequent winners become invisible, unless deliberate action is taken.

 BrianA2's gear list:BrianA2's gear list
Minolta DiMAGE 7 Minolta DiMAGE A1 Konica Minolta DiMAGE A2 Canon EOS 40D Pentax K-r +11 more
Reply   Reply with quote   Complain
RaptorUK
Veteran MemberPosts: 3,266Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to Safelight, Apr 4, 2012

Safelight wrote:

Reasonable changes and enhancements. Here are some thoughts on a handful of the items:

Enable challenge entrants to vote on the challenges they enter, but not on their own entry

Why allow challenge entrants to vote at all? This invites bias.

It should be mandatory to vote in the Challenges you have entered . . . if that were the case you would have voted a lot more than you already have, more votes reduces the impact of sandbagging.

If you can't vote objectively don't enter the Challenges.

-- hide signature --

Take a look at my album . . . http://www.F1Album.com

 RaptorUK's gear list:RaptorUK's gear list
Fujifilm FinePix S2 Pro Nikon D1X Nikon D2X Nikon D3 Nikon AF-S Nikkor 70-200mm f/2.8G ED VR +7 more
Reply   Reply with quote   Complain
Mike Ronesia
Senior MemberPosts: 2,494Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to RaptorUK, Apr 4, 2012

Would you also have the gymnasts in the upcoming Summer Olympics do the judging in their events? No matter how you say it I just can't wrap my head around how it is appropriate for people to judge a competition they are entered in.

If you want more votes then make them vote in another challenge.
--
It's easier to ask for forgiveness then to ask for permission.

 Mike Ronesia's gear list:Mike Ronesia's gear list
Panasonic Lumix DMC-GF1 Panasonic Lumix DMC-GH2 Panasonic Lumix DMC-GH3 Panasonic Lumix DMC-GM1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS +9 more
Reply   Reply with quote   Complain
RaptorUK
Veteran MemberPosts: 3,266Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to Mike Ronesia, Apr 4, 2012

Mike Ronesia wrote:

Would you also have the gymnasts in the upcoming Summer Olympics do the judging in their events? No matter how you say it I just can't wrap my head around how it is appropriate for people to judge a competition they are entered in.

No I wouldn't, there is no way that can be made anonymous . . .

The people in the competition have a vested interest in that competition . . . it is in their interest for EVERYONE to vote fairly in that competition. People that can't bring themselves to be fair have no business voting in any Challenge and must be excluded.

If you want more votes then make them vote in another challenge.

In practical terms that isn't possible to do, I already asked how you would do it and got no response . . .

http://forums.dpreview.com/forums/read.asp?forum=1044&message=40933361

-- hide signature --

It's easier to ask for forgiveness then to ask for permission.

-- hide signature --

Take a look at my album . . . http://www.F1Album.com

 RaptorUK's gear list:RaptorUK's gear list
Fujifilm FinePix S2 Pro Nikon D1X Nikon D2X Nikon D3 Nikon AF-S Nikkor 70-200mm f/2.8G ED VR +7 more
Reply   Reply with quote   Complain
Mike Ronesia
Senior MemberPosts: 2,494Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to RaptorUK, Apr 4, 2012

RaptorUK wrote:

In practical terms that isn't possible to do, I already asked how you would do it and got no response . . .

For every 20 votes cast you can enter one photo into a challenge. Computer can do the tracking. If I've voted 127 times I can enter 6 photos. System will not allow me to vote in any challenge I've entered. These numbers are all tracked so it's just a matter of a little code.

It's not that hard.

 Mike Ronesia's gear list:Mike Ronesia's gear list
Panasonic Lumix DMC-GF1 Panasonic Lumix DMC-GH2 Panasonic Lumix DMC-GH3 Panasonic Lumix DMC-GM1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS +9 more
Reply   Reply with quote   Complain
RaptorUK
Veteran MemberPosts: 3,266Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to Mike Ronesia, Apr 4, 2012

Mike Ronesia wrote:

RaptorUK wrote:

In practical terms that isn't possible to do, I already asked how you would do it and got no response . . .

For every 20 votes cast you can enter one photo into a challenge. Computer can do the tracking. If I've voted 127 times I can enter 6 photos. System will not allow me to vote in any challenge I've entered. These numbers are all tracked so it's just a matter of a little code.

It's not that hard.

I think it will encourage poor voting . . . when people have to vote to be able to enter . . . and have to rush to do it. We need more quality not less.

-- hide signature --

Take a look at my album . . . http://www.F1Album.com

 RaptorUK's gear list:RaptorUK's gear list
Fujifilm FinePix S2 Pro Nikon D1X Nikon D2X Nikon D3 Nikon AF-S Nikkor 70-200mm f/2.8G ED VR +7 more
Reply   Reply with quote   Complain
Safelight
New MemberPosts: 12
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to RaptorUK, Apr 4, 2012

Why conflate the goal of more/better voting with the goal of fairness/minimize conflict of interest? Users who enter a challenge are inherently conflicted when voting on the same challenge. While I applaud the virtue of impartiality I can't expect it -- nor can I enforce it.

If the problem you're identifying is the volume or quality of voting, it should be evaluated outside of the merits of entrants voting on their challenges.

Reply   Reply with quote   Complain
Safelight
New MemberPosts: 12
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to Mike Ronesia, Apr 4, 2012

Sounds fair.

Reply   Reply with quote   Complain
Safelight
New MemberPosts: 12
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to Safelight, Apr 4, 2012

A voting style option I'd like to see: allow the host to specify a "vote for favorite"-only vote. Users get one vote (no stars). The top 3 photos voted on, are the winners.

I didn't articulate my motivation behind offering an alternative voting proposal. As someone relatively new to the site, though not new to voting, I see certain usability problems around the current voting mechanism that may lead to poor voting results. It’s especially evident in challenges with many entries. Going through all the entries and carefully deciding on the number of stars to award for, say, 100 entries will lead to a meeting with a carpal tunnel specialist in the not too distant future. And then you also have visual fatigue working against you. How many images can you view and offer a fair assessment on? While "judging" you may rush through the start of the slideshow because you have many images to get through but you may also abandon the end of it because you’re overloaded. They say in marketing surveys, keep it to the fewest questions possible if you want quality results. It’s not that different here. If I were asked to look through a body of images and pick 3 or 4 that I liked (with a grade or not), I’d be more successful than if I were asked to grade all.

Another thought I'd like to offer for those concerned about fairness: there are frequently many good images in the challenges that can't all win, nor even be in the top 3. Perhaps 2 or 3 randomly selected past winners or hosts from other challenges are invited to review challenges and award up to 3 honorable mentions (that do not necessarily need to be the top 3 images).

Reply   Reply with quote   Complain
Slynky
Senior MemberPosts: 2,343Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to RaptorUK, Apr 4, 2012

RaptorUK wrote:

Mike Ronesia wrote:

RaptorUK wrote:

In practical terms that isn't possible to do, I already asked how you would do it and got no response . . .

For every 20 votes cast you can enter one photo into a challenge. Computer can do the tracking. If I've voted 127 times I can enter 6 photos. System will not allow me to vote in any challenge I've entered. These numbers are all tracked so it's just a matter of a little code.

It's not that hard.

(a little editing... :D)

I think it will encourage poor voting . . . when people have to vote in their own challenge in order to be able to enter . . . and have to rush to do it. We need more quality not less.

Exactly !

What's the difference between required to vote in a challenge you entered or a challenge you haven't entered? My guess at the difference is you will get more dishonest votes for those required to vote in their own challenge.

-- hide signature --

Take a look at my album . . . http://www.F1Album.com

-- hide signature --

A veteran is someone who, at one point in their life, wrote a blank check made payable to "The United States of America," for an amount of "up to and including my life".

(...from the generation that still uses capital letters and punctuation...)

 Slynky's gear list:Slynky's gear list
Sony Alpha NEX-5N Sony E 18-55mm F3.5-5.6 OSS Sony E 16mm F2.8 Pancake Sony E 18-200mm F3.5-6.3 Sony E 50mm F1.8 OSS +3 more
Reply   Reply with quote   Complain
onlooker
Veteran MemberPosts: 3,749Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to RaptorUK, Apr 4, 2012

RaptorUK wrote:

Mike Ronesia wrote:

Would you also have the gymnasts in the upcoming Summer Olympics do the judging in their events? No matter how you say it I just can't wrap my head around how it is appropriate for people to judge a competition they are entered in.

No I wouldn't, there is no way that can be made anonymous . . .

You're not implying that if it could be made anonymous they would vote more objectively, are you?

it is in their interest for EVERYONE to vote fairly in that competition.

Actually, no, it isn't. It is in their interest that everyone EXCEPT them vote fairly. That gives them an advantage, and that's what we're seeing.

Reply   Reply with quote   Complain
RaptorUK
Veteran MemberPosts: 3,266Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to Slynky, Apr 4, 2012

Please don't do that . . . if you want to quote me then quote what I actually have written otherwise it isn't a quote . . .

When a person enters a Challenge and has to vote in that Challenge they have 7 days to do it . . . there is no rush, no panic, they know the rules, or at least have a better chance of knowing the rules, because they read them prior to entering. They have a stake in the Challenge as they are taking part in it, they have an interest in the subject matter because at some point they took a picture that may, at least partially, fit the theme.

These people are the best people to vote on the Challenge . . .

Dishonest people are dishonest people, they will be dishonest voters in Challenges they haven't entered just as easily as Challenges they have entered, the solution is to BAN them.

Slynky wrote:

RaptorUK DID NOT write:

Mike Ronesia wrote:

RaptorUK wrote:

In practical terms that isn't possible to do, I already asked how you would do it and got no response . . .

For every 20 votes cast you can enter one photo into a challenge. Computer can do the tracking. If I've voted 127 times I can enter 6 photos. System will not allow me to vote in any challenge I've entered. These numbers are all tracked so it's just a matter of a little code.

It's not that hard.

(a little editing... :D)

I think it will encourage poor voting . . . when people have to vote in their own challenge in order to be able to enter . . . and have to rush to do it. We need more quality not less.

Exactly !

What's the difference between required to vote in a challenge you entered or a challenge you haven't entered? My guess at the difference is you will get more dishonest votes for those required to vote in their own challenge.

-- hide signature --

Take a look at my album . . . http://www.F1Album.com

-- hide signature --

A veteran is someone who, at one point in their life, wrote a blank check made payable to "The United States of America," for an amount of "up to and including my life".

(...from the generation that still uses capital letters and punctuation...)

-- hide signature --

Take a look at my album . . . http://www.F1Album.com

 RaptorUK's gear list:RaptorUK's gear list
Fujifilm FinePix S2 Pro Nikon D1X Nikon D2X Nikon D3 Nikon AF-S Nikkor 70-200mm f/2.8G ED VR +7 more
Reply   Reply with quote   Complain
RaptorUK
Veteran MemberPosts: 3,266Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to onlooker, Apr 4, 2012

onlooker wrote:

RaptorUK wrote:

Mike Ronesia wrote:

Would you also have the gymnasts in the upcoming Summer Olympics do the judging in their events? No matter how you say it I just can't wrap my head around how it is appropriate for people to judge a competition they are entered in.

No I wouldn't, there is no way that can be made anonymous . . .

You're not implying that if it could be made anonymous they would vote more objectively, are you?

More objectively ? there is no "more" they currently have no vote or score from the contestants . . .

The discussion is irrelevant, how could one gymnast see another perform and not know who the other gymnast was ? but imagine some way it could be done . . . then yes I believe competitors scoring each other could be done objectively.

it is in their interest for EVERYONE to vote fairly in that competition.

Actually, no, it isn't. It is in their interest that everyone EXCEPT them vote fairly. That gives them an advantage, and that's what we're seeing.

Sorry you have it backwards . . . if it is in the interest of all the Entrants for all the Entrants to vote unfairly then what is the end result ?

The end result is that all the scores from all the Entrants will be 0.5 and will be a complete waste of time because they will all nullify each other . . . why would it be in the interest of all the Entrants to waste their time ?

If you are referring to one specific Entrant then you also have it backwards . . . if one particular Entrant votes unfairly he/she should be BANNED . . . this also is probably not in the interest of that particular Entrant.

Of course to effectively facilitate this Hosts must be able to see the real IDs of voters . . . and the DPReview general Challenge rules would need to be written/amended to explain that unfair voting (sandbagging) is prohibited and breaches may result in a BAN.

There are, no doubt, Sandbaggers, but not in all Challenges . . . I don't see any evidence of Sandbagging in the last Challenge I entered . . . and I voted in it as well.

http://www.dpreview.com/challenges/Challenge.aspx?ID=6398

-- hide signature --

Take a look at my album . . . http://www.F1Album.com

 RaptorUK's gear list:RaptorUK's gear list
Fujifilm FinePix S2 Pro Nikon D1X Nikon D2X Nikon D3 Nikon AF-S Nikkor 70-200mm f/2.8G ED VR +7 more
Reply   Reply with quote   Complain
onlooker
Veteran MemberPosts: 3,749Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to RaptorUK, Apr 4, 2012

RaptorUK wrote:

onlooker wrote:

it is in their interest for EVERYONE to vote fairly in that competition.

Actually, no, it isn't. It is in their interest that everyone EXCEPT them vote fairly. That gives them an advantage, and that's what we're seeing.

Sorry you have it backwards . . . if it is in the interest of all the Entrants for all the Entrants to vote unfairly then what is the end result ?

That's not what I said.

The end result is that all the scores from all the Entrants will be 0.5

It doesn't work that way in real life, and the cheaters know it well. Your argument operates in idealistic absolutes, they operate in reality. They are winning.

Reply   Reply with quote   Complain
RaptorUK
Veteran MemberPosts: 3,266Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to onlooker, Apr 5, 2012

onlooker wrote:

RaptorUK wrote:

onlooker wrote:

it is in their interest for EVERYONE to vote fairly in that competition.

Actually, no, it isn't. It is in their interest that everyone EXCEPT them vote fairly. That gives them an advantage, and that's what we're seeing.

Sorry you have it backwards . . . if it is in the interest of all the Entrants for all the Entrants to vote unfairly then what is the end result ?

That's not what I said.

That is what you said, see the text above, I haven't changed it . . .

The end result is that all the scores from all the Entrants will be 0.5

It doesn't work that way in real life, and the cheaters know it well. Your argument operates in idealistic absolutes, they operate in reality. They are winning.

I guess you didn't look at the link I posted ? Sandbagging is not prevalent in all Challenges.

Sandbaggers and Cheats can easily be dealt with . . . BAN them.

Part of the problem is that DPReview do not discourage Cheating, we have serial Cheats that are still allowed to enter Challenges, why ?

-- hide signature --

Take a look at my album . . . http://www.F1Album.com

 RaptorUK's gear list:RaptorUK's gear list
Fujifilm FinePix S2 Pro Nikon D1X Nikon D2X Nikon D3 Nikon AF-S Nikkor 70-200mm f/2.8G ED VR +7 more
Reply   Reply with quote   Complain
Joe Pineapples
Senior MemberPosts: 1,711
Like?
Agree with RaptorUK on this one...
In reply to RaptorUK, Apr 5, 2012

Rather than make the experience difficult for everyone, get tough on the cheats. From what I have seen, the majority have no trouble voting fairly in challenges, and I assume many voters are also entrants.

An arbitrary rule about "20 votes in any challenge per entry" or whatever just encourages click, click, click, ... click, all-done!

From what I've read, my favorite option so far is: if you enter a challenge then you must also cast a valid vote in that challenge. A valid vote is to rank your top 10 entries in order.

Joe

Reply   Reply with quote   Complain
Mike Ronesia
Senior MemberPosts: 2,494Gear list
Like?
Re: Agree with RaptorUK on this one...
In reply to Joe Pineapples, Apr 5, 2012

Joe Pineapples wrote:

From what I've read, my favorite option so far is: if you enter a challenge then you must also cast a valid vote in that challenge. A valid vote is to rank your top 10 entries in order.

Joe

So I can just pick the 10 I think are worst so mine does better?
--
It's easier to ask for forgiveness then to ask for permission.

 Mike Ronesia's gear list:Mike Ronesia's gear list
Panasonic Lumix DMC-GF1 Panasonic Lumix DMC-GH2 Panasonic Lumix DMC-GH3 Panasonic Lumix DMC-GM1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS +9 more
Reply   Reply with quote   Complain
RaptorUK
Veteran MemberPosts: 3,266Gear list
Like?
Re: Agree with RaptorUK on this one...
In reply to Mike Ronesia, Apr 5, 2012

Mike Ronesia wrote:

Joe Pineapples wrote:

From what I've read, my favorite option so far is: if you enter a challenge then you must also cast a valid vote in that challenge. A valid vote is to rank your top 10 entries in order.

Joe

So I can just pick the 10 I think are worst so mine does better?

Sure you can . . . and that makes it even easier to ID you as a Cheat . . . and then you get BANNED.

No single measure is going to fix all the issues . . . we need a joined up system to pull the Challenges back up to where they should be . . . thy have been neglected for so long, if DPR are serious they need to make a lot of effort.

-- hide signature --

It's easier to ask for forgiveness then to ask for permission.

-- hide signature --

Take a look at my album . . . http://www.F1Album.com

 RaptorUK's gear list:RaptorUK's gear list
Fujifilm FinePix S2 Pro Nikon D1X Nikon D2X Nikon D3 Nikon AF-S Nikkor 70-200mm f/2.8G ED VR +7 more
Reply   Reply with quote   Complain
Keyboard shortcuts:
FForum MMy threads