Proposed Changes to Challenges: Community Discussion

Started Mar 15, 2012 | Discussions
Scott Everett
dpreview AdminPosts: 938Gear list
Like?
Proposed Changes to Challenges: Community Discussion
Mar 15, 2012

Hey everyone,

Below is an outline of some proposed changes we are considering. Some of these are smaller modifications that have come up in the forums frequently, some are structural changes to hosting and voting intended to improve usability and effectiveness of challenges. Undoubtedly there will be ideas not covered here that have come up in the past, but we wanted to focus on what seemed to be the areas that most needed an update or overhaul:

Immediate Changes:

Voting Changes

  • Enable challenge entrants to vote on the challenges they enter, but not on their own entry

  • Introduce rollover help text for each voting level to display a description of what that score means (essentially, standardized scoring levels/meanings)

  • Introduce a button to allow voters to flag an entry as not fitting the challenge theme. After certain # of flags is reached, the entry is put on probation for the host to review.

  • Revise voting algorithm to account for partial voting and full voting by a single voter. In other words, votes cast by users that vote on entire set of images are given higher weighting than votes cast by users that vote on less or single entries.

Challenge Structure Changes

  • Introduction of an additional challenge type that the host chooses when creating a new challenge: the host can choose the challenge winner (removing the public vote)

  • Addition of optional challenge constraints on who can enter the challenge (selectable by challenge host): Minimum # of challenges entered

  • Addition of optional challenge constraint on who can vote on the challenge: Minimum # of challenges entered, and/or, minimum # of votes cast

  • Ability to limit challenge voting to contestants in the challenge

  • Introduction of monthly Dpr hosted challenges with giveaways (more info to come)

Challenge Host Features

  • Add the ability for a host to see a voter's vote spread (histogram), and the ability to disqualify from this interface (if there is suspicious voting). A single page would display all voters at once, displaying name, # of votes, histogram of distribution of votes.

  • Create a requirement for hosts to leave a comment on entries they disqualify (and make it visible to disqualified user)

General Changes

  • Ability for users to re-title their image and fix typos in image description (only before voting)

  • Ability for users to fill in EXIF data only when it is not automatically identified by DPR, and only before voting

  • Re-introduce challenge stats for users, with expanded view of data

  • Make photo description visible (and toggle-able) in slideshow view

  • Create a challenges search and explore page to enable users and visitors to scan challenge winners and contestants.

  • Enable users to "favorite" photos, and then see those photos in a single page.

  • Remove social buttons from voting pages (but keep on winners pages, and on challenge overview pages)

  • Introduction of a cap on the # of challenge slots available each week (to prevent too many chalenges from being created, and to hopefully increase activity in existing challenges)

Medium/long Term Changes:

  • creation of a new voting system that enables voters to view all entries in a new lightbox format, and then check off photos seen in full size that they wish to place in a top 5 or top 10 selection (photos marked for exclusion from top 10 would become slightly grayed out). Scoring would then be point driven, # of appearances in a top ten would determine # of points awarded. This format would de-emphasize the power of a single voter, but would likely require more voters to be effective. This format would also need new weighting algorithm to be created.

  • Creation of a challenge ideas voting board (akin to Digg, stackoverflow, etc..), allowing the community to propose challenge ideas, and then vote on which ideas get move into an active challenge first. In combination with the proposed cap on # of weekly challenges, the goal with his would be to ensure the most popular challenges ideas were running at all times.

  • introduction of some variation of user levels driven by activity (voting, challenge entries, hosting). Goal would be to recognize valuable members of the community. The scope and approach to this is still just a brainstorm on our side.

In terms of next steps, we'd like to begin working on some of the short term changes in the next few weeks. But before we do that, I'd like to get some feedback on what is outlined above, probably answer some questions, and boil down the list to what feels like the best mix of updates.

Please feel free to voice your support or concerns over any of the changes, there is no single solution and definitely no single correct way to do this. But my hope is that we can use this as an opportunity to correct some of the issues that have been going on, and also just improve the whole experience for everyone involved.

And lastly, it is important to note that these challenges are built for and ultimately driven by you, the community. So, anything we implement technically, we want to do so with an eye towards enabling the community to share great photography, inspire one another, and learn.

Thanks!

-Scott

 Scott Everett's gear list:Scott Everett's gear list
Canon EOS 5D Mark II Olympus E-1 Canon EF 16-35mm f/2.8L II USM +2 more
Mike Ronesia
Senior MemberPosts: 2,539Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to Scott Everett, Mar 15, 2012

Wow, you guys have been busy. That's a lot to think about. A lot of it looks good some I'll have to think about.

The first bullet is how it works now I think.
--
It's easier to ask for forgiveness then to ask for permission.

 Mike Ronesia's gear list:Mike Ronesia's gear list
Panasonic Lumix DMC-GF1 Panasonic Lumix DMC-GH2 Panasonic Lumix DMC-GH3 Panasonic Lumix DMC-GM1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS +9 more
Reply   Reply with quote   Complain
Slynky
Senior MemberPosts: 2,424Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to Scott Everett, Mar 15, 2012

Scott Everett wrote:

Hey everyone,

Below is an outline of some proposed changes we are considering. Some of these are smaller modifications that have come up in the forums frequently, some are structural changes to hosting and voting intended to improve usability and effectiveness of challenges. Undoubtedly there will be ideas not covered here that have come up in the past, but we wanted to focus on what seemed to be the areas that most needed an update or overhaul:

Immediate Changes:

Voting Changes

  • Enable challenge entrants to vote on the challenges they enter, but not on their own entry

I think this is already the case.

  • Introduce rollover help text for each voting level to display a description of what that score means (essentially, standardized scoring levels/meanings)

Sounds possibly helpful.

  • Revise voting algorithm to account for partial voting and full voting by a single voter. In other words, votes cast by users that vote on entire set of images are given higher weighting than votes cast by users that vote on less or single entries.

Looks promising.

Challenge Structure Changes

  • Introduction of an additional challenge type that the host chooses when creating a new challenge: the host can choose the challenge winner (removing the public vote)

  • Addition of optional challenge constraints on who can enter the challenge (selectable by challenge host): Minimum # of challenges entered

  • Addition of optional challenge constraint on who can vote on the challenge: Minimum # of challenges entered, and/or, minimum # of votes cast

  • Ability to limit challenge voting to contestants in the challenge

Looks to be some nice tools for the hosts. I will let those who have hosted speak to these, though.

  • Introduction of monthly Dpr hosted challenges with giveaways (more info to come)

Challenge Host Features

  • Add the ability for a host to see a voter's vote spread (histogram), and the ability to disqualify from this interface (if there is suspicious voting). A single page would display all voters at once, displaying name, # of votes, histogram of distribution of votes.

This sounds great. To clarify, do you mean disqualify an entry or that person's voting? (I'm assuming voting).

And when you say, name, you mean forum name and not that numerical digit that now shows?

  • Create a requirement for hosts to leave a comment on entries they disqualify (and make it visible to disqualified user)

General Changes

  • Ability for users to re-title their image and fix typos in image description (only before voting)

  • Ability for users to fill in EXIF data only when it is not automatically identified by DPR, and only before voting

  • Re-introduce challenge stats for users, with expanded view of data

Will this start over from scratch?

  • Make photo description visible (and toggle-able) in slideshow view

  • Create a challenges search and explore page to enable users and visitors to scan challenge winners and contestants.

  • Enable users to "favorite" photos, and then see those photos in a single page.

  • Remove social buttons from voting pages (but keep on winners pages, and on challenge overview pages)

Am assuming removal from anywhere that will enable a person to "pinpoint" his entry for social networks...

  • Introduction of a cap on the # of challenge slots available each week (to prevent too many chalenges from being created, and to hopefully increase activity in existing challenges)

With more info on how the back-up will be handled when the challenge bucket is full?

Medium/long Term Changes:

  • creation of a new voting system that enables voters to view all entries in a new lightbox format, and then check off photos seen in full size that they wish to place in a top 5 or top 10 selection (photos marked for exclusion from top 10 would become slightly grayed out). Scoring would then be point driven, # of appearances in a top ten would determine # of points awarded. This format would de-emphasize the power of a single voter, but would likely require more voters to be effective. This format would also need new weighting algorithm to be created.

But not place the images selected in any particular order...a photo will get its only score based on the number of times it was chosen in the top 10...?

  • Creation of a challenge ideas voting board (akin to Digg, stackoverflow, etc..), allowing the community to propose challenge ideas, and then vote on which ideas get move into an active challenge first. In combination with the proposed cap on # of weekly challenges, the goal with his would be to ensure the most popular challenges ideas were running at all times.

I see a partial answer to my question above.

  • introduction of some variation of user levels driven by activity (voting, challenge entries, hosting). Goal would be to recognize valuable members of the community. The scope and approach to this is still just a brainstorm on our side.

Sounds interesting. Would be nice to see some sort of acknowledgement of those people who spend time bettering the community by hosting.

And lastly, it is important to note that these challenges are built for and ultimately driven by you, the community. So, anything we implement technically, we want to do so with an eye towards enabling the community to share great photography, inspire one another, and learn.

I believe we appreciate the hand-in-hand approach. "Give us some tools and we'll make challenges work for DPR." Thanks for the attention.

Thanks!

-Scott

 Slynky's gear list:Slynky's gear list
Sony Alpha NEX-5N Sony E 18-55mm F3.5-5.6 OSS Sony E 16mm F2.8 Pancake Sony E 18-200mm F3.5-6.3 OSS Sony E 50mm F1.8 OSS +3 more
Reply   Reply with quote   Complain
Jan2009
Regular MemberPosts: 461Gear list
Like?
Re: Can I have fries with that please? :))
In reply to Slynky, Mar 15, 2012

WOW!! Amazing recommendations..

Its too much to digest right now and I am having a hard time how to break the system from the get go Just kidding..

But very impressive. I will give further comments once I have reviewed this in much details.

Thanks DPR and Scott and the rest who made this possible.

Some questions:
1. When is this changes gonna take effect?
2. Until when will you solicit feedback?

Initial Impressions:

Love it that host has much more power now (I was never a host:) but mainly they can identify the cheaters! and regulate their challenges.

I like the return of statistics, the toggle button on description and much more. Promise I will give some more feedback

 Jan2009's gear list:Jan2009's gear list
Canon PowerShot D10 Canon EOS 40D Canon EOS 7D Canon EF 50mm f/1.8 II Canon EF-S 10-22mm f/3.5-4.5 USM +2 more
Reply   Reply with quote   Complain
Scott Everett
dpreview AdminPosts: 938Gear list
Like?
Thread will remain open through the weekend...
In reply to Scott Everett, Mar 15, 2012

I want to allow enough time for everyone to read through the changes, voice their opinions, and of course provide some further context and answers to questions.

-s

 Scott Everett's gear list:Scott Everett's gear list
Canon EOS 5D Mark II Olympus E-1 Canon EF 16-35mm f/2.8L II USM +2 more
Reply   Reply with quote   Complain
Mike Ronesia
Senior MemberPosts: 2,539Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to Scott Everett, Mar 15, 2012

Scott, how hard would it be to set it up so that the site could resize oversized photos to fit in challenges with size rules? Maybe have some pre-configured sizes like, 1600:900 or max height 900? These fit most screens and in most challenges is all that is needed. Will help keep the bandwidth down as well.

Scott Everett wrote:

Immediate Changes:

Voting Changes

  • Enable challenge entrants to vote on the challenges they enter, but not on their own entry

Current system and this is what promotes sandbagging. Some will drive down some other/all other photos to boost there own.

  • Introduce rollover help text for each voting level to display a description of what that score means (essentially, standardized scoring levels/meanings)

Might help, surely won't hurt and easy to do.

  • Introduce a button to allow voters to flag an entry as not fitting the challenge theme. After certain # of flags is reached, the entry is put on probation for the host to review.

Good idea. I would keep the number fairly low like 2 or 3 with the host having the ability to override with maybe a checked box that says "host approved". This box could be on all photos that have been reviewed and approved by the host. This would let people know the challenge is being monitored.

  • Revise voting algorithm to account for partial voting and full voting by a single voter. In other words, votes cast by users that vote on entire set of images are given higher weighting than votes cast by users that vote on less or single entries.

Can't hurt.

Challenge Structure Changes

  • Introduction of an additional challenge type that the host chooses when creating a new challenge: the host can choose the challenge winner (removing the public vote)

I like this with limited use and maybe host have to meet some criteria.

  • Addition of optional challenge constraints on who can enter the challenge (selectable by challenge host): Minimum # of challenges entered

Good idea, I'd like to see as many options for this as possible. Something like 2 top ten finishes or I can see even having one for people that have no top 10 finishes. We would need to mix it up so everyone has challenges they can enter.

  • Addition of optional challenge constraint on who can vote on the challenge: Minimum # of challenges entered, and/or, minimum # of votes cast

Like, but with some constraints.

  • Ability to limit challenge voting to contestants in the challenge

EDIT, I reread this. I can see some using it, I'm not a personal fan as I like to vote but I never vote in challenges I enter.

  • Introduction of monthly Dpr hosted challenges with giveaways (more info to come)

Like

Challenge Host Features

  • Add the ability for a host to see a voter's vote spread (histogram), and the ability to disqualify from this interface (if there is suspicious voting). A single page would display all voters at once, displaying name, # of votes, histogram of distribution of votes.

Yes, please. May require DPR approval or some second level check just to make sure we are doing the right thing.

  • Create a requirement for hosts to leave a comment on entries they disqualify (and make it visible to disqualified user)

I already do this but I guess it's not required. Sounds simple enough to make it so you can't DQ without putting something in the box.

General Changes

  • Ability for users to re-title their image and fix typos in image description (only before voting)

Please! Most of my withdraws are because of typos or something correctable.

  • Ability for users to fill in EXIF data only when it is not automatically identified by DPR, and only before voting

No editing will help with capture date challenges. Won't stop the hard core cheaters but will stop some of it.

  • Re-introduce challenge stats for users, with expanded view of data

Mixed emotions on this.

  • Make photo description visible (and toggle-able) in slideshow view

Yes

  • Create a challenges search and explore page to enable users and visitors to scan challenge winners and contestants.

Yes

  • Enable users to "favorite" photos, and then see those photos in a single page.

Yes

  • Remove social buttons from voting pages (but keep on winners pages, and on challenge overview pages)

Yes

  • Introduction of a cap on the # of challenge slots available each week (to prevent too many chalenges from being created, and to hopefully increase activity in existing challenges)

Not sure on this. You have a lot of activity and a lot or variety. I like the ability to go through the open challenges and pick something I like. Sometimes it popular with a lot of entries and sometimes it's more obscure with only a few, but there are plenty to chose from and if a popular one fills up quickly there will always be another. I'd rather see bad hosts and or unpopular themes be changed.

 Mike Ronesia's gear list:Mike Ronesia's gear list
Panasonic Lumix DMC-GF1 Panasonic Lumix DMC-GH2 Panasonic Lumix DMC-GH3 Panasonic Lumix DMC-GM1 Panasonic Lumix G Vario 14-45mm F3.5-5.6 ASPH OIS +9 more
Reply   Reply with quote   Complain
chrisby
Regular MemberPosts: 219Gear list
Like?
Re: Thread will remain open through the weekend...
In reply to Scott Everett, Mar 15, 2012

Wouldn't it be helpful to open threads with the bullet points or at least to cluster some bullit points with the appropriate subject line?
It would make discussion on particular items much easier.

 chrisby's gear list:chrisby's gear list
Canon PowerShot S90 Canon PowerShot S95 Panasonic Lumix DMC-TS3 Canon PowerShot G1 X Canon EOS 40D +4 more
Reply   Reply with quote   Complain
libertylady
Regular MemberPosts: 176
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to Mike Ronesia, Mar 15, 2012

Good work on thinking over the problems and advantages of the challenges Scott. Thanks so much.

I wasn't clear if the facebook/social networking button would only be removed during the voting. I assume it would be removed UNTIL the challenge winners have been announced. Otherwise the problem of solicitation would not be addressed completely.

Yeah on bringing back the stats. I will have to be more diligent in my entries to bring that up. People that don't want to know, don't have to look!

I am not fond of the idea of only challenge entrants to vote in that challenge unless it is specialized like the challenges where everyone comments and then corrections are allowed with the second entry. I am unclear if it is a good idea to even vote in challenge you have entered or if it might result in prejudiced voting. I am just not sure. But I do think that probably should remain as it is now in order to get enough voting.

I haven't thought of the idea of limiting the challengers to particular group but I do like that idea if there is some variety. The idea of limiting it to only those who have never come in the top 10 or those who have come in the top 10 , for example, sounds really interesting. Variety sounds really good to me. (Not all the time, just occasionally I would think). Already we seem to have the ability to limit it to those who have posted so many times to forums or to those that have voted a certain number of times. This seems helpful.

Good ideas. It will be interesting to see what everyone says about the suggestions. It feels good to know DPR is paying attention and helping us out. Thanks.

Reply   Reply with quote   Complain
Jan2009
Regular MemberPosts: 461Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to Scott Everett, Mar 15, 2012

Hi Scott,

I will put LOW , MEDIUM and HOT towards what I think needs to be implemented on the challenges.

Voting Changes

  • Introduce rollover help text for each voting level to display a description of what that score means (essentially, standardized scoring levels/meanings)

LOW: . Good for new voters.

  • Introduce a button to allow voters to flag an entry as not fitting the challenge theme. After certain # of flags is reached, the entry is put on probation for the host to review.

MEDIUM: . SMART suggestion! And hopefully disqualified image will not affect entrants statistics (as before). Also this gives immediate feedback to member to submit image wisely. Flag should also require comments from members to give opinion why they think it did not fit the challenge.

Maybe submitting member need enough time to know his image is under probation, We need to make sure he has enough time to re-submit a new one.

  • Revise voting algorithm to account for partial voting and full voting by a single voter. In other words, votes cast by users that vote on entire set of images are given higher weighting than votes cast by users that vote on less or single entries.

HOT : WONDERFUL suggestion! This is my favorite in the Voting Changes. lets move on.

Challenge Structure Changes

  • Introduction of an additional challenge type that the host chooses when creating a new challenge: the host can choose the challenge winner (removing the public vote)

MEDIUM: ...hmm I wonder how many hosts have I befriended or angered I just wish host will be able to give feedback on why they pick the winner. +1 on this new feature.

  • Addition of optional challenge constraints on who can enter the challenge (selectable by challenge host): Minimum # of challenges entered

MEDIUM: ..Wow, I like this by invitation only makes you feel special.

  • Addition of optional challenge constraint on who can vote on the challenge: Minimum # of challenges entered, and/or, minimum # of votes cast

HOT This one I really like! This is a challenge for those long time Challenge participants.

  • Ability to limit challenge voting to contestants in the challenge

MEDIUM ..This should be the reverse, I think Contestants SHOULD be required to VOTE to enter the Challenge, but I don't know how to implement that.

  • Introduction of monthly Dpr hosted challenges with giveaways (more info to come)

HOT If the giveaway is 5DMIII or Nikon D800 oh man the Tamron challenge may get refreshed so thread carefully. But I recommend giving the new CHallenge format some time and see if tweaks work.

Challenge Host Features

  • Add the ability for a host to see a voter's vote spread (histogram), and the ability to disqualify from this interface (if there is suspicious voting). A single page would display all voters at once, displaying name, # of votes, histogram of distribution of votes.

HOT! HOT! HOT! BEST FEATURE for sure ! And hope challenge host can share them so we can laugh at the cheater. I hope he/she is not in this forum ;).

  • Create a requirement for hosts to leave a comment on entries they disqualify (and make it visible to disqualified user)

MEDIUM: GOOD thanks! I think adding members comment will help as suggested above.

General Changes

  • Ability for users to re-title their image and fix typos in image description (only before voting)

MEDIUM: Good. Now we don't have to Withdraw our image and resubmit again for wrong title.

  • Re-introduce challenge stats for users, with expanded view of data

HOT! I really hope they can add new feature like VOTING ACCURACY on the stat, like how accurate the members picked the TOP 3 or 5. It means we are training our eye on spotting the best images and composition.

  • Make photo description visible (and toggle-able) in slideshow view

MEDIUM I like this also, because sometimes I want to understand the reason behind a snapshot before I give a low vote.

  • Create a challenges search and explore page to enable users and visitors to scan challenge winners and contestants.

LOW: Need to see how this turns out.

  • Enable users to "favorite" photos, and then see those photos in a single page.

MEDIUM: Nice. or maybe Add favorite Photographers too.

  • Remove social buttons from voting pages (but keep on winners pages, and on challenge overview pages)

HOT! HOT! YEEEESSSSSS!!! So long Facebook fan votes.

  • Introduction of a cap on the # of challenge slots available each week (to prevent too many chalenges from being created, and to hopefully increase activity in existing challenges)

MEDIUM: I just hope there will be enough to join and selection is wide enough and not (just for example Flower challenge on Fridays

Medium/long Term Changes:

  • Creation of a challenge ideas voting board (akin to Digg, stackoverflow, etc..), allowing the community to propose challenge ideas, and then vote on which ideas get move into an active challenge first. In combination with the proposed cap on # of weekly challenges, the goal with his would be to ensure the most popular challenges ideas were running at all times.

MEDIUM: VERY Interesting! I like it.

  • introduction of some variation of user levels driven by activity (voting, challenge entries, hosting). Goal would be to recognize valuable members of the community. The scope and approach to this is still just a brainstorm on our side.

MEDIUM: I'm keeping an open mind on this

 Jan2009's gear list:Jan2009's gear list
Canon PowerShot D10 Canon EOS 40D Canon EOS 7D Canon EF 50mm f/1.8 II Canon EF-S 10-22mm f/3.5-4.5 USM +2 more
Reply   Reply with quote   Complain
Doug Pardee
Veteran MemberPosts: 7,439Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to Scott Everett, Mar 15, 2012

[Note: I'm a relatively new participant, and have not yet been a host.]

Voting Changes

  • Introduce rollover help text for each voting level

That would be helpful.

  • Introduce a button to allow voters to flag an entry as not fitting the challenge theme.

Agree, along with the suggested modification for marking an entry as having been approved by the host.

Challenge Structure Changes

  • Introduction of an additional challenge type that the host chooses when creating a new challenge: the host can choose the challenge winner (removing the public vote)

Suggested added variation: the host can select finalists that will move on to normal public voting. This is something that barb_s has been experimenting with.

  • Addition of optional challenge constraints on who can enter the challenge (selectable by challenge host): Minimum # of challenges entered

If too many challenges use this, it's going to be very difficult for new entrants to get started.

  • Addition of optional challenge constraint on who can vote on the challenge: ... minimum # of votes cast

If too many challenges use this, it's going to be very difficult to get new voters.

  • Introduction of monthly Dpr hosted challenges with giveaways

This one spooks the heck out of me. The cheating's bad enough when there's no prize on the line.

Challenge Host Features

  • Add the ability for a host to see a voter's vote spread (histogram), and the ability to disqualify from this interface

This is just within a single challenge? How about a way to see that a given voter has a history of being a problem in other challenges?

Addition

  • Add the ability for the host to allow user comments to be made to entries, but the comments will not be visible until after voting is complete. Even better if the user can see his/her own comments ahead of time, so they remember what they wrote.

General Changes

  • Ability for users to re-title their image and fix typos in image description

Oh yes, please.

  • Make photo description visible (and toggle-able) in slideshow view

Yes.

Addition

  • Disallow withdrawing from a challenge once voting has begun. Or finished.

Medium/long Term Changes:

  • creation of a new voting system that enables voters to view all entries in a new lightbox format, and then check off photos seen in full size that they wish to place in a top 5 or top 10 selection (photos marked for exclusion from top 10 would become slightly grayed out). Scoring would then be point driven, # of appearances in a top ten would determine # of points awarded. This format would de-emphasize the power of a single voter, but would likely require more voters to be effective. This format would also need new weighting algorithm to be created.

The trick would be the weighting algorithm. I've noticed that barb_s was experimenting with various approaches in her challenges that used a panel of judges to select the finalists; I don't know if she's yet come to a conclusion about which approach worked best.

  • Creation of a challenge ideas voting board

I'd think this would slow the challenge process down even farther than it already is.

-- hide signature --

The LightZombie Project: volunteers providing support for LightZone. http://lightzombie.org/

 Doug Pardee's gear list:Doug Pardee's gear list
Olympus Stylus 1030 SW Canon EOS 350D Canon EF-S 17-85mm f/4-5.6 IS USM Adobe Photoshop Elements 10 Snapseed +20 more
Reply   Reply with quote   Complain
Deleted pending purge
Senior MemberPosts: 1,197Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to Doug Pardee, Mar 16, 2012

Nice propositions, at a first glance. I guess I'll re-read it slowly in the morning before trying to evaluate. But so far, it looks promising. Thanks!
--
Fil

Reply   Reply with quote   Complain
Slynky
Senior MemberPosts: 2,424Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to Doug Pardee, Mar 16, 2012

Doug Pardee wrote:
[Note: I'm a relatively new participant, and have not yet been a host.]

Voting Changes

  • Introduce rollover help text for each voting level

That would be helpful.

  • Introduce a button to allow voters to flag an entry as not fitting the challenge theme.

Agree, along with the suggested modification for marking an entry as having been approved by the host.

Challenge Structure Changes

  • Introduction of an additional challenge type that the host chooses when creating a new challenge: the host can choose the challenge winner (removing the public vote)

Suggested added variation: the host can select finalists that will move on to normal public voting. This is something that barb_s has been experimenting with.

  • Addition of optional challenge constraints on who can enter the challenge (selectable by challenge host): Minimum # of challenges entered

If too many challenges use this, it's going to be very difficult for new entrants to get started.

  • Addition of optional challenge constraint on who can vote on the challenge: ... minimum # of votes cast

If too many challenges use this, it's going to be very difficult to get new voters.

  • Introduction of monthly Dpr hosted challenges with giveaways

This one spooks the heck out of me. The cheating's bad enough when there's no prize on the line.

Challenge Host Features

  • Add the ability for a host to see a voter's vote spread (histogram), and the ability to disqualify from this interface

This is just within a single challenge? How about a way to see that a given voter has a history of being a problem in other challenges?

Addition

  • Add the ability for the host to allow user comments to be made to entries, but the comments will not be visible until after voting is complete. Even better if the user can see his/her own comments ahead of time, so they remember what they wrote.

I like this idea from Doug.

General Changes

  • Ability for users to re-title their image and fix typos in image description

Oh yes, please.

  • Make photo description visible (and toggle-able) in slideshow view

Yes.

Addition

  • Disallow withdrawing from a challenge once voting has begun. Or finished.

I like this idea, too.

Medium/long Term Changes:

  • creation of a new voting system that enables voters to view all entries in a new lightbox format, and then check off photos seen in full size that they wish to place in a top 5 or top 10 selection (photos marked for exclusion from top 10 would become slightly grayed out). Scoring would then be point driven, # of appearances in a top ten would determine # of points awarded. This format would de-emphasize the power of a single voter, but would likely require more voters to be effective. This format would also need new weighting algorithm to be created.

The trick would be the weighting algorithm. I've noticed that barb_s was experimenting with various approaches in her challenges that used a panel of judges to select the finalists; I don't know if she's yet come to a conclusion about which approach worked best.

  • Creation of a challenge ideas voting board

I'd think this would slow the challenge process down even farther than it already is.

-- hide signature --

The LightZombie Project: volunteers providing support for LightZone. http://lightzombie.org/

-- hide signature --

A veteran is someone who, at one point in their life, wrote a blank check made payable to "The United States of America," for an amount of "up to and including my life".

(...from the generation that still uses capital letters and punctuation...)

 Slynky's gear list:Slynky's gear list
Sony Alpha NEX-5N Sony E 18-55mm F3.5-5.6 OSS Sony E 16mm F2.8 Pancake Sony E 18-200mm F3.5-6.3 OSS Sony E 50mm F1.8 OSS +3 more
Reply   Reply with quote   Complain
jezsik
Senior MemberPosts: 2,565
Like?
Really good news!
In reply to Scott Everett, Mar 16, 2012

You've got some great ideas and I look forward to seeing how they affect the challenges.

My only comment is that disqualified images should be visible to everyone. This way, people can see what images are not acceptable.

Reply   Reply with quote   Complain
RossAndrew
Senior MemberPosts: 3,073
Like?
Re: Proposed Changes to Challenges: Community Discussion Or May Be
In reply to Scott Everett, Mar 16, 2012

Challenge structure change.

(1) If the host is the only one to choose the winner as in your first suggestion, won't this just bring out all the megalomaniacs as hosts. ( after reading some of the forums, there are more then a few ).

(2) If you start to restricting entry's and voters on number of the minimum times they have either entered or voted, surly this will reduce numbers when we need to increase them.

(3) I don't believe that any one that enters a challenge should have the right to vote in that challenge at all. I have never, and thought that is how was meant to be.People will vote lower scores on the better photos to lift there chances of winning.

(4) I think a lot of your voting changers are getting a bit to technical and will probably tend to drive a lot of Luddites like myself away from voting, and then we won't be able to meet the minimum number of votes to enter any challengers.

(5) Can I suggest.

That a pool of say 100 voters is drawn form all members across the site ( this would be part of your duty as a member to take part) and that the pool votes on 5 challengers in a 4 day period, then the next pool of 100 would then vote on the next 5 challengers over the next 4 day period. All voters would be anonymous. The 5 challengers for that week would all end on the same day and 4 days latter the winners would be announced. this would also bring a lot of talented people into the voting side of the site, this would hopefully lift the standard of work that is entered. There may be some bending and trimming of this idea to fit, this is only meant to be a mud map but worth a few seconds thought. I think that it must be kept very easy and simple.

Reply   Reply with quote   Complain
jrtrent
Veteran MemberPosts: 3,225
Like?
some good, and some very chilling, ideas
In reply to Scott Everett, Mar 16, 2012

Immediate Changes:

Voting Changes

  • Introduce rollover help text for each voting level to display a description of what that score means (essentially, standardized scoring levels/meanings)

Excellent idea.

  • Revise voting algorithm to account for partial voting and full voting by a single voter. In other words, votes cast by users that vote on entire set of images are given higher weighting than votes cast by users that vote on less or single entries.

Horrible idea: There have been some threads or posts advocating that voters vote on every image, and I've done that on occasion, but I really don't like giving low votes, so my preference is to vote on relatively few images per challenge. It would be interesting to know what percentage of voters currently vote on every image, and I wonder how many of those that don't will simply stop participating once their votes are devalued. What happened to this idea from the Challenges Help article:

"We chose a rating system over a ranking system because it frees you (the judges) from having to judge every image. Voting on only a subset of images in a challenge has no negative impact on the fairness of final result so take it easy."

Challenge Structure Changes

  • Introduction of an additional challenge type that the host chooses when creating a new challenge: the host can choose the challenge winner (removing the public vote)

Totally removes interest in the challenge, at least from those not entering.

  • Addition of optional challenge constraints on who can enter the challenge (selectable by challenge host): Minimum # of challenges entered

Some hosts have done this already, but I'm not sure why limiting the number of possible entrants is a good thing.

  • Addition of optional challenge constraint on who can vote on the challenge: Minimum # of challenges entered, and/or, minimum # of votes cast

Again, would seem to reduce rather than build interest in the challenges.

  • Ability to limit challenge voting to contestants in the challenge

Another good way to make sure only a limited number of visitors to the site will have any interest in the challenge.

Challenge Host Features

  • Add the ability for a host to see a voter's vote spread (histogram), and the ability to disqualify from this interface (if there is suspicious voting). A single page would display all voters at once, displaying name, # of votes, histogram of distribution of votes.

Worst idea I've ever heard, and a very strange notion of voting. In effect, if the host doesn't agree with a particular voter's choices, those votes are simply tossed out. Hard to see why anyone would have an interest in participating under such a scheme.

General Changes

  • Make photo description visible (and toggle-able) in slideshow view

Another excellent idea!

  • Create a challenges search and explore page to enable users and visitors to scan challenge winners and contestants.

That would be fun.

  • Enable users to "favorite" photos, and then see those photos in a single page.

Yes.

Medium/long Term Changes:

  • creation of a new voting system that enables voters to view all entries in a new lightbox format, and then check off photos seen in full size that they wish to place in a top 5 or top 10 selection (photos marked for exclusion from top 10 would become slightly grayed out). Scoring would then be point driven, # of appearances in a top ten would determine # of points awarded. This format would de-emphasize the power of a single voter, but would likely require more voters to be effective. This format would also need new weighting algorithm to be created.

I love this idea. Every selection is one the voter would place in a high-ranking position--no negative votes.

Reply   Reply with quote   Complain
Slynky
Senior MemberPosts: 2,424Gear list
Like?
Not Actually Related to Voting Fraud Problems but...
In reply to Scott Everett, Mar 16, 2012

...while you're at it, there has been discussion about whether or not images submitted to a challenge should be visible during the submission stage (with both some pros and cons on that topic). I, for one, have had what I thought were good ideas and submitted an image only to see somewhat of a copycat image be submitted afterwards (though it's difficult to prove I was copied, of course).

Additionally, sometimes people avoid a challenge when they can see ahead of time there are a lot of images that already will likely beat them out.

Other contests at other sites use this concept.

But, giving the host a switch to turn on or off "blind submissions" should be a consideration.
--

A veteran is someone who, at one point in their life, wrote a blank check made payable to "The United States of America," for an amount of "up to and including my life".

(...from the generation that still uses capital letters and punctuation...)

 Slynky's gear list:Slynky's gear list
Sony Alpha NEX-5N Sony E 18-55mm F3.5-5.6 OSS Sony E 16mm F2.8 Pancake Sony E 18-200mm F3.5-6.3 OSS Sony E 50mm F1.8 OSS +3 more
Reply   Reply with quote   Complain
outer segment
New MemberPosts: 9
Like?
Re: Proposed Changes to Challenges: Community Discussion
In reply to Scott Everett, Mar 16, 2012

Hello Scott,

Thank you very much for sharing your ideas/plans. Please see my comments below.

Best wishes,

outer segment

Immediate Changes:

Voting Changes

  • Revise voting algorithm to account for partial voting and full voting by a single voter. In other words, votes cast by users that vote on entire set of images are given higher weighting than votes cast by users that vote on less or single entries.

Fantastic idea! Helps minimize the effect of solicited votes! And it encourages voting.

Please, do even more to encourage people who enter challenges to cast votes. For example, in an instructions page, or in the text that appears before one submits an entry, state explicitly that it is a matter of fairness to judge the images of others, if you want them to judge your image. More votes per entry means less effect of sandbaggers, vote solicitors and double account owners. And, it means the final rankings are less the product of chance and more reflect the true “popular vote”.

A rule of thumb could be: cast as many votes as you would like to get yourself on your entered image (I consider 25 votes and above per challenge entry as reasonable…).

Challenge Structure Changes

  • Introduction of an additional challenge type that the host chooses when creating a new challenge: the host can choose the challenge winner (removing the public vote)

I, personally, would not enter DPR challenges just to be judged by one single person. Popular vote is what I’m after…!

Here’s what I really wish for : Introduction of an additional challenge type that the host chooses when creating a new challenge:

Mandatory Voting Challenge – every challenge entrant has to vote on all other images submitted to the challenge, or her/his entry is disqualified. That way, the entrants can be sure to receive lots of votes (i.e. 49 in a 50 entries challenge). Thus they can be sure to get back a statistically more meaningful result, with a minimized impact of sandbaggers etc. Potential bias of voters who have an own entry in the challenge would cancel each other out.

  • Addition of optional challenge constraints on who can enter the challenge (selectable by challenge host): Minimum # of challenges entered

  • Addition of optional challenge constraint on who can vote on the challenge: Minimum # of challenges entered, and/or, minimum # of votes cast

I like both above points and I think, they could greatly improve the respective challenges’ quality (…since it would make life harder for some sorts of cheaters).

  • Ability to limit challenge voting to contestants in the challenge

Maybe, this is not so useful. It seems like not enough people vote in the challenges they enter. If such option is introduced, this should then be combined with mandatory voting for those who enter (see above).

Challenge Host Features

  • Add the ability for a host to see a voter's vote spread (histogram), and the ability to disqualify from this interface (if there is suspicious voting). A single page would display all voters at once, displaying name, # of votes, histogram of distribution of votes.

Sounds good, BUT…

  • The ability of hosts to disqualify votes should be limited to after the voting phase ends. That’s because, during voting phase my vote distribution might occasionally look strange for one or two days, since I sometimes make a very rough first ranking and, later, I do a refined ranking.

  • Second, more importantly, as a voter I’d have to know when a host disqualifies my votes. Otherwise, I’d always have the uncertain feeling “Who knows. Maybe the host did not appreciate my votes and my votes are not reflected in the final results. Maybe I wasted my time voting and I don’t even know.” That would be demotivating voters.

General Changes

  • Re-introduce challenge stats for users, with expanded view of data

YEAH! It was always a motivation for me to see my (low) overall score…

  • Make photo description visible (and toggle-able) in slideshow view

Yes, please. Toggle-able.

  • Remove social buttons from voting pages (but keep on winners pages, and on challenge overview pages)

Yes, that would be really important. Don’t make it too easy to solicit votes. Make it easy for winners to brag on social NWs, if you will…

  • Introduction of a cap on the # of challenge slots available each week (to prevent too many challenges from being created, and to hopefully increase activity in existing challenges)

It is not obvious to me that the number of challenges is a problem. Isn’t it rather a problem that too many people submit images but are not willing to grapple with the other’s photos and vote on them? Or is that what you are getting at, anyway – concentrate the current number of votes on less images?

Medium/long Term Changes:

  • creation of a new voting system that enables voters to view all entries in a new lightbox format, and then check off photos seen in full size that they wish to place in a top 5 or top 10 selection (photos marked for exclusion from top 10 would become slightly grayed out). Scoring would then be point driven, # of appearances in a top ten would determine # of points awarded. This format would de-emphasize the power of a single voter, but would likely require more voters to be effective. This format would also need new weighting algorithm to be created.

Sounds good, in principle. IMO, the main question for this proposal (and also the current system) is: How to get more voters to be effective?

Reply   Reply with quote   Complain
outer segment
New MemberPosts: 9
Like?
Re: some good, and some very chilling, ideas
In reply to jrtrent, Mar 16, 2012

(...) I really don't like giving low votes, so my preference is to vote on relatively few images per challenge.

If you don't give out low scores on images you don't like and you wait for others to do the "dirty work" then you should not be surprised if these images end up top-ranked, ahead of those you think deserve the good ranks ...

Reply   Reply with quote   Complain
Ed at Ridersite
Forum ProPosts: 15,371
Like?
Re: Proposed Changes to Challenges: Community Discussion Or May Be
In reply to RossAndrew, Mar 16, 2012

RossAndrew wrote:

(1) If the host is the only one to choose the winner as in your first suggestion, won't this just bring out all the megalomaniacs as hosts.

I agree. Furthermore, a host is perfectly free to pick their choices and post them in a thread here any time they want. Some hosts already do that.
--
AEH
http://aehass.zenfolio.com/
Question: What do you do all week?
Answer: Mon to Fri. Nothing, Sat & Sun I rest!

Reply   Reply with quote   Complain
Slynky
Senior MemberPosts: 2,424Gear list
Like?
Re: Proposed Changes to Challenges: Community Discussion Or May Be
In reply to Ed at Ridersite, Mar 16, 2012

I can see the concern. I would add:

IF some hosts acted like that, then it wouldn't take long before their challenges were shunned. So, a sort of governmental "checks and balances" system is at play here.

Also, it might be a checkbox that a host turns on (or off) for whatever reason and then that info is shared with the people who consider entering or not...that this is one of those challenges where if the voting looks to be "screwed up", then the host will override the decision.

Ed at Ridersite wrote:

RossAndrew wrote:

(1) If the host is the only one to choose the winner as in your first suggestion, won't this just bring out all the megalomaniacs as hosts.

I agree. Furthermore, a host is perfectly free to pick their choices and post them in a thread here any time they want. Some hosts already do that.
--
AEH
http://aehass.zenfolio.com/
Question: What do you do all week?
Answer: Mon to Fri. Nothing, Sat & Sun I rest!

-- hide signature --

A veteran is someone who, at one point in their life, wrote a blank check made payable to "The United States of America," for an amount of "up to and including my life".

(...from the generation that still uses capital letters and punctuation...)

 Slynky's gear list:Slynky's gear list
Sony Alpha NEX-5N Sony E 18-55mm F3.5-5.6 OSS Sony E 16mm F2.8 Pancake Sony E 18-200mm F3.5-6.3 OSS Sony E 50mm F1.8 OSS +3 more
Reply   Reply with quote   Complain
Keyboard shortcuts:
FForum MMy threads