It's our (the entrants) fault that the challenges don't work well.

Very well said At first, I thought exactly like you. So I voted mostly in challenges that I would have liked to enter, but did not (since I did not have an image that was suitable or, by any means, good enough).

However, after (quite) a while I thought I knew my voting behavior well enough, and I was confident to do a similar judgement in challenges where I have an iron in the fire. Actually, now I think, one can in fact come to a better judgement of the peers' entries in a challenge after first putting one's mind to the challenge task and shoot/select & submit an own image.