who needs judges?

like I said, I’ll consider everything. I wish I could continue this convo but a good friend of mine had a family member die sooo I’ll chat up with you guys later :wink:

thanks for all input!

I agree(!) with jbanes. You people need to post slower. :wink:

Since voting is a damn complex topic (I mean it), I asked someone with lots of knowledge in that area.


If you consider more than two options the lesser_evil spoiler problem arises with single mark voting (aka Plurality or First Past The Post).

This problem does NOT get reduced at all with giving people several points for free allocation (aka Cumulative – normally only recommended for multiwinner methods and even then it sucks).

But what if options got a score according to the inverse of their rank (without equal ranking except at the end)? This is the Borda Count. It can give quite an incentive to give a complete ranking. Say you only want to rank 4 of ten candidates. Then the first gets 3 points, the second 2, the third 1. But if you rank them all, the first gets 9, the second 8, the third 7 etc. This doesn’t seem to reduce the chance of winning for an option if several similar options are introduced. But consider the following:

Alice, Bob, Carl decide on their favourite movie.
Alice says: “My favourite movie of all time is Fist Of The North Star.”
Bob says: “My favourite movie of all time is Fist Of The North Star, too”
Carl says: “I liek Police Academy it is LOL!!1”

It seems the winner is clear. However Carl remembers he also likes some other movies.
Carl: “Police Academy I is best, Part II is almost as good as I, III is almost as good as II, IV is almost as good as III, V is almost as good as IV, VI is almost as good as V, and VII is almost as good as VI. Fist Of The North Star is almost as good as Police Academy VII. What are your opinions?”
Alice and Bob remark how much they hate all Police Academy movies and both give the following movie ranking:

  1. FOTNS
  2. PA I
  3. PA II
  4. PA III
  5. PA IV
  6. PA V
  7. PA VI

Now guess who the Borda winner is 0_o

The chance for an option to win increases in Borda when similar options are in the race. I think this is much weirder than the opposite problem. There are several versions of Borda but they all suffer to quite some extent from this. It is the most idiotic voting method I know.

It is much better when people are free to give ratings. However with ratings it is almost always best for a tactical voter to only use the extremes of the scale. Even if you have not the slightest idea how others vote! Big scales give an illusion of precision. They tempt people who are honest to have less influence on the outcome than tactical voters. So it is better to have a small scale. Binary is enough. The binary version of ratings is named Approval Voting and is advocated by political scientist Steven Brams.

When somebody first sees the difference between a First Past The Post ballot

MARK ONE OPTION. THE OPTION WITH MOST MARKS WINS.
A ( )
B ( )
C ( )
D ( )
E ( )
F ( )

and an Approval ballot

MARK ONE OR SEVERAL OPTIONS. THE OPTION WITH MOST MARKS WINS.
A [ ]
B [ ]
C [ ]
D [ ]
E [ ]
F [ ]

he might ask: “Isn’t that against ‘One Man One Vote’?”

Is he against women voting rights?

“Umm no. But candidate A and B are the most popular so almost everybody is voting for one of them. My true favourite is A, so I vote for A. And only A. Somebody else said he likes C so he will mark B and C and that is unfair.”

Why?

“Because he has two votes and I have only one!”

Wait a second. You have it easy because your true favourite is somebody who will probably finish first or second. The other knows that his favourite chance to win is slim so he has to compromise by voting someone equal to his favourite. You would prefer to be in his situation?

“Butbutbut the method is crap anyway coz what happens if there are similar candidates? You know candidate E(litist McLefty) who is an elitist leftist has an identical twin F(renchy McLefty) with identical positions. Those that vote for E also vote for F and vice versa which increases the chance that one of them wins. I am glad they are not triplets.”

You have it backwards. First Past The Post punishes similar candidates by splitting their support. This lack of punishment in Approval is not the same as giving a bonus to similar candidates like the Borda Count does.

“No you don’t understand. I mean I voted for A, okay? Someone else can vote B, C, D, E, F? Those ballots have different weights. The other is five times heavier! That is logic.”

Sigh. According to that logic, a ballot that approves all candidates would be the strongest. In First Past The Post, adding one ballot to a tie may break it and adding another ballot may recreate the tie. One can view this as evidence of equal ballot strength. However when there are more than two options, two First Past The Post ballots can not completely cancel each others effect. The two candidates in the close tie get further ahead of the rest. But in Approval two ballots can completely cancel each other and you gave just an example for that.

“But I like First Past The Post more. Approval is ugly and unintuitive. And it forces you to be dishonest.”

However unpopular your favourite is with the crowd, you can rely on one thing with First Past The Post: You never have a reason to put the option you hate most at another place than the bottom. If there are several options you hate equally most, those go to the bottom no matter what your tactical concerns are.

This is also true for Approval. It adds the following: You never have a reason to put the option you love most at another place than the top. If there are several options you love equally most, those all go at the top no matter what your tactical concerns are. Approval is symmetrically completed Plurality.

Well, I agree (now) :slight_smile:

[See also: http://en.wikipedia.org/wiki/Approval_voting]

edit:

One addition… immediate results shouldnt be shown at all. Regardless which voting system is used. (For preventing tactical voting)

edit2: sense++ :stuck_out_tongue:

Woogly, Its nothing against you or your system, but I wouldn’t trust my cat with my password, im afraid it might go tell other cats that and one of them might write it down…

I still think that getting a big amount of judges is a good idea. And letting them use whatever system you come up and let them register on your site. If you decide to want judges this year again, i’ll be more than happy to be one…

DP

you act like you’re giving me the password, but I never see it at all. you trust all of the subnets between your computer and this site including through your ISP. I highly doubt my script would go tell other scripts about your password. as stated before, if you dont trust it, temporarily change your password …

The approval system works for me, I guess… But I do think it should be modified slightly to give an average between 0 and 1 instead of letting the one with the most votes win. Not a big deal, but I think it’d be better in this particular case :slight_smile:

Morre, can you please eleborate with an example. Its sorta unclear what you mean. Do you mean using a scale for the rating or an alternative way for representing the results? o_O

Well, I do know what I meant, but on second thought it’s silly :slight_smile:
Never mind that last suggestion.

Btw we can run the poll with this forum software without any modifications. Well, that is if we use approval voting.

Say there are 50 games at the end. Then there are 50 options, 50 votes per user, the poll exipres in x days and [x]Only show the results after the poll has expired.

And the question could be something like “What were the best games this year?”. You could vote for one game, for your personal top 3, top 5 or top 10… or even for all (voting for all or not at all has the same effect).

of course its a good idea to have an overall best 4k game vote but i think we should also put in other stuff, like sub catogories like, best gameplay, best graphics, best technical achievement, etc etc, and maybe go even further with top platformer, top puzzle game, top shooter, top action etc. (maybe even like top 5)

not only that but we should rate different parts of a game, gameplay, replay value, sound, graphics etc. this way we should get much more accurate view of the contest.

this way alot more of the contestants will be happier with the outcome, as it can be quiet a let down if you put a lot of hardwork into a game and it doesn’t get the recogition it actually deserves.

although this may sound like a lot of work on the part of the voters a few quick check boxes, pull down menus should make it quick and easy.

thats just what i think would be best IMO, donno but what do u guys think?

I am actually more keen on a few “experts” as judges as I am more interested in comments about different sections about the entries rather than who wins :slight_smile:

I doubt anyone will put critical comments down if the voting is public.

I like the different sub categories idea, but it may be hard to make a simple way to vote. Anyway, the best gameplay, best graphics, best technical achievement and best sound/music would be great. I think it would be better for this categories to get only one vote and not a top x.

If searching for the critical comments, I think too that public vote will stop lots of critical comments.

I think that if we make people vote, make people vote the games they played. I seriously doubt that all the people who may vote will play all the games, maybe played the ones they get attention for screenshots or name (for example) and then vote the best from the ones they played, so rather than a best 3/5/10 I think it would be better a list with all the games and then the voter rate the games that he/she played with the different categories required (pulldown menus to make it easy) and a final score that he/she thinks the game deserves (so the game may lack sound, for example, but it could score 10 on the final score) and finally a textbox for comments (maybe not required but very appreciated) . This way it could take a long time to vote for ones that played all the games but it would be more fair than a “top something” IMHO.

I think we should just text IDOL01 into our Cingular phones to vote for Latoya London!

After reading this thread, I believe the title is answered : “who needs judges?” The 4k Contest and it needs them badly.

agreed, sir monkey :stuck_out_tongue:

volunteers?

cough me! cough

DP

Honestly, it’s simple and works - it’s transparent, in a way that voiting by anonysmous people isnt’. And you can be sure a handful of people almost certainly aren’t cheating because people know who they are.

You do need to tell the judges in advance waht teh “objectvie” criteria are - this is the palce where it fell down last time (as noted before, TBH I think simply reading the judges comments from last time it would be easy to do it really well this time with minimal effort. Why re-invent the wheel, why not just improve? Shrug)

Finally…I’m speaking at the Casuality conference in a week’s time. I would really really really like to be able to point people at 4k entries whenever I come across someone interested in it. Please, authors, make sure you’ve got something up by the 7th feb ;).

If you need an extra one, and we have a deadline for judging ;), and someone can write guidelines, and they actually all work this time :P, then I’m happy to do it again.

What do you need more entries for? Just point them at Xero and you’re done! (I kid! I kid!) :stuck_out_tongue: ;D

wow I have not seen blah^3 in a while! since you don’t have a game in the contest adam, we’d love to have you as a judge. I’m trying also to get some of my gamer friends from Dell (that’s where I work). those guys really don’t care about the technology aspects, just if the game is good quality fun.

3 tech judges and 3 gamer judges would be a sweet balance

I’d be willing to judge depending on the time frame. I’m moving to Austin, TX mid-Feb, so will be indisposed for a few weeks.