Judging panel results

The results have been published.

I must apologize for the delay, I disappeared for a few days because I was on a conference in London and had problem doing any work online. And the judges required a deadline extension. Unfortunately, princec did not deliver and so we only have 3 judges then, and that is all good.

See results:

Thanks for the reviews! ;D

Thanks to all contestants for participating and congratulations to ALL the winners :slight_smile:

oh interesting how some of the judging results are so close to each other :slight_smile:

btw something looks odd about the java4k judging grades e.g. Simple4k got a grade of 2.3% but was reviewed by the judges as 36%,38%,39%

The scores must be normalised to the full range 0%-100%.

I don’t know what two of you guys were thinking when you rated my game above 4Kanabalt, but thanks! ;D

PS Appel, I expect it’s your busy schedule, but in case it’s forgetfulness instead: the community voting page still has “Review not ready” for all the games.


Yes, the normalization method gives a weird result for Simple4k. I don’t believe it deserves such a low score. But nothing we can do about it now. I hope Gavin doesn’t get too offended. :\ Just check the judges actual score because that’s a more accurate representation.

The results are interesting, both community and judges consider 4Kube3D to be a winner, and that’s great. Then the order changes totally almost, clearly difficult year to compare games.

I agree, you should / must simply take the average.

Thanks for the effort of publishing the results! (I have no idea how much work it is :))

Hey wow, what are these medals? :smiley: I feel like a winner!!!

How do these medals work? How do I give someone a medal?

It’s just a side effect of the (reworked) appreciation feature! :slight_smile:

Yes and no. No because the Pearson rank correlation coefficient between the overall judges’ score and the community vote is 0.75, which is only a hair below what it was last year (0.78). Yes because the correlations between the judges this year are much lower than last year (0.32, 0.59 and 0.52 this year vs 0.65, 0.45 and 0.57 last year).

You mean the correlation per judge with the community votes?

correlation judge A and community: 0.32
correlation judge B and community: 0.59
correlation judge C and community: 0.52

or the correlation between the judges themselves?

correlation judge A and B: 0.32
correlation judge B and C: 0.59
correlation judge C and A: 0.52

both are interesting to know :slight_smile:

The correlation between the judges themselves.

The per-judge correlations with the community votes this year are 0.62, 0.58, 0.65; vs last year 0.59, 0.81, 0.55. If Demonpants wants to get into game reviewing as a career, he could do worse than to cite these figures ;D

sorry dudes, just been too busy!

Cas :slight_smile:

I’m surprised i won, so many great games this year!

Big thanks for the comments on the forum and of course to the judge panel for doing the reviews!
How long did it take to review all games?

they took forever thanks to Riven ;D

Congrats Erik Byström on winning BOTH the community voting and judging panel voting, a clear winner.

I spend maybe 10-15 minutes per game. So, that’s 3-4 games per hour, and I usually don’t spend more than 2 hours per evening. With 45 games that can spread over many evenings, but I try to include 1 or 2 games during the day as well, here and there. So, overall it takes a full week to finish the reviewing, doing 6-7 games a day. But my pleasure :slight_smile:

yup, congrats on becoming the Java4k 2011 winner (should make a nice addition to your CV :)).

About 2 weeks to randomly just play through entries and about 3 hours (over 3-4 days) to write up the reviews.

Two evenings and two nights, as I was under the impression I was the last judge, holding up everything. :-\

At the time, I wasn’t, but after Cas didn’t have the time to review the entries, in the end, I was the last judge… but still, I could have spread it over a few more days, as appel was having connectivity issues. Blah blah blah.

I still haven’t played any of them even now! Next time try to organise it so I’m not having a baby and releasing on Steam :wink: heh

Cas :slight_smile:

Thanks judges! Excellent reviews of my game. Very helpful feedback. Lookout Java4K 2012! ;D

And a big congrats to Erik for a well deserved win!