Judging Results

Contests 4K Game Competition - 2009
#1

Score tables: http://java4k.com/index.php?action=games&method=grades&cid=5
Reviews: http://java4k.com/index.php?action=games&method=reviews&cid=5

Have fun!

Thanks to all the judges for their effort! Remember, the judges did their best given what they were handed. I hope everyone walk away satisfied, although there might be a handful of games that are missing a review from a judge or two due to problems running the games - I do not feel that it affected the outcome. If anyone thinks his game is unfairly treated he can contact me.

Thanks all.

edit: score-table updated at 22:20 GMT+0 to remove 0 score vote from calculations.

#2

Congrats Markus! (I’ll get you next year! :P)

I thought 4x4k coming 25th was a bit harsh - that was my favourite! Great multiplayer fun (bit tough on the spacebar…)

#3

I think I was injuried since ChrisM was not able to play my game. I have attached it in the previous thread could you please review and add some score!?!?! >:( :’( :o Having zero in all categories drop me very down in the list for PacPitfall.

Also there were other games that ChisM was not able to play. Since other judges were able to play I think it’s not fair with us!!!

Well I have to defend my game and effort on doing it. :wink:

#4

Thanks, but I really think the order is fine - a lot of people had trouble playing 4x4k due to sensitivity, it’s just a bit too hard.

I’m never sure about including zero scores (due to failure to run) into averages makes sense. I don’t think it’s worth revisiting since the contest has never been really been about the results - but maybe next time use that old remove the lowest score tactic?

Congrats again to everyone that was part of the contest this year. Fantastic quality and loads of fun games to show off (lets port them all onto Android now and make millions! :)).

Also a big thanks to the judges that evidently spent ages going through every game and giving thoughtful feedback!

Kev

#5

Well, I can’t say that Frequent Flier was unfairly judged - the reviews follow the rules. It’s just a shame that it came last because now noone will try it out, and I think the game has enough merits that it’s definitely worth playing at least once (if you have a mic ;)).

#6

I think that there should be a better way of scoring the game that did not work for one of the judges than giving it 0%. Arithmetic mean of other three reviews would be better i think.

#7

Well, about the zero scored games, what other score could I possibly give them? Obviously there were issues with launching these since all of the other games launched. I hated to score these three this way, but there was no alternative. And I am not the only judge who had issues.

I tried running them in Firefox and Safari. No dice. As for PacPitfall, I was able to play the jar file you posted and it did work, just after the competition ended. It was a good attempt at the Pitfall gameplay, pits, vines, etc. Just not very engaging. 78% Overall, 83% Presentation, 75% Technical. There you go :slight_smile:

I think Kev’s suggestion might be acceptable for next year. Or, judging can be done by splitting the games evenly across multiple judges.

As for people defending their games vs. the scores they may have received, that is expected! Remember, we are judging based on our personal observations and are just that, our reviews.

#8

Thanks appel/judges for all your time and effort. Really enjoyed my first 4K comp thanks to you guys! Great work!
C u next year 4 more.

#9

?! It probably didn’t affect first place, but that’s a much weaker statement than not affecting the outcome.

BTW darkfrog, could you say which version of Java you were using?

#10

I didn’t expect so detailed reviews ! The wait was really worth them and thanks to the judges for their time. The results seem fair and in particular the first place is completely deserved - congratulations to Markus who left everyone but him… dead ! :wink:

However I must admit I was a bit disappointed by results concerning some of my favourite games, as for World Rally Driver 4k that I found almost perfect technically and on the gameplay level, as well as 4x4k which was not so hard when you remembered to push [space]… And not to speak of Asteroid Alert! and its impressive Elite-like radar, Q*Bert4K, Jetp4k and Deathchase for their perfect conversion, Gravitational Fourks for its physics and graphics, and I suppose that J-Type suffered from being Squarius direct sequel…

My final comment will be that next year it would be nice to reconsider the hosting solution for the games, as it seems it caused a few problems in this 2009 contest. See you in 2010 !

#11

Think I forgot.
THANKS APPEL!

I think past organizers will tell you quite how hard and time consuming it is to do this contest, so really, thanks for doing it!

Kev

#12

Of course for some of the judges some games did not work so they gave it 0% and thats fine, but I think that overall score of the game shoudl not be affected by it ;).
Its rather a suggestion for a further contests, because i think once the results are officialy published they should not be touched(unless someone cheated or something like that).

#13

Yeah, last time the results started getting fiddled with we ended up in a bickering mess, 2 years ago was it.

It’s just good the contest is still running after all these years and that we’re still getting fun games to play :slight_smile:

Kev

#14

Hey

I agree that nothing is ever perfect ;D But as long as we try to achieve perfection, then time is on our side.

Hopefully we’ll be able to use what we’ve learned from this year (huge number of games, better formulated rules for judges) and apply it to the judging process next year. So, next year we might have a different type of a judging process, better, faster.

For games receiving zero score in all categories from one judge, then I recommend reading what the other judges had to say. Perhaps a feature is missing from the judging process so a judge can “pass” on a game, and then his zero score won’t count.

#15

I want to avoid fiddling with the results. It’s never possible to please everyone, I’m sure there are games there that ARE better than the score indicates, and are lower in score order than lesser games, but that’s life :slight_smile: At some point we’ll have a user feedback/score on all these games so the community can decide on it’s own, until then we’ll have to use this imperfect judging review process, which although does give all games equal fair shake.

#16

ok that’s fine. This contest is really really great! Also write down the suggestion of public vote! We could spread the competition doing it! Thanks appel and the judges!

#17

Can I make a suggestion that for next year the handling of “can’t run” games be changed? I’m a bit miffed that I tested on a whole bunch of machines and never had anyone who had a problem. It would have been nice to have been contacted and informed of the problem rather than getting a (very score damaging) 0/0/0. :-\

Score aside, I’d still like to fix whatever issue darkfrog had so no-one else bumps into it in the future, platform and VM version would go a long way.

#18

I have finished to read all reviews, and I agree with userek : the game score should not have been affected by the impossibility for a judge to launch it. Now I wouldn’t be shocked if the classments were recalculated without counting the 0%… However reviews are of course more important than scores, and even if a recalculation was made I suppose the top 3 in every classment wouldn’t be affected. Beside this problem, the organization of the contest was very good and certainly represented a huge work, so please don’t take any offense from this very final comment !

#19

I very much think you should’ve notified the community about this issue before posting the scores and asked for their opinion. Also, we’ve seen the 0-score “couldn’t run” issue several times before, and solutions to the problem were posted every time (including median, or an average of the middle three scores, or counting these scores as dash (-), not affecting the scores). I also think you could’ve contacted these authors to give them a chance to fix the problem.

Two of my games wouldn’t run on darkfrog’s machine (with vista64). I think this is at least as likely to be an issue related to the JVM or obfuscator as my code. Either of the solutions above or having a standard platform for the judges to test on would’ve solved this issue.

In this case, the result lowered the average score for these two games (Bridge4k and Pixeloids4k) from around 90% (or slightly above) to around 75%, in effect moving them about 30 places down in the score list. And I’m not the only one with this problem.

Frankly, I’m astonished this wasn’t fixed beforehand because we’ve seen this issue before.

With that said, I appreciate all the work of both organizers and judges. But I do think this particular issue could’ve been solved in a hundred different ways that would’ve caused way less bitterness and given us a more fair scoring system.

EDIT: Out of curiosity, could you post a list of the scores as they would’ve been with 0% (“couldn’t run”) scores not counted? I’d like to see it, at least…

#20

Thanks appel, ChrisM, darkfrog, demonpants and Mark for the detailed reviews.

Looking forward for next year’s 4K contest :slight_smile: