for the record, the judge in question was running Windows XP SP2.
please note that the rules are presented clearly, but the adaptations learned from the year do not appear until the next year. since I have been running the contest, there has not ever been a Windows-only rule, and I don’t even remember one. (and don’t bother referencing one, because I believe you)
regardless, fullscreen is a gamble, even on windows. the situation with Xero is regrettable, but wouldn’t ever be completely avoidable (unless you all want to donate a handful of computers of the exact same hardware specs to me for each of the judges). this is one of the gotchas of using a judges’ board that is both voluntary and long distant.
(also, I have been around since contest #1. try not to remember my “rock/paper/scissors” game and other creations… >_>)
I’ve actually been harping on this for a few years now, so how about we see about getting this codified? Do we require all games to run on all three platforms, or can they optionally only run on specific platforms? The former would mean that game developers would have an added burden of testing all systems, and having to work around platform-specific bugs.
The latter would make the lives of developers easier, but it would force the judges to test on one of the systems the developer claims to support. (For example, Xero would have only been testable on Windows and Mac.) There are tradeoffs either way, but the important thing is that we’re clear on what is accepted and what isn’t.
[quote]regardless, fullscreen is a gamble, even on windows.
[/quote]
Writing code is a gamble. Technical issues can and will occur. That doesn’t mean that judges shouldn’t make a best-effort to solve the issue. While I was a bit upset that Blah was unable to judge several items in year 3, he did make an effort. Here are a few of his comments:
[quote]JM4K - Full screen game that doesn’t work even on windows, crashes out
Pang 4K - broken - supplied instructions fail to work in windows or mac
[/quote]
He included several other comments related to code that wasn’t Mac or Linux friendly, but mostly just dinged them a bonus point. As you may remember, Blah’s difficulties were what lead to the creation of the executable rule that forbade code that required batch files or command line sequences. This rule was very effective in preventing the same issue from occuring last year.
Now again, we face a similar issue. How do we want to adapt to the problem this year? My take is that the judge should be required to perform basic troubleshooting before his score is accepted on the matter. If we don’t want to do that, then other options include:
Allowing judges to abstain.
Use Truncated Mean Scoring
[quote]the situation with Xero is regrettable
[/quote]
It is regrettable, but it also isn’t the point. You know that I would argue this whether it was my game or not. (Actually, I might argue a lot less if it was just mine. ;)) I participate in solving these issues every year.
The point is that a problem developed last year in more than one entry. Several suggestions were offered to solve the issue, some without making significant changes to the existing system. Let’s do like we do every year: discuss the matter and make a decision about how it should be solved. Our current system is getting really close to excellent. If we can continue to shake out the bugs, hopefully we can get it near to perfect.
[quote]but wouldn’t ever be completely avoidable (unless you all want to donate a handful of computers of the exact same hardware specs to me for each of the judges). this is one of the gotchas of using a judges’ board that is both voluntary and long distant.
[/quote]
The distance involved is well understood. My own suggestion is not to make the machines uniform, but to:
Make a decision on the allowable platforms for judging.
Make a clear decision on what that means for the contestants.
Have the contest manager (in this case, you Woogley ;)) certify that a judge made a best effort to get a game to run before accepting their score. (Remember, once the scores are published, it’s too late for a judge to say, “Oh look, I got it to run!”)
1 & 2 were not an issue in last year’s contest. However, I think they should probably be resolved now, as Java Gaming is now approaching 100% feature compatibility across platforms. Thus we’re more likely to experience issues going forward.
3 is the key point as it relates to last year. My suggestion is a simple one that places no extraordinary burden on the judges, contest manager, or developers. If a judge certifies that he’s made a best effort, then by all means ding the contestant for it. I just don’t think it’s a good idea to be accepting scores that are incomplete due to no fault of the developer.
[quote](also, I have been around since contest #1. try not to remember my “rock/paper/scissors” game and other creations… >_>)
[/quote]
I trying to remember. Wasn’t there a year or two you didn’t submit an entry? Or is that my faulty memory? ???
If I remember correctly (I can’t go back and check because javaunlimited.net is blocked by my company), the issue with my game, JSquares, was that the judge had JRE 1.5 installed. Since my JNLP file specified that the required version was “1.4” and not “1.4+”, he had to download 1.4 in order to run the game. In his eyes, this was my application requiring an additional download, and thus a 0 score. Does this count as a fault of the developer, or a failure of the judge to make a best effort? (In my opinion, the JNLP file was not part of the entry, and thus I should have been allowed a chance to fix the JNLP file).
It seems to me that the judges should be required to meet some compatibility minimum - such as having the required target JVM version installed (1.5 was disallowed, he should have had 1.4 installed).
Not trying to whine about my score - I had a blast last year, and i’m looking forward to this year.
but what you’re saying is that I should’ve let you alter the JNLP after the March 1 due date, which isn’t fair. this is why you should test, and get your friends to test, and their friends to test. surely somebody has a 1.5 JRE (especially if it was posted on JGO??)
judging is taking a large step in another direction this year, with two new systems in place. if you haven’t read about it, they are: 1) chosen judges board and 2) people’s choice award
even with the chosen judges board we cannot guarantee a “compatibility minimum” per se, the best I can tell you is test test test. we could introduce a “standard” OS for judging, but being a java-themed contest (and thus platform-independent)…
[quote]even with the chosen judges board we cannot guarantee a “compatibility minimum” per se
[/quote]
I don’t think anyone is asking for one. Merely guidance on what to expect during testing and judging. Remember, everyone is doing this for fun. Throwing them unexpected curve balls is no picnic.
curveballs aren’t being thrown by me, they are just happening. who’s to blame? who cares. the “infamous” Xero-Zero judge clicked the link, it crashed his system, he gave it a 0. sucks, but that goes back to using fullscreen is a gamble… not just across OSs, but across hardware.
no matter what judging system we have, something has the chance of blowing up. we’re writing some pretty insane code here. the best compatiblity I can give you is use the recommended Java 1.5. that doesn’t mean your code won’t break in certain situations, though.
Whatever. Apparently it’s eaiser to say, “we’ll spread the blame around, then ignore that issues occurred” rather than looking at ways of improving the contest like we’ve done in previous years. Perhaps I’m an old dinosaur around here. Sorry I showed up.
sigh how am I not looking at ways to improve the contest? just because I don’t let everything out in the open months early does not mean that there isn’t anything behind the curtain.
did you not read that I have 2 new judging alternatives? -_-
This is the 4k competition… It is meant to be fun making them… all this bickering about how it is to be scored seems silly…
The way i see it, the competition is to attempt to create a game in 4k, i.e. the competition is your skills vs that 4096 byte barrier!
Gaming is such a subjective thing that I would not hesitate to say that there is no “best” game that is produced.
I definetly do not go into the competition to “win” I go into the competition for the challenge and to see what my “best” can produce. If other people like my game then that is nice, but not a large motivator for me.
I hate to come across as a whiner, especially since I am actually satisfied by my result, but…
Since the JNLP is NOT part of the entry, but is an OPTIONAL deployment feature, then yes, I think I should have been allowed to fix it. It would not have affected the size, performance, gameplay, design, implementation, or anything else of my actual entry. My entry was posted on JGO, and a great number of my friends did play it. No one else reported the issue with having to download additional components.
This, I think, is the biggest point I feel needs to be made. If the contest requirements state that compliance with a certain version of the JMV is mandatory, then the judges MUST have that version installed in order to do their job. I don’t care what operating system it’s on - it just has to be the correct version of the JVM.
I believe I will do just that. Now, on to coding my entry(ies) for this year’s contest…
