What do you think of an AI contest?

Game Mechanics Artificial Intelligence
#1

hi guys!

Some time ago, i was toying around a new game, and it turned out that making the AI was a real challenge. I tried some simple stuff but the results were pretty poor and i feel there is much potential on developping AI. So i thought, it would be even more interesting to focus only on the AI and make a contest out of it!

…As you can see, it is very vague now. On the other hand, i can’t say more so that the contest could take place “correctly”, unvailing the details to everyone at the same time. The only thing i can say now is that it has much potential and can be implemented with a variety of techniques. …And it’s not that easy. …And the AI’s will compete against each other. …It’ll be fun, you’ll see!

If there is enough people wanting to take part of such a contest, then i’ll make a small framework to build the AI on the top of it and to visualize how it performs or “quick-testing” it. Then start dates and deadlines will be set and details unveiled.

What do you think of this?

PS: I promise it’ll be interesting ;D

#2

that would be cool. What im thinking is that someone builds a game of some sort, and we have no players, just AI involved. And we see which AI wins :slight_smile:

But the thing is, if someone does it in jME then jME accustomed users will have an advantage over Xith3D ones as they can optimise for things…etc. And the opposite is true.

Thus the underlying technology MUST not be revealed.

What kind of game? fighting seems a pretty nice challenge Like tekken3 or something?

DP

#3

I like it! I’d take part in a contest like that

#4

cool! :wink:

It’s exactly what you meant, what i’ll provide is a simple game skeleton where AI players compete against each other. However, i can’t tell the sort of game now. No evolved libs will be used, just plain standart java2D, so that there is no need to install libs nor compatibility issues. Then you’ll use some interfaces acting as a layer between the game and the AI.

#5

I see the Robocode project being the ideal framework for this competition.

http://robocode.alphaworks.ibm.com/home/home.html

They even have leagues setup and demo bots that you can test your code against. It seems silly to invent a new framework, and the robocode stuff is already written in Java!

The hard part of any competition like this is the judging, how do you decide a winner? It’s easy if your AI is testing skills like fighting, the last bot standing wins, but how do you test other routines? If the test is path finding then things like A* have already been developed and are widely seen as being the best they can be for given situations.

Andy.

#6

sounds like fun, but robot wars has been around since the days of the appleII. maybe we should shoot for something that looks at other ai application dynamics.

how about:

  • each player gets to control n number of bots

  • winner determined by team that destroys the other team’s base first.

  • optional twist could be adding landscape factors where bots could get trapped, slowed, or damaged if not careful.

  • another optional twist might be to allow the agents to modify the environment (blow up barriers, create barriers kinda thing.)

  • bots should be resource bound. no bot should be able to fire an infinite amount of rounds, etc…

  • scoring system would take into number of bots you had left when you won. it should also be a time-limited event with a slight penalty for draws to prevent sand-bagging strategies.

  • implementation of the framework might best be done by providing a bot battle ground server where your bot proxy is controlled by issuing bot commands. all queued up bot commands that are received are played out based on timestamp for a given game unit timeslice. controlling clients are free to send their bots commands as often as they like, but each command type should have an effective window of completion. additional commands received while performing an uncomplete task for a particular resource system of the bot is ignored. for example if client A sends 100 fire commands in 5 ms and we deem firing to take 10ms, then the bot will only fire once. but, you could send a fire command and a move command 1 ms apart and have them execute immediately because they involve two separate resource systems of the bot (locomotion system and weapons system.)

just a few thoughts…

#7

Hey that sounds like a fun game, I wanna play it! :slight_smile:

#8

yeah, robocode is a nice cool app… however, this was a proposal for an alternative challenge. Making such an improved robocode would be out of my capabilities anyway and way to much coding. :-X

#9

this has progressed from something managable to something that has to be done by a trillion developers. In other words, it doesn’t need to be that complex.

I have created a nice little AISystem that sits on top of jME. All it does is fire events for each entity (or global events), set agent actions (for each entity), send messages to other entities (or globally too) and obtain entitites from collisions. I think this is the ideal framework to get started on. Its managable and every extendable. Its only meant to provide a consistant frameowork and not meant to be a complete AI solution to all AI problems. But I did write it, so I might have an advantage…Any other ideas?

Ok, idea. ;D
Has anyone played Worms? You know, the little worms that kill other worm using different gun methods? why not that, but with AI. It could only be used as starters. No pathfinding is completely necessary, just testing routines and effectiveness…last surviving wins :slight_smile:
What ya think?

DP

#10

I thought this was supposed to be a poll on having an AI contest. It seems now that you’re having to write your own framework to run the contest before you’ve even started to design/build the agents that are to be compared.

Thats why I suggested Robocode, the framework is already written, people can start writing the agents straight away, it’s a level playing field since no-one here (I presume) wrote Robocode.

Is this a contest for developing AI agents or see who can write the best AI framework environment?

Keep it simple and more people will enter. If you want to do more than Robocode provides, extend Robocode, the source is available, why start from scratch?

Andy.

PS: I’m in no way affiliated with Robocode. I just don’t see the benefit in writing a new framework for this contest, and then debugging that framework during a contest.

#11

[quote]Is this a contest for developing AI agents or see who can write the best AI framework environment?
[/quote]
it is neither. it is a solicititation for interest in an ai contest.

#12

yeah, this discussion is going in every direction…
To state it again: if enough people are motivated i can work and provide a simple framework and the contest would be to write the best agent AIs.
…but i don’t feel enough people will take part.

#13

[quote]yeah, this discussion is going in every direction…
To state it again: if enough people are motivated i can work and provide a simple framework and the contest would be to write the best agent AIs.
…but i don’t feel enough people will take part.
[/quote]
i think you are right. seems, we are all more excited about writing or proposing such a framework than providing the ai for it! maybe we should have an ai plug-in framework writing contest? or maybe we need an ai agent contest that has cash prizes? on a related frontier, there are some pokerbot writers that are doing pretty well these days.

#14

the thing is, if I start coding an AI into a framework that I dont feel secure enough with, i will most definetly do badly because im having to go around every limitation of that framework. But some people might feel comfortable with the framework, hence they get an advantage.

I suppose that will be true for which ever framework anyone sets out…

But I do think its a good idea. I feel that this contest should be game related, rather than “let see which snail gets more leaves!”

DP

#15

– You probably don’t care since I know very very little about AI but…

If you make it game related hows it going to be judged purely on the AI? Other factors will become prevalant.

Forcing your AIs to solve the same problem in the same environment is surely the only way to actually have a contest of AI programming.

I have a feeling plenty of people would take part if someone gave them a simple agreed upon framework in which to write their AI.

Whoever produces that framework is instantly ruled out of competing. If the framework is done correctly it shouldn’t actually matter what the rendering technology is, infact it might be a good task for the developer of the framework to produce renderers in other technologies while the contest goes on. So, if we were to take MisterX’s idea…

He writes a framework to support the AI contest. The initial renderer is written in Java 2D. This means that every competitor can get up and running without needing to understand yet another rendering library. Then while the competitors write their AI plugins MX can continue to write any renderer that the competitors have mentione (JME just for dp ;))

The other thing that seems pretty important is to keep the framework closed source. Otherwise unscrupulus types like me will just look at the source and work out a way to “fix” the contest or simply to cheat :slight_smile:

– Kev

#16

There’s a lot of work involved to prevent cheating.
I suggest you look at MUD development and read all the tricks and tactics to prevent malicious code screwing things up - e.g. counting of how long code has been inside a loop in order to decide whether it’s hung, etc.

Also look at JRobots and ask them for advice / help.

NB: JRobots has had massive problems keeping more than a tiny handful of people interested enough to code regularly. Such competitions keep dying, then being ieresurrected, then dieing again.

Part of the problem is that it’s usually f****ing lots of work to get a bot that does reasonably well instead of being completely crapped on by the compeition :(. IMHO that is largely due to the design of JRobots etc - they have arenas that are too small for a start.

With my genetic programming hat on, I’d say that they tend to be binary success: you win or you die. You need a game where success is much more analogue, i.e. you will tend to rack up a lot of points before dieing; perhaps use a large arena and seed it with targets that can be shot at for points, or something like that - anything so that most entrants don’t just get “you killed 0 enemies, for a score of 0”.

#17

PS if it goes ahead I’ll do a GP plugin that plays it ;D. A friend is just finishing his PhD and he’s released the distributed evolutionary framework (using RMI, but nobody’s perfect :P) as open source, so I’m keen to put it into action 8)

#18

a gp approach would be interesting (given that bots are programs in and of themselves), but i have yet to see any research that correlates the complexity of the problem space to the degree of structure necessary within the resultant gp, thus providing some hint about the number of generations and the computational feasibility of the approach.

#19

[quote]With my genetic programming hat on, I’d say that they tend to be binary success: you win or you die. You need a game where success is much more analogue, i.e. you will tend to rack up a lot of points before dieing; perhaps use a large arena and seed it with targets that can be shot at for points, or something like that - anything so that most entrants don’t just get “you killed 0 enemies, for a score of 0”.
[/quote]
good insight. every adaptive system requires good feedback in order to successfully evolve.

#20

Somebody mentioned JRobots.

IBM has created something similar called Robocode and two other programming games (a racing game and a strategy game).

http://alphaworks.ibm.com/keywords/game

might be worth checking out before you start coding your own AI coding arena.

Edit: Doh, I need to go to bed and get some sleep. I see now that RoboCode already has been mentioned.