I need to study multiplayer architecture for RTS games. Any suggestions?
There won’t be a dedicated server, other than a lobby, don’t have server resources for dedicated servers.
A player that starts a session will be the server for those that join it.
I’m going to adding it to my Galactic Commander game ( http://gamadu.com/games/gcom ).
I have a running RTS style game on my webpage. Extremly crappy. But it does support a lobby style setup.
I personally just use basic TCP.
Try read up on Multi-thread TCP connections. With TCP you wont need to keep sending all the data from the game.
Just update the requests made from each client forwarded to all clients. as each client should handle the request the same.
KeyDown, send move forward request.
KeyUp, send stop moving forward request.
Well, that’s not really what I’m wondering about. TCP, UDP and multithreading is given.
I’m wondering about the architecture. How do things happen bechind the scene when a player tells a unit to move to some location. What data is transfered where, who makes the logic updates, how do other players get this information, the whole lifeline. And, let’s not forget, synchronization… that is how do I ensure all players are seeing the same thing.
Also, how to deal with rapidly spawned gameobjects, such as bullets (e.g. 1-4 per second per unit)?
Are you suggesting that all clients are connected to one server, which handles requests from all clients, and propagates to all other clients? (I’m thinking the same)
What do you mean by this? It’s an RTS, not a first-person-shooter ???
well, only send a packet when something is actually done by someone.
so dont send a packet for every bullet.
send a packet for when the attack starts, and for if/when it ends.
or if someone moves someone, dont send a packet for each frame moving. send a packet for the original command.
hope I could be helpful,
h3ckboy
Very helpful, thanks.
I’m gonna research this more though.
I know there was a page around here somewhere which had a lot of helpful hints on what to send.
What h3ckboy said is true, you should only send packets when a action starts and then when it finish.
Depends how secure you want the game, the server(player hosting) should do everything or if that is going to be too resourceful then do basic checking on everything.
For my MMO I was working on , when the player walked anywhere it send update packets every few steps to keep in sync, when they did go out of sync the server would just walk to the spot where the player is on the client. If the client walked too much out of sync or that the server couldn’t reach the clients position then the client would warp back the player.
Hi, for the multiplayer peer-to-peer RTS that I made (http://hsaka.webs.com/lib/ninjawars.jnlp), I used TCP and a variation of Lock-Step synchronization (http://www.gamasutra.com/view/feature/3094/1500_archers_on_a_288_network_.php) which I found out about from these sources:
http://www.java-gaming.org/index.php/topic,19880.msg159952.html#msg159952
http://www.java-gaming.org/index.php/topic,21336.0.html
http://www.java-gaming.org/index.php/topic,10516.0.html
http://www.cokeandcode.com/rtsnetworking
[quote]Are you suggesting that all clients are connected to one server, which handles requests from all clients, and propagates to all other clients? (I’m thinking the same)
[/quote]
That is how I did it.
hey appel, check these out.
http://www.java-gaming.org/index.php/topic,3434.0.html
http://www.java-gaming.org/index.php/topic,10516.0.html
and gander through this form topic for GregPierce, blahblahblahh and kev (among many others) all had great networking discussions. also check out darkstar. I use it for a CubeWars shooter and it did pretty well.
it is all about the simulation. Architecture is pretty simple.
Abstract with layers.
You have 2 different setups possible
Commands are initiated by clients. Commands act on the simulation.
Commands are serialized and sent to the server/star network
Commands are put into executing buckets.
When a buckets is synched to be identical on all computers, the commands are executed in that order.
The sim is up to date on all computers.
Buckets are timeframes to recieves commands before the synch check and execution. Typically between 50-150 ms.
any question?
The details depends on the scale of the game. for a smaller scale not really RTS game that was first to multilayer you can check out netrek. I would be careful about using existing games as “good” examples as most of the network code out there truly sux. Most of it was written by people who clearly knew nothing about networks.
Anyway here is my $2 worth. 
If the scale of the game means that a state update is large (say 1000 of units) then synchronized simulation is the only way to really go. The network is still the bottleneck and you only need to “transmit orders”.
Comerical RTS games use a P2P network connection pattern which is pretty terrible as for 4 players you have 16 connections and only one need to be bad to lag the whole game. They get away with it because most games are 2 player, and most 4+ players are typically lan players.
Server is the way to go. But as you suggested a player hosts the server. 4 players mean 4 connections so its less likely to lag. However the server player gets to see things in the future ever so slightly. Even on a lan this is generally more stable.
Finally we have TCP/UDP arguments. Most of the time people just say UDP “because its faster”. This is not generally true. In some instances TCP is faster due to the stateful connection --NATs, firewalls and routers can “cache” and compress TCP packets and often do.
For a synchronized simulation TCP preserves order and has reliable transport and has flow control. So i would always start with that. Really pings across the EU and even to NZ were the same for or better for TCP compared to UDP.
Now I have a large scale (1000 of units) RTS game with 2-16 players. I use synchronized simulation with a server client architecture.
A client may issue a command. Say “send these 100 units to position x,y”. This is sent to the server, the server schedules the command into a game turn, then sends out all the commands from all players for that turn. The clients then put the commands into their game turn schedules. Game turns commence at about 5 per second currently. Note that schedules can and do lead games turns slightly. It should be quite easy to see how the “network” controller, and or local controller can plug into this model. Synchronization is achieved via discrete game turns.
The trick here is that a lot of behavior is “local” to the unit. ie its not a command. If a unit is on guard, it will attack anything that comes close, the client does not need to send a command for this. Same with formation behavior etc.
There are details. like a client predicts what game turn the command will end up on and makes the units move --resyncing later if needed. This reduces the “apparent” lag.
Only last thing. If you use UDP and you resend packets. You must have flow control.
I would abstract the network details to the GameEngine and have flow control anyways at the level of the GameEngine.
UDP, TCP IPX (I remember IPX to be lightning fast on LAN for Age of Empires) or whatever. Who cares. The Network layer is interchangeable.
Your game engine recieves Game Commands with schematically an CmdID:PlayerID:CmdCount:Parameters
The Network engine construct the GameCommand and send it to the above layer using some kind of consumer/producer pattern.
Flow control checks the CmdCount relative to each Player. If a command is missing, ie. CmdCount is 343, 345,346,347. Something went amiss. Where is CmdCount 344? I would have the “GameTurn”/Bucket class check this. The protocol used to actually transmit the bytes is irrelevant. With this, it is easy for the simulation to be synchronized. After that you can optimize the basic idea with Acknoledges to speed things up a little and smooth the communicaitons.
I would prefer using UDP for Acks.
delt0r, what do you think about this? Clients connect to the server via TCP, and all run the same simulation. Clients send requests to change the simulation to the server. If valid when the server receives them, they are sent to all clients, which updates their simulation. No turns, requests queued to be executed in the future, or other magic.
That’s not flow control. Flow control is reducing the amount of data when the network starts balking. So lets say i have a congested network, and i send my 100 UDP packets per second. However after a second the client detects that it only recivied 90 packets. It then sends a request for the missing 10 packets. Now the server sends 110 packets for the next second. However since the network is congested only 80 packets get through…until total collapse. This happened in the early internet before TCP had proper flow control. I have tested this on some quite popular games… they do the same thing… They flood the network with extra packets, this is not how you deal with a congested network. Its also why game forums are full of “connection” and “desycn” threads. Moderns games with only a few exceptions are crap with respect to the network layer.
Yes you can do that but its very difficult. The theory is synchronous model vers asynchronous. You also must have some way of doing a n-way resyn. I don’t see any advantage of the approach.
Hmm…
a basic flow control can be achieved by doing a ping / pong cycle between client & server.
client : send a ping every time he received a pong
server : send a pong (+ all necessary game packets ) every time he receive a ping
the ping/pong cycle may also be used to synchronize clocks between server & client (you will then use the server synchronized clock later client side to synchronize units animations the same on all clients)
Here is another possible idea for the game :
I will do all the game logic / control server side ( not everything : only units move & states) and let the client display animation on unit based on their active state & location.
client side you can use “time tracks” for each units/entity :
thoses tracks are filled when receiving packets from server, eg : “unit 12 is at location 10,15 at timestamp=100205ms”
when displaying a units you compute it location by reading (interpolating) its tracks at the rendering time using the synchronized server clock minus some delay (3/4 secondes seems ok for an RTS)
track sample :
unit 12 is at location 10 at time 1000
unit 12 is at location 20 at time 2000
rendering frame at server time minus delay, let says 1500:
unit 12 is rendered at location 15 for that frame
inded you have to only send delta change to client
for a simple case you will have two possible packets going from server to client :
unit move to x,y at time T
unit attack x,y at time T
=> two tracks : location, action (both time based)
client will also perform some kind of logic but nothing that can change a fight results or influence other players, but it can choose to rotate the sprite to match the direction it is moving to, or animate it based on its current action & life points , etc…
client can choose to render in different way but everything will end the same, because client are never able to influence/impose a game state change, they only act as a passive viewer
OK. Let’s take that semantic. Now let’s imagine.
I think my point still stands. That is implement flow control above the network layer. Let me explain.
The bucket of commands will not be emptied until “Turn Acknoledges” for that bucket have been exchanged between all peers in the synchronized simulation. If Turn Acknowledges says that something wrong with bucket of Peer #2 (missing commands), all simulations hang and wait for Peer#2 to resolve its issue. It might be network congestion and the protocol used is irrelevant.
Now I think you can call this “Flow Control”. UDP, TCP are just protocols to send packets. A command may need several packets to be serialized over the network.
Flow Control of TCP may be nice, in that you don’t have the implement “Flow Control”. But if you implement a network layer with UDP, you implement flow control in that layer as well. That I will concede. Is this what you meant? That a network layer with UDP must check for lost packets and implement a recall?
Fair enough, but it doesn’t desync the synchronized simulation since it has its own Flow Control as described above.
Now what’s the best way to implement Flow Control in the UDP layer just below the SimulationFlowControl, you tell me.
At the receiving end, UDP Layer receives packets and tries to build a command object right? Then it publishes the command to the SimulationFlowControl for execution in the correct simulation turn. UDP layer also receives packets that are Acknowledges. Acknowledges are given to the SimulationFlowControl that knows what to do with them.
my main point still stands: SimulationFlowControl doesn’t know or care about whether the publisher of commands and acknowledges uses UDP or TCP.
Mordan my point is that you don’t need to do flow control if you use TCP. The IP stack does it for you. UDP with reliable transport you need to add flow control. ie do more work. Yes flow control is not about synchronization. But with congestion collapse you don’t have a multi-player game. Its not just about resending lost packets… its about not sending more packets than the network can handle.
TCP does way way more than just send packets. And most of its quite complicated state machine is there for a very good reason, often from lessons learnt the hard way. Its a good idea to use all that hard work rather than assuming you can do a better job than 30 odd years of work and refinement.
we agree and the high level of the game engine all this is abstracted out.
usually when using UDP it is because you dont care about missing or unordered packets (as in video playback), if you want to use UDP but care about missing packet & order this better to use TCP because finally you will end by implementing your own TCP protocol wich will probably not make a big difference with the original TCP.
EDIT :
[quote]Mordan my point is that you don’t need to do flow control if you use TCP
[/quote]
what you mean ? you can still overcharge the network with TCP