Event based TCP, IO/NIO and other questions

Networking & Multiplayer
#1

Hello
I’ve finished up a small server client library for myself. I’m only stuck with a couple of questions on how to improve it. At the moment i’m using normal IO for sockets instead of NIO. I looked NIO up and it varies a lot what people think about it. Some say it increases performance, others say it only increases performance on large file transfers and others say it doesnt increase performance at all, but rather makes it more scaleable. The only advantage that i can see is the non blocking write and reading. And it has a channel selection. However this isn’t very important on a game server i was thinking by myself, since i dont write a lot of data (positions, entity creations and entity actions mostly) and i have few clients (never more then 20). So would it make sense to rewrite the library and to make use of NIO?

Also secondly, as it is now i always have a ping of 3ms when localhosting, and i’m aware of the reason that causes this. Basically every connection is a thread in which is read and written. but since i dont actually dont do anything most of the time i added a Thread.sleep(1) call, just to give the application some rest (and not drain all the cpu). Is there any way to circumvent this? my loop on both the server and the client looks something along the lines of:


while(connected){
  if (inputStream.available() != 0) {read in packet}
  if (!packetsToSend.isEmpty()) {send a packet}
  Thread.sleep(1);
}

It works very well and i get a good connection to somebody from america (i’m from the NL) with an average ping of around 130. Of which i’m happy. But i like to improve it some more since sometimes there are some odd spikes.

And lastly i got another question. How to syncronize the connection and the game properly? As it is now i’m using libgdx and my own library. Whenever i get a packet that belongs to the game, i put it in an arraylist (and if not, i handle the packet right away). Then every update cycle of libgdx i read out the arraylist. something along the lines of:


while ((packet = client.getPacket()) != null) { check for the packet and execute the correct functions blabla}

where get.Packet() is:

return (connection == null) ? null : connection.getReceivedPacket();

and the getreceivedPacket is:

    public Packet getReceivedPacket() {
        if (receivedPackets.size() > 0) {
            Packet packet = receivedPackets.get(0);
            receivedPackets.remove(0);
            return packet;
        }
        return null;
    }

This solves multithreading problems quite well and i never get a problem doing this. I was just curious if there are other (better?) ways to handle this.

This must’ve been quite a read, i’m sorry for that! I’d highly appreciate it if somebody could help me out on one, or multiple, questions i’ve got!

#2

Separate the sending and receiving of data. The client read will block until there’s data available so there’s no need to keep polling for it.

Have the server’s game loop send updates of the game state rather than sending packets in the client thread which should be used for receiving data.

There’s no “speed up” in NIO vs IO in terms of ping per se - the performance gain is primarily in scale. It’s like dealing cards in a poker game. Blocking IO is like dealing 5 cards to player A, then dealing 5 cards to player B etc while non-blocking is like dealing 1 card to Player A, then 1 card to player B etc until everyone has 5 cards. Sort of anyway.

Here’s a few links that have been posted before:
https://developer.valvesoftware.com/wiki/Source_Multiplayer_Networking
https://developer.valvesoftware.com/wiki/Networking_Entities

#3

Thanks for your response. Looks like i don’t need NIO after all, lucky me :slight_smile: .
The polling idea is alright, i’ll implement that (will have to figure some threading issues out, but no biggy).

as for the sending of the game state, i’m not sure i understand. That would mean that the server has a gameloop. That’s something that my server doesn’t have at the moment, and sending the gamestate could work. My servers job as is is:
If somebody wants to join, check if they are allowed to join. Then send him an connection id. If the packet received is a gamepacket then it will just send it to all the other clients.

#4

Naturally it depends entirely on what kind of game/server it is.

[quote=“elamre,post:1,topic:43905”]
this and your concerns of synchronization tells me you should be running a game server and have the clients send input to the server. You don’t have to do that, though. You could simply echo whatever it is a client is doing to all other clients through the server - but this is bound to have synchronization issues or even end up with clients with vastly different game states. However, it depends on what kind of game it is and it may not end up being an issue at all.

#5

Well it is a RTS. so synchronisation errors are a big thing that i’m trying to avoid. All the clients have the same entities from the server. And if you make a unit for example, you will request the server. The server will then send a new packet to all the clients that a new object has been made with the identity type and position included.
With the movement i send the current position of the id, and the first node to go to. And as soon as an entity reaches a node i let everybody know so they can interpolate if the entity isn’t there quite yet. I think this might be a problem since at the moment i’m using the delta time for movement, and i’m afraid that if you have a slower computer that your unit might end up somewhere later or something.
I’m thinking of changing the movement so that units will move everytime they receive an update packet from the server. This way your units will only be behind your lagg, but this requires a heavy amount of data though. Also i’m definitely using tcp so the packets will always arrive in correct order.
As for the server, i try to do as few calculations on it as possible. It should just be a link that all the players can communicate with each other with a very low level of cheating prevention (checking your resources if you want to make a unit, if you dont have enough no new unit creation packet will be send).

#6

Someone needs to be in control in some way for proper synchronization. You can get away with what you’re doing if you add someone (The Server best choice) to at least keep score of the time or more specifically, the time step (this also requires a fixed time step for logic/updates).

That’s somewhat how Starcraft 2 does it and if a client falls behind the time step of the server - all clients wait (pause) for the lagging client to catch up.

SC2 has a pretty neat system altogether. Their replays of hour long games take a mere ~50kb. The replay files actually pretty much only hold all of the actions/inputs sent by the clients and when the replay is played back the game engine simply replays the actions/input of the event and ends up with an exact replica of the actual game.

More info at blizzhackers.cc SC2 subforum on that off-topic subject.

#7

I’ve had some nice reads.
One thing though, i can make the reading blocked. But how would you do the writing? You will always have to wait poll there to see if there are packets in the list?

#8

I don’t get the analogy. I think what you describe is two different types of multiplexing. Blocking is waiting for someone to text you back and texting back immediately while holding a text message conversation with them. Non-blocking is putting the phone on silent, using the internet while you wait and checking for new messages whenever you feel like it.

That’s not really true. If you have a game with two players and a pre-established connection, then synchronization is trivial. Just send state changes and acknowledgements for each player. There is no server in that scenario. Extending it to more players is just adding a client to the network. Peer to peer RTS games with more than two players was done in early multiplayer RTS games, back when players had worse internet connections and computers.

That is exactly what NIO gives you. It won’t have an effect on network transfer speeds. (Latency or throughput) Network speeds are much slower than local data processing, so it is not a bottleneck. Just remember to flush packets. If you use TCP then they will be buffered at each router along the network path. TCP also creates lag be require all lost packets to be resubmit even if they are redundant or outdated.

Determinism is essential to synchronization. Small differences mean the synchronization is lost. If there is no way to detect or prevent it then it will snowball into much bigger differences and cause worse problems. You may use fixed time steps for game state updates so that all players make the same updates. You can still use interpolation for graphics because from the program’s point of view that is just cosmetic. Rendering should be separate from logic in a well designed game. Determinism means the same input creates the same output. Iff players are synchronized they have the same game state. If they receive the same instructions and process them the same way, a deterministic process will give them the same output. If synchronized players make the same deterministic updates, they remain synchronized at the end of the update. Problem (mostly) solved. (The trick is making sure everyone has the update instructions.)

So if you want that are synchronized to stay synchronized after processing other players’ commands, you wait until everyone receives an update and make sure they all get the same thing. I think this requires an ack system, which TCP uses but you will want to make your own anyway. Meaning you should use UDP.

You want the game to make updates at a fixed rate. 10 to 20 is enough for an RTS game. Lower might be possible. Higher may or may not be better but will be harder to accomplish reliably. If you do this, you will give every player enough time to make computations and reduce network speed requirements. Meaning fast or slow computers and fast or slow connections can participate.

Know that you will never prevent cheating. You can prevent many hacks by using synchronization with or without a server with no extra work. You won’t be able to prevent players from seeing more than they are intended to see (reveal enemy spies hacks), but will your players have the incentive to do that? You could also use a telephone to coordinate with an ally, share a computer lab with your teammate, have a friend hover over your shoulder to point out things you miss, or get your grandparent to play on your behalf. Some people call this cheating, others don’t. Having unbalanced weapons or glitches will be worse for you than these cheats. If you think you need a server to prevent cheats, you do not have a good synchronization system. Your server’s ability to prevent cheats is only as good as the synchronization system. If you have a “prevent cheating” mentality prioritized over the “make a fun game” mentality, you’re focused on the wrong thing.

#9

I feel obligated to respond to this because I essentially just finished networking my engine and it is designed for an RPG\RTS game in mind.

Essentially I have used the Observer Pattern to observe my entities in the world - this was implemented before I started working on the networking (it is a good pattern to have for your entities in general for a lot of tasks.) The observers are notified when an Actor leaves\enters the world or when a Character acquires an item, begins and ends attacking something, when some entities health changes, when they are moving to a new world node etc etc.

So the server runs the game as would any other client using a common library (JevaCore) and extentions (JevaRpg) the only difference is the server registers observers that will process these observations made to its world and broadcast them to clients. Like-wise, the client registers observers for its assigned player Character and notifies the server when it has changed their inventory, attempted to move them somewhere etc.

With this general design you get a very minimal but powerful implementation on both the server and client side with virtually zero boilderplate code since it is essentially just building off of the JevaCore engine and the JevaRpg extention library. It is quite elegent because you don’t put a foot-print on your engine\game with a bunch of networking and synchronizing code. Networking, Game and Engine code are all neatly seperated.

I got this done with ~9 small class files in the server (and the same amount in the client.) - To be fair, they share a common networking infrastructure beneath that is a couple more classes though but it doesn’t concearn the game\engine at all.

For networking I queued all outgoing\incomping messages into a ‘snapshot’ and dispatched these snapshots at a fixed interval (200ms). The advantage to queuing them up is that it reduces networking traffic. If you are sending many messages you get a much better compression ratio if you compress them in groups rather than individually - to state the obvious.

I’ve been using TCP/IP - depending on how intensive your game is you may want to use UDP. The problem with UDP is that isn’t reliable for order and packets can be dropped. This is ideal for an FPS game for example because if you are transmitting the speed, velocity and acceleration and you miss a packet, the client can probably survive off client-side prediction while it waits another 50-100ms for the next packet.

RTS Games can transmit synchonizations on a much more abstract level and thus the data inside of them is usually valid for a longer term (i.e, traversing path nodes) rather than something as variable as acceleration and velocity. This is why usually for RTS games concise messaging is important and due to the simplicity of the messages the overhead induced via TCP\IP is minimal and unavoidable anyways (because you can’t just throw away a message like ‘travel these nodes’ where as throwing away something like ‘set velocity’ (when the next velocity is ms away) is compeltely safe.

#10

P2P lockstepping isn’t really part of this topic and it still requires synchronization of the turns/timesteps of the game. Even so you would most commonly find a server in there somewhere for instance when starting the clients at the same time and other tasks such as match ups.

#11

The main thing I would say is don’t solve already solved problems. Google Netty, Apache Mina, Grizzly, and Akka for starters.

I’ve been writing network code for around 15 years, and there is just no reason to write your own networking for a server from scratch.

Non blocking io is pretty much the standard now days. Finding a modern server that doesn’t use non blocking io, especially in the java world, is pretty rare. That’s primarily because it would be silly to write a networking framework that limited itself to just a few hundred connections at most, so no one does it anymore.

Chris