[SOLVED] Kryo/Kryonet compression and buffer

Engines, Libraries and Tools
#1

Hi there,

I’ve changed around my whole network code to use Kryonet instead of gzipped strings. Unfortunately I ran into three connected issues, and I was wondering if there is a way to solve them.

  1. Whenever a player connects to the server the area around him is sent from the server, at the moment I’m sending an area of 2000x2000 tiles to give the player a view distance of 1.000m. While this never was a problem before the Kryonet TCP send bytebuffer gets filled up very quickly and the whole server crashes, never recovering. I can catch the error and restart the server, but I was wondering if there was a better way to handle it.

  2. The only way I managed to “solve” it was to start the server up with new Server(25 * 1024 * 1024, 1024 * 1024); Is allocating 25mb of buffer per connection really the way to solve such a problem? What about when people have created huge structures, do I have to go in and increase the number just in case the player loads that part of the map at startup?
    I could lower the amount of data sent somewhat or throttle the sending of the land, but I’d prefer to have it work without having to rewrite a lot of code and decrease the experience of the players.

  3. Back when I used gzipped CSV’s an empty container (an array of 4356 shorts) took about 500-700 bytes, and now they take 8kb. Is there a way to compress the data or does it have to take as much as each individual value? I was looking at the Deflate serializer, but I didn’t get it to work on a short[].

Kind regards,
Mike

#2

Wait what are you sending? I would just send a int for every tile and then the client does the work…
example
int tile = 1; so tile is grass
.
.
.
you understand… or make the player request the tiles… do not ever send that much of data it just if you do make it not playable until all data is send like 1000 ints per sending…

#3

I’m sending shorts to save space, one short per tile height (sending ints for types sounds horrible by the way :)).

Does anyone have any idea about the above queries?

#4

Why does it sound horrible?

#5

Because I highly doubt that you have 4294967294 different kinds of tiles. As it is a value that will be stored/sent a lot you should keep it to the smallest primitive type that you can.

Mike

#6

I’m still wondering about 3, anyone with any ideas?

As for 1 and 2: It became a whole lot better when I added a logic thread and didn’t send all those messages from the same thread that handles the (de)serialization. :slight_smile:

Mike

#7

Kryo compresses every single integer value during serialization. This apparently interferes with gzip compression.

#8

Every number separately? That’s… interesting :slight_smile:

Nate, any nice idea?

#9

Kryo uses varints for integer fields by default. Using varints means that 0 to 127 (or -64 to 63 if optimize positive is false) use 1 byte. Larger ints use more bytes, up to 5.

2k * 2k = 4M shorts = 8M bytes that you are sending. You can use DeflateSerializer to compress the data. Kryo doesn’t do any compression by default.

#10

If I change the register to (ContainerMessage is the message that contains all the short[], so I tried with and without that one just in case):

kryo.register(ContainerMessage.class, new DeflateSerializer(kryo.getDefaultSerializer(ContainerMessage.class)));
kryo.register(short[].class, new DeflateSerializer(kryo.getDefaultSerializer(short[].class)));

It doesn’t differ a single byte on the resulting size.

Mike

#11

As for the above question, it was a bug in Kryo that Nate just fixed. Thanks!

Mike