Networking techniques

Newbie & Debugging Questions
#1

Howdy.

I do not make anything that is networked… ever. I want to change that but when I do networking my packets are just strings with a syntax.
When I see other peoples protocols however, it seems they are sending different bytes - which would obviously be a lot faster, and lighter if that is in fact how it works.

Now to my question:
Is there any articles/books (anything!) that tells about this topic for Java applications? I really want to learn about this, but I want to make sure I do not read something outdated, or something bad. :slight_smile:

#2

If you want to get something done, keep it simple.

If strings do the job for you, and you don’t run into trouble, stay with strings. Binary network APIs can be a huge time saver or a massive time sink.

Especially if you are just starting out, doing networking with strings is just fine. You should get a feel of the problems to solve, which will come in handy when you start using network APIs later.

A poor example maybe, but you (should) start Java programming with command-line apps, to get a feel for the language, not dive into Swing for the first few months.

#3

Thanks for your response! :smiley:
I have made some things (not a lot though…) that uses strings in packets, but I really want to learn to use both.

About the last statement: I almost only create command-line applications :smiley: It’s only for my own eyes, and working with swing is so… unpleasing :slight_smile:

#4

I’ve been using ObjectOutputStream. I made an Interface called GameMessage which has methods for various calls any message might need, like isHeartbeat() (a heartbeat message is not sent to client, but keeps the server from disconnecting the client) isEchoed() (an echoed message is immediately sent back to every client as long as it’s valid, useful for chatrooms etc.) isDisconnect() (sent from a client when they’ve disconnected) etc. Then I make any sort of class and as long as it’s Serializable and implements GameMessage it can be sent around no problem.

If you’re using bytes, the process is pretty much the same as sending objects or strings, you just need some way of determining what’s being sent and sending it in a reasonable way. Typically you have a certain byte value that indicates the start of a new message and/or have one to indicate the end of a message. As you receive bytes, you buffer them into a byte array, then when you’ve got a complete message you interpret what it is from your bytes. Complication here arises from needing to restrict data bytes from equaling the start/end byte values. If they do, you’re borked. To remedy this, You can also just assume that every message is X bytes long, read in that many bytes, then figure out what it means. In this case, however, you might as well just send an object or a string (in my opinion) because you’re wasting enough bytes that you’re losing the advantage of the byte stream.

#5

If you use Serialization, how are you going to stop me from hijacking all your heap memory by sending a bogus byte-array size?

#6

Ignoring that java.util.Serialization sucks… You definitely want to go with writing a length. This in no way wastes enough bytes that it warrants just sending strings. Sending a length takes only 1 byte for small messages and 2 bytes for nearly any other messages.

That said, it usually doesn’t matter what you are sending over the wire. Abstract that away so you send an object and get an object on the other side. Then you can switch out your serialization without affecting your app. Step #1: get something working.

#7

I have no idea. :slight_smile:

The way I set my server up for the moment though that’ll just lock up your own client thread and still leave the server capable of handling everything else while it tries to pull in your junk. But I guess it would indeed eventually break the heap. Is there any way around this, or should I just stick with fixed byte sizes?

#8

It’s the same old security refrain: don’t trust the client. Have some sanity checks on incoming data size, I would guess.

#9

Another option is 3rd party serialization libraries like Google P Buffers. This is what I considered using last time I did networking.

#10

Yeah but since I’m using ObjectInputStream there is no way of just cutting off the stream halfway through. It will start reading the next object and won’t stop until it’s totally read.

#11

Read length, close connection if length is too large, receive TCP data until we have that many bytes, present bytes to java.util.Serialization, close connection if serialization fails.

This! Or, somewhat more specifically, this! :slight_smile:

#12

I don’t know any, but allow me to share what I’ve learnt the hard way. (Not always through my own errors - I’ve had plenty of pain fixing broken protocols written by other people too). These rules apply equally to network protocols and file formats (for level data, save files, etc).

  1. Always have a header which includes a version number. Even for version 0.0.1. Otherwise when you make a breaking change to the protocol/format you will have major problems.
  2. If a field is of variable length, either encode the length and write that first (my preferred option) or ensure that they are self-limiting. For strings this means that you have a delimiter character and an escaping mechanism. For integers which are usually small, study UTF-8’s approach.

There’s more to good network protocol design than this, but getting these basics wrong will hurt you.