[SOLVED] What's a Packet in Java?

Networking & Multiplayer
#1

Hi,

With Java Sockets you send and receive messages through the InputStream and OutputStream classes. The most basic tutorial reads lines of strings from these streams (as in a chat application or something).

What if you would like to send a bunch of primitive data types? One would think that wrapping the Streams into a DataInputStream and DataOutputStream would work - but here’s a problem. Each time you write to these streams the message is sent directly! You can’t bunch up a, say, 1 Byte and 2 Integers and a String ( in that order ) and then send the ‘package’ as whole through the Sockets Streams.

What I’ve been accustomed to from another language, is that you build a ‘package’ first, and then send the package - like this f.ex:


clearbuffer();
writeByte(1); // Used to identify what sort of message this is
writeInt(x);
writeInt(y);
writeString("Teddybears");
sendMessage(socket.outputStream);

This package would be extremely light as it would only be the size of the bytes that is sent.

In Java, I’ve noticed people sending whole Objects through Serialization. Which, to me, seems highly unnecessary - and a lot more heavy in size f.ex:

class PositionMessage() implements Serializable {
int x;
int y;
String msg;
}
...
PositionMessage msg = new PositionMessage();
msg.x = x;
msg.y = y;
msg.msg = "Teddybears";
// code to serialize the msg ( turn it into bytes )
sendMessage(socket.outputStream);

and to identify what kind of message is sent you use the ‘instanceof’ keyword, and for each different type of message you create a new class.

I would think that this Java solution, while it might work, creates a lot more unnecessary weight in my packets and that there should exist a more controlled way, like the former solution, in Java. Is there?

#2

Regarding not sending data immediately:

  1. use BufferedOutputStream and .flush() when you’re done.
  2. use ByteArrayOutputStream and .toByteArray() when you’re done.
#3

o.O --> re-reads about Buffered Streams.

Looking closer at BufferedOutputStream and BufferedInputStream, they both write and read bytes of data… which is good. How would you convert these into primitive data types?

I’m assuming you would use a Scanner for this, but how would you know in what order the bytes are supposed to be read if you can’t figure out the message ID that… that we put in front of every single message as a byte value… kind of answered my own question there :stuck_out_tongue:

So to lay it all out, would this be a suitable design in Java, using Scanners

Scanner s = null;
try {
s = new Scanner (
               new BufferedInputStream ( socket.getInputStream() );
byte b = s.nextByte(); 
     /* we know the first value is going to be a byte that is going to function as the ID 
         to identify what kind of message it is and therefore we will know what values 
         to read from it further down*/
if (b == 1) readPosition(s);
} finally {
if (s != null) s.close(); // close the scanner
}

// Methods
readPosition(Scanner s) {
int x = s.readNextInt();
int y = s.readNextInt();
String str = s.readNextString();

// do stuff with the information
}

I haven’t done any tests, but to me this looks like it might work. Would you agree?

#4

Mix 'n match…

DataOutputStream out = new DataOutputStream(new BufferedOutputStream(socket.getOutputStream()));
DataInputStream in = new DataInputStream(new BufferedInputStream(socket.getInputStream()));
#5

Scanner is slow, use BufferedReader.
As for writing, you can wrap an ObjectOutputStream around the output stream since it can write primitives, strings, and objects.

EDIT: You can also take a look at my networking library, which uses a Packet system and handles the sending for you.

#6

The BufferedReader can’t read primitives though. The DataInputStream ( and Scanner ) does however… so changing the Scanner into DataInputStream…


try {
DataInputStream din = new DataInputStream (
               new BufferedInputStream ( socket.getInputStream() );
byte message_id = din.readByte();
     /* we know the first value is going to be a byte that is going to function as the ID
         to identify what kind of message it is and therefore we will know what values
         to read from it further down*/
if (message_id == 1) readPosition(din);
} catch (IOExceptione ){ }
 
// Methods
readPosition(DataInputStream din) {
int x = din.readInt();
int y = din.readInt();
String str = din.readUTF(): // I assume this reads a string?

// do stuff with the information
}
#7

Part of my buffer class:


import java.nio.ByteBuffer;
import java.util.Arrays;

public class Buffer {

	public Buffer() {
		this(128);
	}

	public Buffer(int size) {
		buffer = new byte[size];
	}

	private byte[] buffer;
	private int readPos = 0, writePos = 0, sizePos = 0;

	public byte[] array() {
		return Arrays.copyOf(buffer, writePos);
	}

	public int get() {
		return buffer[readPos++] & 0xFF;
	}

	public int getSigned() {
		return buffer[readPos++];
	}

	public int getUShort() {
		return buffer[readPos++] << 8 | (buffer[readPos++] & 0xFF);
	}

	public int getShort() {
		return (buffer[readPos++] & 0xFF) << 8 | (buffer[readPos++] & 0xFF);
	}

	public int getInt() {
		return (buffer[readPos++] & 0xFF) << 24 | (buffer[readPos++] & 0xFF) << 16 | (buffer[readPos++] & 0xFF) << 8 | (buffer[readPos++] & 0xFF);
	}

	public long getLong() {
		return (buffer[readPos++] & 0xFFL) << 56 | (buffer[readPos++] & 0xFFL) << 48 | (buffer[readPos++] & 0xFFL) << 40 | (buffer[readPos++] & 0xFFL) << 32 | (buffer[readPos++] & 0xFFL) << 24 | (buffer[readPos++] & 0xFFL) << 16 | (buffer[readPos++] & 0xFFL) << 8 | buffer[readPos++] & 0xFFL;
	}

	public double getDouble() {
		return Double.longBitsToDouble(getLong());
	}

	public int getLEShort() {
		return (buffer[readPos++] & 0xFF) | (buffer[readPos++] & 0xFF) << 8;
	}

	public int getLEInt() {
		return (buffer[readPos++] & 0xFF) | (buffer[readPos++] & 0xFF) << 8 | (buffer[readPos++] & 0xFF) << 16 | (buffer[readPos++] & 0xFF) << 24;
	}

	public String getString() {
		StringBuilder bldr = new StringBuilder();
		char c;
		while ((c = (char) buffer[readPos++]) != 0)
			bldr.append(c);
		return bldr.toString();
	}

	public int readPosition() {
		return readPos;
	}

	public int remaining() {
		return writePos - readPos;
	}

	public int writePosition() {
		return writePos;
	}

	public Buffer put(ByteBuffer buffer) {
		int len = buffer.position();
		ensure(len);
		System.arraycopy(buffer.array(), 0, this.buffer, writePos, len);
		writePos += len;
		return this;
	}

	public Buffer put(byte[] buf) {
		int len = buf.length;
		ensure(len);
		System.arraycopy(buf, 0, buffer, writePos, len);
		writePos += len;
		return this;
	}

	public Buffer put(byte[] buf, int off, int len) {
		ensure(len - off);
		System.arraycopy(buf, off, buffer, writePos, len - off);
		writePos += len - off;
		return this;
	}

	public Buffer put(int b) {
		ensure(1);
		buffer[writePos++] = (byte) b;
		return this;
	}

	public Buffer putShort(int s) {
		ensure(2);
		buffer[writePos++] = (byte) (s >> 8);
		buffer[writePos++] = (byte) s;
		return this;
	}

	public Buffer putInt(int i) {
		ensure(4);
		buffer[writePos++] = (byte) (i >> 24);
		buffer[writePos++] = (byte) (i >> 16);
		buffer[writePos++] = (byte) (i >> 8);
		buffer[writePos++] = (byte) i;
		return this;
	}

	public Buffer putLong(long l) {
		ensure(8);
		buffer[writePos++] = (byte) (l >> 56);
		buffer[writePos++] = (byte) (l >> 48);
		buffer[writePos++] = (byte) (l >> 40);
		buffer[writePos++] = (byte) (l >> 32);
		buffer[writePos++] = (byte) (l >> 24);
		buffer[writePos++] = (byte) (l >> 16);
		buffer[writePos++] = (byte) (l >> 8);
		buffer[writePos++] = (byte) l;
		return this;
	}

	public Buffer putDouble(double d) {
		return putLong(Double.doubleToRawLongBits(d));
	}

	public Buffer putLEInt(int i) {
		ensure(4);
		buffer[writePos++] = (byte) i;
		buffer[writePos++] = (byte) (i >> 8);
		buffer[writePos++] = (byte) (i >> 16);
		buffer[writePos++] = (byte) (i >> 24);
		return this;
	}

	public Buffer putLEShort(int s) {
		ensure(2);
		buffer[writePos++] = (byte) s;
		buffer[writePos++] = (byte) (s >> 8);
		return this;
	}

	public Buffer putString(String s) { // cannot use character 0
		ensure(s.length() + 1);
		for (int i = 0; i < s.length(); i++)
			buffer[writePos++] = (byte) s.charAt(i);
		buffer[writePos++] = 0;
		return this;
	}

	public Buffer end() {
		buffer[sizePos] = (byte) (writePos - sizePos - 1);
		return this;
	}

	public Buffer start(int opcode) {
		ensure(2);
		buffer[writePos++] = (byte) opcode;
		sizePos = writePos++;
		return this;
	}

	public void clear() {
		readPos = writePos = sizePos = 0;
		Arrays.fill(buffer, (byte) 0);
	}

	private void ensure(int size) {
		int required = writePos + size;
		if (buffer.length <= required) {
			byte[] temp = new byte[required * 2];
			System.arraycopy(buffer, 0, temp, 0, buffer.length);
			buffer = temp;
		}
	}

	public void rewind() {
		readPos = 0;
	}

	public void skip(int n) {
		readPos += n;
	}

}

writing objects sucks, and sending single packets also sucks, you should buffer them and send them after X period of time.

#8

Is no one going to recommend NIO? The psuedo-code that JonJava wrote as an example can be neatly implemented with a ByteBuffer. Why stick to the old IO library when ByteBuffers are so convenient?

#9

Haha, I was about to say that counterp’s Buffer class looks like java.nio.ByteBuffer.

@OP
At this stage, it is best for you to use java.nio.channels.SocketChannel as you can just use the ByteBuffer class to read and write. However, ByteBuffer does not have a way to write Strings so you will have to get the bytes from a string:


byte[] data = "this is my string".getBytes("UTF-8");
#10

Yeah, I guess you can compare it to NIO byte buffer, except mine is faster and no annoying flip method, it also doesn’t have some of the same methods, but it has support for signed and unsigned types (or as much support as you can get in java). You can also specify the size of a packet with mine using the start and end methods (if you have a packet with varying size and it is important that you know how big it is).

#11

I had a moderately (didn’t understand eveything) thorough (took me 2 hours) look through your library (the networking bits) - it looks very clean. Since I’m just now, as a result of this thread, getting to know the java.nio package it still is a bit over my head. But I found other outlets for tutorials and explanations, this blog post in particular was very informative for me since I was already accustomed to the java.io package and how the java.net sockets work.
http://blogs.oracle.com/slc/entry/javanio_vs_javaio

Thanks for pointing this out.

I was slightly unsure on how to read Strings, thanks for clearing that up. Of course the byte size of a string will vary but since I can identify what type of message is incoming I’ll be able to add the strings size in front of the string in the buffer as a byte value (for example) and that way know how many bytes to read.

The biggest difference between the old Socket from java.net and the new SocketChannel* from java.nio is that SocketChannel implements non-blocking mode. Which means that the program won’t “lock up” when trying to read for input like it would on the old java.net Socket (you would have to make a new Thread to listen for input in order to circumvent our program from locking up).

*The java.nio SocketChannels are built from the old java.net Sockets.

Here’s a brief revealing outtake from the blog post linked above:

[quote]SocketChannel

SocketChannel is different to FileChannel: The new socket channels can operate in nonblocking mode and are selectable. It’s no longer necessary to dedicate a thread to each socket connection, Using the new NIO classes, one or a few threads can manage hundreds or even thousands of active socket connections with little or no performance loss. It’s possible to perform readiness selection of socket channels using a Selector object.
[/quote]