[Edit: latest source and discussion here: http://www.java-gaming.org/index.php/topic,18019.0.html, in 'Game Showcase under title ‘Multiplayer top-down view shooter’]
Dear Developers,
I’ve made a pair of custom Serialization streams to combat 2 important Serialization problems in a networked game I’m trying to build.
Problem 1: new Objects are created when deserializing the game world 40 times per second. This causes the Garbage Collector (gc) to periodically hog processor time (15% GC tick count) to cleanup the redundant objects. When this happens in the middle of a screen paint (rendering takes 75% of processor time), the game appears to lag badly, making the whole application appear to ‘chug’ as the gc kicks in about every second.
Problem 2: on the client VM, since the GameWorld object is recreated all the time, the client has no reference to its own up-to-date Player object, it has to search for its name field in the list of Players. Similarly there are other things that I want to keep a reference to on the client without having to give names to everything and looping through lists to find the right Object.
These problems led me to think - why not have a Serialization class that just updated objects by replacing their primitives but not the whole object? In other words, why does deserialization have to create new objects when they already exist?
So that’s what I made, and I called it SuperSerialization (SS). The code for the new SS streams is posted at:
http://www.slavebot.net/SuperSerializable/SuperSerializableIntro
The ‘SuperSerializable’ (SS) streams are made for network computer games where the developer wants to update the client’s game world so that it is the same as that on the server. The SS streams allow you to update objects rather than replace them. Now we can send our whole game world over the network without having to recreate the object in deserialization.
How Does It Work?
The four key classes are SSObject, SSAdapter, SSObjectOutputStream and SSObjectInputStream. They are not very big classes (< 350 lines each) so I recommend that you look at the code, but I’ll try to explain how it works.
The SSObjectOutputStream extends DataOutputStream. It defines two key methods writeSS(SSObject sso) and writeDone() and clearStoredExceptFor(SSObect sso). SSObjectInputStream works similarly, with readSS(SSObject sso) and readDone().
The SSObjectOutputStream can not update normal Objects. It can write some types of Objects (see attemptWriteNonSS()), but it can only update objects that implement the interface SSObject (the object must also have a no-argument constructor). The SSObject interface defines methods getCode(), writeSS(SSObjectOutputStream out), readSS(SSObjectInputStream in) and collectMemberSSCodes(ArrayList objectCodes). The getCode() method returns the unique integer code that is used to identify it. Without this unique identifying number, on the client side we couldn’t find the unique object that needs updating. This is why the SS streams can’t write normal objects - their hash codes are not unique so two different objects can’t be discriminated. writeSS and readSS are just like normal Serialization’s writeObject and readObject - they are called to write/read the object’s data to/from the stream.
Since all SSObjects read in SSObjectInputStream are stored in the HashMap storedObjects, when these objects are no longer used they need to be recycled and put back in spareObjects, so collectMemberSSCodes(ArrayList objectCodes) is used to find out what SSObjects need to be kept in storedObjects. Any objects implementing SSObject need to be able to find all SSObject references that the object contains and add their code to the list to avoid having these SSObject members recycled.
There is an adapter class SSAdapter that implements SSObject and calls the SS streams’ default write and read methods as well as the default collectMemberSSCodes method. These methods use the Reflection API to find the SSObject’s specific fields and then writes/reads them to/from the stream. Any that you don’t want written you can mark with the transient modifier, just like normal Serialization. Similarly, static fields are not written.
How Can I Use It?
To use the SS streams you just get your game objects to extend SSAdapter and have a no-arg constructor. On the server, your game loop would look like this:
ByteArrayOutputStream byteOut = null;
SSObjectOutputStream objectOut = null;
byte[] bytes = new byte[0];
try{
byteOut = new ByteArrayOutputStream();
objectOut = new SSObjectOutputStream(byteOut);
} catch(IOException e){}
GameWorld gameWorld = new GameWorld(); // GameWorld extends SSAdapter
while(true){
try{
gameWorld.update(); // do the game logic
byteOut = new ByteArrayOutputStream();
objectOut.setOutputStream(byteOut);
objectOut.writeSS(toBeWrittenObj);
objectOut.writeDone();
bytes = byteOut.toByteArray();
// then send the bytes over the network
}catch(java.io.IOException e){}
}
On the client, it would look like this:
ByteArrayInputStream byteIn = null;
SSObjectInputStream objectIn = null;
byte[] bytes = new byte[0];
try{
byteIn = new ByteArrayInputStream(bytes);
objectIn = new SSObjectInputStream(byteIn);
} catch(IOException e){}
GameWorld gameWorld = null;
while (true){
try{
bytes = …; // Get the bytes sent from the server
byteIn = new ByteArrayInputStream(bytes);
objectIn.setInputStream(byteIn);
gameWorld = (GameWorld)objectIn.readSS();
// after the first objectIn.readSS(), gameWorld is created.
// Thereafter, it is only updated but a reference is still returned.
objectIn.readDone();
objectIn.clearStoredExceptFor(gameWorld );
// gets rid of and recycle old SSObjects that are no longer referred
// to in the GameWorld so that they don’t cause a memory leak.
// This can be done every so often, not needed after every readSS call
// now paint the gameWorld etc.
}catch(java.io.IOException e){}
}
Complications
This works fine when all of our game objects extend SSAdapter, but all of their fields and those field’s fields, etc must extend SSAdapter. For objects that can’t extend SSAdapter you need to sub-class them and implement SSObject. My game world contains ArrayLists so I sub-classed it in ArrayListSS (see the posted code).
This works fine, now my ArrayListSS’s can be updated. But some classes are final like String. For these, we have to use SSObjectOutputStream.attemptWriteNonSS() and have a custom way of writing the string. These Strings won’t be able to be updated, they will be newly created with each readSS(), just like normal Serialization, but worse. Worse because of this: say you store a player’s name in a list in the GameWorld, and also in the Player object. On the server, these two references are to the same object, so name == player.name. But when the client updates the GameWorld with readSS, name .equals(player.name) but name != player.name since the references are not to the same object.
This problem is not serious however since with careful design it can be overcome. For example, if the GameWorld stored the list of Players rather than just name Strings, the two names will be == since their Player objects would be ==.
In my version of SSObjectOutputStream.attemptWriteNonSS(), arrays, Strings, and Colors can be written but have the above problem. Having too many of these objects in your game world will cause big GC pauses since they are newly made with each readSS method call.
Note that over-riding SSAdapter.writeSS() and readSS() makes for faster Super Serialization since no reflection is needed. If you do over-write these methods, note that it is only called once on that object, so you must write all fields defined in this object’s class-level as well as all super-class levels (unlike in Serialization’s writeObject and readObject). Also, for any SSObject fields that are members of the object being written, you should call SSObjectOutputStream.writeSS(theObject) on them.
Garbage Collection
For my game world, -Xprof showed that garbage collection went from 15% (regular Serialization) to 5% (SS) and that is with a GameWorld that had 293 SSObjects and 43 normal non-SSObjects written/read. If there were fewer of these Objects or with a better way of transfering them, garbage collection could near 0%. Unfortunately the 5% GC tick count is still far too high and makes the application appear laggy as the GC chugs into action about once a second.
This leads to my second problem: can we do something more efficiently with the non-SSObjects to update them without re-creation on every call to readSS()?
Thanks, any help would be appreciated.
Keith
(I’ve also put up this post on the Java Serialization Forum http://forum.java.sun.com/thread.jspa?threadID=675403&tstart=0)
