networking libraries

Hi all!

Im going to do a project for school in which i’ll do some research on various client synchronisation / prediction techniques. Since i’ll (try) to implement various demo’s demonstrating the effect using different network models (client/server, peer to peer) and protocols (tcp/ip & udp, i’m searching for a generic library that saves me from developing the network communication part.

Below are some libraries I found, but would like some feedback on:

Mina: http://directory.apache.org/subprojects/network/features.html
The feature list sounds complete, but Im wondering about it’s performance (wont all these abstractions and generic handlers make things slower?) Does anyone have expirience using it in games?

JNAG: https://jnag.dev.java.net
The description does sound very usefull, but it doesn’t seem to be a very active project :wink:

JEnet:
I like the idea of being able to send some reliable packets, but when I would choose to use Mina there’s probably no easy way to combine Mina and JEnet?

ODE networking: https://odenetworking.dev.java.net/
Im also interested in looking at this library, as it involves synchronizing object(physics) states over clients, which is something I want to investigate too. However the project space is empty. Neither can I access the SVN repository the author linked in this message: http://www.java-gaming.org/forums/index.php?topic=10733.0

Anyone got any expirience / thoughts to share on this?

Thanks!

Martijn

Anyone ???

My only comment is that socket code really isnt that hard. shrug

And if you are planning onm comapring techniques do you really want unknown processing handling in the middle?

MINA is cool! It uses NIO as underlaying protocol. Seems pretty fast, though I didn’t
do a speed test.

[quote]My only comment is that socket code really isnt that hard. shrug

And if you are planning onm comapring techniques do you really want unknown processing handling in the middle?
[/quote]
Yes you’re right, I’ll try to implement my own nio server then. The real work will be in the game protocol and how to sync states… i think…

Sorry for the delay. JNAG is involving slowly because I do some other things at the same time.
I wouldn’ t recommend you to use it until it is more developed.

I also use MINA in some little project.
It’s an interesting library if you would like to build a custom protocol layer over TCP/IP.
Actually I can say it have some limits due to the fact that isn’t so stable; documentations isn’t clear (sometimes is simply too old) and there isn’t a “state framework” that help to build a statefull client/server interaction (not a problem if you don’t need it… and nothing soo difficult to write).
Do not seems to have speed issue (of course, I use it only in little environments).
I think is a good idea to take a look about it.

Check out this post:

http://www.java-gaming.org/forums/index.php?topic=12384.0

-Matt Hicks

Also, the ODENetworking project has been abandoned in favor of jME-Physics Networking (which is available for download at the JGN project as well). I originally created ODENetworking to create an abstract physics networking API for use with any games that utilize ODE but since I develop exclusively on jME and jME-Physics has some proprietary changes to ODE internally it wasn’t possible to do this and remain compatability with jME-Physics. I have been seeking someone to take over the ODENetworking project and port jME-Physics Networking to ODENetworking, it technically shouldn’t take too much effort, I’m just not willing to take the time to do it.

-Matt Hicks

I’d like to advertise my HeadQuarter system a bit. (http://cvs.sourceforge.net/viewcvs.py/drts/projects/headquarter/).

Here are some facts

  • based on NIO
  • binary message format
  • send, receive and distribute any kind of message (package objectbus)
  • transparently maintain shared states (spatial data, properties, hierarchies: types/groups/…)
  • clear idea of shared ‘identity’
  • notion of ‘realtime’
  • services like clock synchronization
  • currently actively developed
  • BSDL

Herkules,

No offense, but looking at that API it looks incredibly more complicated. I intend to look into it further though.

I would be very interested in some constructive feedback on my API as it’s still pretty new and being developed.

-Matt Hicks

No offense. But let me learn. What do you consider to be complicated?

Sorry, I really wasn’t trying to offend, I just came in to offer an alternative and respond with some constructive criticism. I’m happy to explain:

Your SimpleChatClient for example, I kind of ripped some code but I think I’ve got the basic gist of it here:

NetStation mStation = new NetStation();
BusLine mLine = mStation.createLine(address, port);
ChatClient mChat = new ChatClient(mStation, chatId);
mChat.addListener(new ChatListener() {
	public void newMessage(ChatMessage msg) {
		System.out.println(msg.getSenderName() + ": " + msg.getMessage());
		mChat.clear();
	}
});

mChat.send(mChatName, "Hello Chat!");
mStation.flush();

That to me seems a few too many things to keep track of. Further, the ChatClient is part of the project itself, so assuming this was a custom implementation where they were creating this from scratch it would also need something similar to the ChatClient implementation I’m assuming:

public class ChatClient
{
    private final BusStation            mStation;    
	private final Identity				mProtocolID;
    private final ChatReceiver          mReceiver           = new ChatReceiver( this ); 
    private final ArrayList             mListeners          = new ArrayList();
    private final BusTicket             mTicket;
    
    
    /**
     * Creates a new instance of Chat.
     */
    public ChatClient( BusStation station, Identity protocolID )
    {
        mStation	= station;
		mProtocolID	= protocolID;
		mTicket		= new BusTicket( mProtocolID );

		mStation.add( mProtocolID, mReceiver );
    }

    
    /** 
     * Shutdown the chat.
     */
    public void close()
    {
        mStation.remove( mProtocolID, mReceiver );
    }
    

     /**
     * Get this list of messages that came in after latest <code>clear()</code>.
     */
    public Iterator getIncoming()
    {
        return mReceiver.getIncoming();
    }
    
    
    /**
     * Release list of incoming messages.
     */
    public void clear()
    {
        mReceiver.clear();
    }

    
	/**
	 * Get number of incoming messages.
	 */
    int getMessageCount()
    {
        return mReceiver.getMessageCount();
    }
    
    
	/**
	 * Retrieve a certain message.
	 */
    ChatMessage getMessage( int idx )
    {
        return mReceiver.getMessage( idx );
    }
    
    
    /** 
     * Send message anonymously.
     */
    public void send( String text )
    {
        BusTicket ticket = createTicket( ChatConstants.SENDER_IS_ANONYMOUS );    
        ticket.putString( text );
        sendTicket( ticket );
    }
    
    
    /**
     * Send message giving the name of the sender.
     */
    public void send( String sender, String text )
    {
        BusTicket ticket = createTicket( ChatConstants.SENDER_IS_NAME );
        ticket.putString( sender );
        ticket.putString( text );
        sendTicket( ticket );
    }

    
    /**
     * Send message giving the <code>Identity</code> of the sender.
     */
    public void send( Identity sender, String text )
    {
        BusTicket ticket = createTicket( ChatConstants.SENDER_IS_ID );
        ticket.putID( sender );
        ticket.putString( text );
        sendTicket( ticket );
    }


    private final BusTicket createTicket( byte sendertype )
    {
        mTicket.clear();
        mTicket.putByte( sendertype );
        return mTicket;
    }
    
    private final void sendTicket( BusTicket ticket )
    {
		mStation.broadcast( ticket, null );
    }
    
    
    // ---------------------------------------------------------------------------------------------
    //
    // Listener handling
    //
    // ---------------------------------------------------------------------------------------------
    
    public void addListener( ChatListener l )
    {
        mListeners.add( l );
    }
    
    public void removeListener( ChatListener l )
    {
        mListeners.remove( l );
    }
    
    void notifyNewMessage( ChatMessage msg )
    {
        int cnt = mListeners.size();
        for ( int i = cnt-1; i >= 0; i-- )
        {
            ((ChatListener)mListeners.get( i )).newMessage( msg );
        }
    }
        
}

Looks like complication to me. :-p

Here’s an example using JGN for the client aspect assuming there is a server to respond (I make the same assumption for yours):

NetworkingClient client = new NetworkingClient();
if (client.connectAndWait(address, port)) {
    Thread t = new Thread(client);    // This is simply for convenience, in most cases you would call client.update() in your update method instead - it handles sending Noop to the server to let it know its still connected
    t.start();
    client.addMessageListener(new MessageListener()) {
        public void messageReceived(Message message) {}

        public void messageReceived(ChatMessage message) {
            System.out.println(message.getSenderName() + ": " + message.getText());
        }
    });

    // Send a message
    ChatMessage message = new ChatMessage();
    message.setSenderName("Me");
    message.setText("Hello Chat!");
    client.sendToServer(message);
}

EDIT: Oops, forgot to include the best part, the ChatMessage:

public class ChatMessage extends CertifiedMessage {
    private String senderName;
    private String text;

    public void setSenderName(String senderName) {
        this.senderName = senderName;
    }

    public String getSenderName() {
        return senderName;
    }

    public void setText(String text) {
        this.text = text;
    }

    public String getText() {
        return text;
    }
}

Cuts down on the learning curve when it’s straight-up beans you have to create. :wink:

-Matt Hicks

Ok, I see the point.

There is some boilerplate code for these reasons:

  • This chat implementation can maintain any number of chat channels. e,g. 1 working C/S sending messages to all and another P2P. This needs some setup.
  • The chat client buffers messages for most client won’t be able to deal with asynchronously arriving chat messages.
  • chat message senders can be anonymous, a string or an Identity (preferred, who says there has to be a name?). I’d normally assume that a players name is stored as a shared property in the Property subsystem and does not need to be transmitted with every single message.
  • Networking or network message formats don’t shine thru to the application. In fact, a HQ application doesn’t really know wether it works across JVM boundaries or not.

So the (application) code to compare is:

mChat.send(mChatName, "Hello Chat!");

against

    ChatMessage message = new ChatMessage();
    message.setSenderName("Me");
    message.setText("Hello Chat!");
    client.sendToServer(message);

The long code you cite basically corresponds to your source of ‘ChatMessage’ plus some bound container buffering messages.

Does Headquarters support UDP or just TCP?

-Sam

HQ is TCP only for performance reasons. :slight_smile:

Until now, there is no message that may get lost and order is important (or at leas useful)…

HQ heads for minimal network traffic which almost always requires reliability. This issues are discussed in other threads I think.
Implementations using other protocols are possible but currently not on the @todo list.

How ironic, JGN is entirely UDP for performance reasons. :-p

UDP is significantly faster than TCP and much lighter. I agree, the disadvantage to it is not having guaranteed delivery and not guaranteed order. However, I have resolved these problems with CertifiedMessage (guaranteed delivery over UDP), and OrderedMessage (which extends CertifiedMessage and also guarantees the order of delivery).

As for your statement of my creation of ChatMessage being much longer than yours, that is just not true. You have one line of code there, but under the hood you do this:

BusTicket ticket = createTicket( ChatConstants.SENDER_IS_ANONYMOUS );   
        ticket.putString( text );
        sendTicket( ticket );

If I were to count (which I have been known to do on occasion - hehe) I think I come up with the same number. :wink: Sure, I could create a little static method in ChatMessage sendMessage(String sender, String text) and that would offer the same benefit. I wrote it this way to show how much more simplistic utilizing JGN in the same example is.

I’m not looking for competition though, I’m looking for cooperation. My hope is that you’ll see the advantages of JGN and hopefully want to help out. I have no desire to maintain the project all by myself and the goal is to provide a standard that the community can rely on.

-Matt Hicks

I just didn’t dare to create a reliability protocol on my own. I was afraid of having to send too many UDP packets forth and back destroying the UDP advantage quickly while not being able to profit from vJ compression.

So HQ could operate with CertifiedMessage. But as long as TCP performs well esp. on thin-line scenarios…

Well, I created JGN and PhysicsNetworking (an extension of JGN for physics synchronization in games) primarily focused towards first-person shooter style games where speed is key.

I have been considering adding the ability in JGN to use TCP as well, I just haven’t had a need since the extensions I’ve written on UDP seem to fill all of my gaps. The real place where UDP shines is when you’re sending messages that are real-time messages like, “this player is in this location with this rotation”. I also introduced something called a PerishableMessage that is not guaranteed delivery. In fact, if it doesn’t get to its destination in a timely manner it gets rejected entirely. The whole send and forget ideology works extremely well with UDP. This is particularly useful for my PhysicsNetworking that sends PhysicsMessages that simply get sent 10 times per second and if one gets lost it doesn’t matter because there’s another one right behind it more correct anyway.

I did quite a bit of research into this when I first started looking at designing an API and the reason I decided to write my own was because all the fast-paced games seem to use UDP and nearly every API in Java I’ve found uses TCP.

BTW, packet compression is on its way in the near future. :wink:

Features done or coming soon: http://www.captiveimagination.com:88/wiki/index.php/Features

-Matt Hicks

The discussion slips into the UPDvsTCP direction again so I will not continue that after this final comment on HQ’s philosophy here.

HQ won’t send messages that could be dropped. The real place where TCP/reliability shines is when you’re sending messages that are real-time messages like “this player is in this location with this rotation”. HQ uses dead-reckoning to only send the bare minimal amount of data. For dead-reckoning, you compare the actual local position with an estimated remote position. In order to be able to do a valid estimation, guaranteed, timestamped message delivery is mandatory. This way, when moving in a predictable way, NOTHING is send over the line which in all situations is better than any however-optimized message format.
Sending 10xsec brutally always was a no-go for HQ for it overcommits any line introducing higher packetloss and additional latency. You just cannot serve even a handfull of objects (HQ doesn’t have a term like ‘player’) on an ISDN line with an update rate of 10/sec. In FlyingGuns e.g., 1-2 updates/sec occur although the dead-reckoning treshholds are much tighter than necessary for a feasable gameplay.

Message compression does not make much sense for HQ, for currently all message formats are densely packed and (in particular) not self-descriptive thereby avoiding any overhead. The clean notion of Identity prohibits redundency. vJ just drastically reduces TCP overhead (from 40 to 3 bytes or so) whereas it cannot do anything about UDP overhead (20bytes). Everything in HQ is heads for not sending data on high-level in favor of sending it effectively on low-level.

Ok now, for me this is the end of TCPvsUDP before blah³/persson kills us here for being off topic.

My original concern was to point to HQ as an information backbone that is able to deal with shared states over a network. (BTW, HQ does not NEED to work over a network, it can be used locally as well. When FlyingGuns is configured to hold the ‘server’ within one client, this client and server don’t even use loopback to communicate saving a lot of overhead again). HQ is totally agnostic of running C/S or P2P or both. Besides it’s basic infrastructure, it offers proven out-of-the-box services like clock sync and some specific shared states. The basic design allows for many improvements (high-level protocols, range-based services, automatic routing, object ownership, access rights…) that pend to be implemented.