JNAG(Java Network API for Games) Project Approved!

[athomas]

The following project has been approved:

[quote]JNAG: Java Network API for Games

Message from the owner(s)

This project has been started already on my side. I made a client connection and a server connection, and also a stub-like compiler that is using reflexivity.

Actually, it works, but it is not finished.

I am planning to use the annotations of Java 1.5 to describe the network interface more precisely (with more semantic), so that the stubs will do more things and the programmers will have less things to do (= need less time to develop and produice potencially less bugs).

For the performance issue, I will use the compression methods described in the Game Programming Gems books + my own touch.

I also plan to add some utility classes that are using the main library. For example, they will be some predefined mother classes for lobby and chat components

Description

JNAG is a network API specialized for game applications.

It focus on the communication between 2 entities that are typically a client with a server, but can also be used between a client and another client.

The main goals of this project are :

* To match the common needs of the game programmers for network games.
* To make this library as convenient and easy to use as RMI is.
* To obtain the level of performance needed by professional online games.

[/quote]
Please see http://jnag.dev.java.net for details

[blahblahblahh]

1. How is it any better than RMI?
2. If it isn’t done entirely with hand-crafted UDP packets, then IMHO there’s no point at all, as far as games are concerned.

[phazer]

I think it’s very hard (maybe impossible) to create a network library as convenient as RMI which also minimizes the data overhead sent. For example to support polymorphism you must include some kind of type information in the data sent. RMI uses the class name which is natural and simple, but a gaming network library can’t afford this overhead. I typically use an integer to identify the type of object from a set of pre-specified types and then a switch statement to read the correct object from the stream. It’s fast and have very little overhead, but it’s not as convenient as RMI.

I’m not saying the project should not be approved, just that it’s hard to make it as powerful and easy-to-use as RMI.

[kevglass]

I don’t disagree, but that particular problem could easily be overcome with a “handshake” at the start in which the two sides of the connection agree on an integer mapping for classnames.

Kev

[phazer]

Yes, that’s probably a good solution, but how would you know at the start of the communication which classes that will be used in the communication? It is probably better that the sending part dynamically decide a mapping for a class when needed. Maybe RMI does this already, I don’t know.

Even though the library might not be as convenient to use as RMI, I still think it would be very nice to have a network library with RMI functionality, but optimized for speed and very little overhead. I’ve planned to do something similar myself, but it’s better to use something that already exists

Btw, does this library use normal Java serialization or some other custom serialization mechanism?

[kevglass]

Presumably they’ll add some radical new conformance interface/class that forces you to store the data you want to send across the network in a particular way… which of course make it harder to use than RMI.

Kev

[phazer]

With smart use of reflection and bytecode generation you can minimize the amount of manual work that has to be done for each serializable class, and at the same time minimize overhead and maximize performance. There is no need for version handling as found in normal Java serialization.

[blahblahblahh]

For now, “no”. It doesn’t seem to have a coherent point.

Is it trying to replace RMI? Why? For performance? …then it MUST be a cunning UDP scheme (or else not be for games).

RMI is actually “good enough” for a lot of games. And it’s free, extremely widely documented and tested, and comes with every sun JVM automagically.

[phazer]

RMI is certainly “good enough” for most games. But what if you have hundreds or even thousands of clients connected to a server? I don’t think RMI was designed for this. I have some concerns regarding:

• Serialization overhead
• The distributed GC
• RMI creating new sockets on demand

I think there is room for improvement over RMI even when using TCP.

[karmaGfa]

[quote]1. How is it any better than RMI?
2. If it isn’t done entirely with hand-crafted UDP packets, then IMHO there’s no point at all, as far as games are concerned.
[/quote]
Hello,

I am the one who propose this project.

1. It will not do the same things than RMI, but will be used in a similar way. Since they won’t do the same thing, you can understand than one can do better that the other one in what one doesn’t do.

2. At the beginning, I will use TCP, but for the feature concerning the semantic information of the data sent that I want to add later, I will have to use UDP.

For now, java programmers who don’t want to program with RMI for some various reasons have to implement them-self their network layer.

Why games programmers won’t use RMI ? I personnaly tried to use it, but I ran into some problems :

• I could not just send a function call and wait the return of the remote function. I could not also create a thread per new remote call since the number of calls were too numerous. In a game, the programmer often need a “fire & forget” remote function call, whose return type is void.
• I was curious, so I went to see the code generated by the stub … even when it is about to insert an int into a stream, they need an object, so they create a new Integer(value) before they can serialize it. I don’t know if it will affect the performance or not … but I felt a little … afraid.
• RMI was designed for the general purpose, and it makes it nicely. There is some assumptions that are definitly different between the general purpose and the games. For example in games, the network entities knows more about each other, since they all come from the same game development company. They can suppose more things about the behavior of the one that will receive the remote call, and it is usually used to improve the performances of the network.

So, if the programmer don’t want to use RMI, for the previously mentionned reasons, he have to make himself his own RMI-like framework with some solutions to those problems, or he have to give-up the ease to use of the RMI-like framework.

Here, I propose an open source API that will do that for the programmer. The API won’t support all the features of RMI, it just aims to be as easy as it to use.

I will also implement some components that are commonly used in games, as a demo of how to use the library, and as something that can be used directly by the game programmer.

Now, what my API will do that RMI won’t do is to add some special type of information, so that some data can be transmited between the entities in a more efficient way.

A few days ago, I saw your nice network engine. The API I will implement have different purpose. For now, it only deal with the communication between 2 entities.
… and this one will be LGPL, I want to enable everybody to use it for free, in order to promote Java for games.

Regards,
Vincent

Ps: I voted “abstain” when arriving on this thread … that’s more fair

[karmaGfa]

[quote]I think it’s very hard (maybe impossible) to create a network library as convenient as RMI which also minimizes the data overhead sent. For example to support polymorphism you must include some kind of type information in the data sent.
[/quote]
Usually, since each entity are implented by the same team, the entities can be prepared in advance to not have to use this feature.

Perhaps I am wrong (tell me if it is the case), but game programmers don’t use it in their games. JNAG won’t support it if nobody need it.

[quote]RMI uses the class name which is natural and simple,
[/quote]
I agree.

[quote]but a gaming network library can’t afford this overhead.
[/quote]
I agree.

[quote]I typically use an integer to identify the type of object from a set of pre-specified types and then a switch statement to read the correct object from the stream. It’s fast and have very little overhead, but it’s not as convenient as RMI.
[/quote]
What if an API make it for you, and send less than an interger on the network, with a HashMap intead of a switch, and with a final call of your remote function instead of you having to add it in the switch ?

It seems that you may be one of the potencial users of JNAG.

Do you use the polymorphism often in the parameters of the remote functions ? Are they in the main loops of intensive network usage like in a game where each client need to know the new position of a player at every frame ? Are you ready to do a (little) extra work to not use it as RMI enable it ?

[quote]I’m not saying the project should not be approved, just that it’s hard to make it as powerful and easy-to-use as RMI.
[/quote]
Hard, but not so hard. I will post my first results on my project’s page within 2 weeks, as a preview.

[karmaGfa]

[quote]Presumably they’ll
[/quote]
I am 1 man.

[quote]add some radical new conformance interface/class that forces you to store the data you want to send across the network in a particular way… which of course make it harder to use than RMI.
[/quote]
Not at all, but I won’t debate about it before you see how the API will behave.

[karmaGfa]

[quote]With smart use of reflection and bytecode generation …
[/quote]
I am generating java source code, like rmic is doing.

[quote] … you can minimize the amount of manual work that has to be done for each serializable class, and at the same time minimize overhead and maximize performance. There is no need for version handling as found in normal Java serialization.
[/quote]
For the performance issue, I ordered the priorities like that :

• First, the size of the data sent over the network.
• The CPU performance.

My main purpose is to have the less data possible on the network, but I won’t sacrifice the need of CPU time of game developpers neither.

[karmaGfa]

[quote]For now, “no”. It doesn’t seem to have a coherent point.

Is it trying to replace RMI? Why? For performance? …then it MUST be a cunning UDP scheme (or else not be for games).
[/quote]
The UDP scheme is a detail for now, I reserve it for later. It is time consuming to implement things based on it, and for now I need to have the framework well-defined.

I would like to point out that not a lot of games are using UDP, and the players and game developers are still happy with the game they obtain.

[quote]RMI is actually “good enough” for a lot of games.
[/quote]
I am surprised ! Perhaps some turn-based games are using it, but I don’t know about the other games … never heard about them using RMI.

[quote]And it’s free, extremely widely documented and tested, and comes with every sun JVM automagically.
[/quote]
JNAG is also free, it will be documented of course, tested also, and it won’t be big at all. Just a few classes.

I would like to point out also that jogl, with is also a game library (binding), is a lot bigger that what jnag will probably be. JNAG is designed to be in pure java, and can run in an applet without any security issues.

[phazer]

What if an API make it for you, and send less than an interger on the network, with a HashMap intead of a switch, and with a final call of your remote function instead of you having to add it in the switch ?

Yes, that will work. With polymorphism I mean the ability to send different types of objects (which are sub classes of the parameter type) as argument to a method call. This is very common, even in games. Each sub class would of course have its own serialization code.

About the mapping of class names -> int IDs, I think it’s nice to have the possibility to explicitly specify the mapping in the code. That will work better with obfuscation for example and the mapping never has to be sent over the network. If an ID is not found for a class a runtime exception would be thrown. Of course, dynamic mapping can also be implemented.

About serialization: it would be nice if serialization code was generated automatically and dynamically for a class. It would of course take into account transient fields. It would also be nice if you can override the write and read methods for a class (similar to Serializable). And also the instance factory.

About method calling: if you can make it similar to RMI that would be great. Otherwise a simple method ID -> Runnable mapping would probably work for me. I think both the client and server should be able to run single threaded, so asynchronous method calls is a must. This requires some smart buffer handling to store the method calls until they can be put on the socket buffer. If you have 1000 clients you can’t allocate a 1MB buffer for each client :). Also, buffering can cause some latency problems, so message priorities would be nice (for example player positions have high priority, world data low). Synchronous method calls are nice, but not necessary for games I believe.

About TCP/UDP: I think TCP is a very good protocol that will work for almost all games. I think many MMORPG use TCP communication.

Both the client and server should be controlled by a single socket select loop to get best performance.

So, it’s quite a complex network library we are talking about. Are we thinking the same way?

[blahblahblahh]

So, if the programmer don’t want to use RMI, for the previously mentionned reasons, he have to make himself his own RMI-like framework with some solutions to those problems, or he have to give-up the ease to use of the RMI-like framework.

Or…do what thousands (tens of thousands?) of people do and use one of the 5-10 massively popular open-standard (usually open source) alternatives to RMI that RMI was attempting to be compatible / competitive with. The obvious starting point is CORBA. Then you go off and look at the masses of improvements that have been made on CORBA over the last decade…

[blahblahblahh]

I would like to point out that not a lot of games are using UDP, and the players and game developers are still happy with the game they obtain.

Shrug. Serious games with competent network programmers typically use UDP-derived protocols. No computer game ever wants to use TCP (bar a tiny number of exceptions, e.g. possibly card games where things happen only every few seconds or every minute, and other board games with no time limits), but TCP is there and what they want doesn’t yet exist in java for free and open source, so…

If you were proposing a UDP derived protocol then that would seem to me to have much bigger value to games devs (there is NO java implementation of the main free UDP-derived protocols that C and C++ games programmers use on almost every project! Where is “java eNet” for instance? Sob).

About TCP/UDP: I think TCP is a very good protocol that will work for almost all games. I think many MMORPG use TCP communication.

No, “most” games want a working alternative of RDP (TCP clone minus IIOD and with optional GD). Many games put up with TCP because it costs less programmer time; most of those that do suffer because of it (unless they are very simplistic games with no real-time gameplay and little decision making and little data to send)

JNAG is also free, it will be documented of course, tested also, and it won’t be big at all. Just a few classes.

Ah. I was thinking you were aiming at something considerably larger than just a few classes.

[princec]

I vote “yes”, as I’m going to be doing a deathmatch game next year and I want to see some code

…but only on one condition:

It exlusively uses Buffers, and exclusively uses UDP, and looks remarkably like Quake3 communications protocol, which is proven to work.

Cas

[karmaGfa]

[quote]I vote “yes”, as I’m going to be doing a deathmatch game next year and I want to see some code

…but only on one condition:

It exlusively uses Buffers, and exclusively uses UDP, and looks remarkably like Quake3 communications protocol, which is proven to work.

Cas
[/quote]
Yes, it will use buffers.
But no, I can’t guarantee that it will use “only” UDP.

does jogl only bind what you need ?

Thank you for your support, you won’t be deceived.

[karmaGfa]

[quote]RMI is actually “good enough” for a lot of games. And it’s free, extremely widely documented and tested, and comes with every sun JVM automagically.
[/quote]

No computer game ever wants to use TCP (bar a tiny number of exceptions, e.g. possibly card games where things happen only every few seconds or every minute, and other board games with no time limits

??? … no comments.

Ah. I was thinking you were aiming at something considerably larger than just a few classes.

Does RMIC contains a lot of classes ? Is it important for the targeted application ?

Anyway, “big” or “small” is subjective, I don’t want to debate about it, but I think that the size won’t be a problem at all.

Ps: I moved my vote to “yes” as a symbol of me disapproving the reasons of the unique “no” 8)