JNAG(Java Network API for Games) Project Approved!

[karmaGfa]

Or…do what thousands (tens of thousands?) of people do and use one of the 5-10 massively popular open-standard (usually open source) alternatives to RMI that RMI was attempting to be compatible / competitive with.

Can you give some links, please ?
I would like to see them, it can be instructive.

[karmaGfa]

Yes, that will work. With polymorphism I mean the ability to send different types of objects (which are sub classes of the parameter type) as argument to a method call. This is very common, even in games. Each sub class would of course have its own serialization code.

Do you need it because on the server side you use some functions that may be redefined by a subclass of the type of an argument, or is it because you want a customized serialization/deserialization function from the subclass ?

Can you give me an example of where you need it, please ? I would like to visualize concretly in what kind of situation it is may be usefull.

[quote]About method calling: if you can make it similar to RMI that would be great.
[/quote]
thank you :

[quote]About TCP/UDP: I think TCP is a very good protocol that will work for almost all games. I think many MMORPG use TCP communication.
[/quote]
It depends the company. If it is one of the little set of the big and famous development company (for example, Blizzard), they may use UDP because they have about 20 or more programmers just for their network stuff. If it is one of the really larger set of the average development company, they mostly use TCP because they have no time.

Note : We should not say “most of the games” when we speak about the 5% of the games that represents 95% of the sold games.

JNAG will use UDP when I will have time to add it. First, I need to finish the specification of my framework (i.e. how the programmer specify the semantic of the data). After this is finished, everybody can in theory start to use the lib. The UDP stuff is the implementation part - the programmer don’t need to know if the lib is using TCP or UDP, they just need to use it. A reference implementation will be written first (i.e. not specially fast) so that the users can effectively start to use it, then I will start the optimizations of the implementation.

[quote]So, it’s quite a complex network library we are talking about. Are we thinking the same way?
[/quote]
Globally yes.

[phazer]

Can you give me an example of where you need it, please ? I would like to visualize concretly in what kind of situation it is may be usefull.

Ok, let’s say you have a class structure like this:

interface Node {}

class Group implements Node {}

class Leaf implements Node {}

class MyLeaf extends Leaf {}

When a remote method “object.someMethod(Node node)” is called any type of Node object should be allowed to be passed as argument, for example MyLeaf. And at the remote system the correct type will be instantiated (MyLeaf) and deserialized. So, some kind of information about the actual argument type must be included in the method call.

Without this polymorphism no type information would be need to be sent because the receiving part would always know the type of the argument.

Btw, I totally agree about the TCP/UDP issue. Start with TCP and then if needed add UDP support transparently to users of the library. There are disadvantages to UDP, for example being blocked by firewalls.

[princec]

I don’t use JOGL…

What I really need is basically as Blah3 says, RDP implemented on UDP. For RMI-style comms over TCP… guess what! I shall use RMI

Cas

[blahblahblahh]

Howabout doing a java conversion of RakNet, eNet, or similar (all free, all C/C++ only, all very widely used in commercial and hobbyist games)? Or…explaining what you want to do in terms of why its different from doing that?

[karmaGfa]

I don’t know those libraries, I will take a look on them.

Can you provide on this forum the 5~10 other free network solution in Java that you were talking about before, please ? I would also take a look on them.

Probably I won’t port them as you request. I don’t try to do a port but I try to use the capabilities of the language to obtain the best result, and I hope it will cover the needs that one can have when he decides to use the libraries you mentionned.

Regards,
Vincent

[karmaGfa]

Hello,

Good news for the community (I hope :)) :
I have to make a port of a game in Java for my work, and I need to use jnag asap. I cannot spend time for now on the whole set of the functionnalities that I wanted in JNAG, but at least you will have a preview of the lib which will be used in a realistic context.

The lib will continue to involve as I described before, but later.

[phazer]

I’m very interested in using your project in my game engine. Doesn’t look like your project has been approved yet, and maybe it won’t be, so please keep me informed if you release something or put something on the web.

[blahblahblahh]

Can you provide on this forum the 5~10 other free network solution in Java that you were talking about before, please ? I would also take a look on them.

You should be able to find them if you do a little research. I cited enough jargon that you ought to hit most on a single google.

[karmaGfa]

Hello,

I went to see the documentation ofeach of the ones you cited, it was interesting. But none of them was in Java.

[karmaGfa]

btw, from the notice about the “project proposal”, I saw that people have to say why they vote a “no”, so I would like to know the reasons why there is 2 other persons who votes a “no”.

I feel a little depressed … :’(

I would also like to know when the project will be approved or disapproved. Are the administrators waiting for the first sources ? They might come around the beginning of next week since I am working on them at full time now. Should I post them on the CVS repository of the JNAG project ? Can programers access it before the project is approved/disapproved ?

[karmaGfa]

Hello,

I am still fixing some details of the reference implementation, it is not yet ready.

When it will be ready to publish, it will only be an alpha version, so … if you want to use it for a big project, it is better to not use it immediatly.

About the mapping of class names -> int IDs, I think it’s nice to have the possibility to explicitly specify the mapping in the code. That will work better with obfuscation for example and the mapping never has to be sent over the network. If an ID is not found for a class a runtime exception would be thrown. Of course, dynamic mapping can also be implemented.

I found out that such a table don’t need to be specified at all, even when the programmer is using obfuscation.

I am using something dynamic for the mapping, with the possibility to block the call of some specified methods of the exported object on the connection. It can make the receiver of the calls more robust and easier to implement with this feature.

About serialization: it would be nice if serialization code was generated automatically and dynamically for a class. It would of course take into account transient fields. It would also be nice if you can override the write and read methods for a class (similar to Serializable).

For now, the compiler only handle the primitive types of Java + the enumations + some reference handling. It doesn’t support yet the serialization issue, I am waiting to have something working well first, then I will deal with the serialization.

I am actually working on the sent of a reference (local or remote) as an argument of a remote call.

exemple of a Remote interface in JNAG :

public interface ServerGameLogic
    extends Remote
{
  public void login(String login, String password);
  public void insertCoins(int nbCoins);
  public void play(CaracterType caracterType);
  public void logout();

  public void takeMyClientPlayerRef(@CallerSide ClientPlayer clientPlayer);
  public void invitPlayerToJoinParty(@ReceiverSide ServerPlayer ServerPlayer);
}

“@CallerSide” means that the instance that we are giving is a local instance, a stub will be created on the receiver side if it doesn’t exists yet.

“@ReceiverSide” means that the instance that we are giving is a stub representing a remote instance, so on the server side, the method will be called with a local instance in argument.

About method calling: if you can make it similar to RMI that would be great. Otherwise a simple method ID -> Runnable mapping would probably work for me. I think both the client and server should be able to run single threaded, so asynchronous method calls is a must.

For now, the remote calls are made via stubs (I call them “encoders” in my API, since I am using some “decoders” on the receiver side). The calls of the remote instances are made exactly like a local call.

JNAG only (and probably “will only”) support asynchorized method calls.

In the actual Framework of JNAG, the client is using 1 threat with the non-blocking io API. The server is using 2 threats : 1 to accept the connections, and 1 to process the requests and execute the tasks of the server (the actions that the game logic is doing on a time basis rather than on a message-from-client basis, like to update the NPCs, trigger the event of the end of a game, etc …)

This requires some smart buffer handling to store the method calls until they can be put on the socket buffer. If you have 1000 clients you can’t allocate a 1MB buffer for each client :).

work in progress …

Yes, I need to improve this also. For now, I use a basic 10kb output buffer per client, I fix details and I pray to have my tests running like I want

Also, buffering can cause some latency problems, so message priorities would be nice (for example player positions have high priority, world data low).

This is where UDP will join the lib 8)

Synchronous method calls are nice, but not necessary for games I believe.

now that I am thinking … it is really easy to implement synchronous method calls on the top of JNAG. Even if it is inneficient, it may be needed sometime (for easy debug ?). If I see that people need it, I may add it as a auxiliairy util later, but it won’t be a part of the core of JNAG.

[phazer]

[quote]Hello,

I am still fixing some details of the reference implementation, it is not yet ready.

When it will be ready to publish, it will only be an alpha version, so … if you want to use it for a big project, it is better to not use it immediatly.
[/quote]
Ok, cool. I’m willing to do some beta testing.

I found out that such a table don’t need to be specified at all, even when the programmer is using obfuscation.

Ok, it would be interesting to hear how you solved this. Will this work even if the client and server have different names for the same class (which could happen with obfuscation)?

For now, the compiler only handle the primitive types of Java + the enumations + some reference handling. It doesn’t support yet the serialization issue, I am waiting to have something working well first, then I will deal with the serialization.

I hope you plan to implement something that is a lot faster than normal serialization.

I am actually working on the sent of a reference (local or remote) as an argument of a remote call.

exemple of a Remote interface in JNAG :

public interface ServerGameLogic
    extends Remote
{
  public void login(String login, String password);
  public void insertCoins(int nbCoins);
  public void play(CaracterType caracterType);
  public void logout();

  public void takeMyClientPlayerRef(@CallerSide ClientPlayer clientPlayer);
  public void invitPlayerToJoinParty(@ReceiverSide ServerPlayer ServerPlayer);
}

“@CallerSide” means that the instance that we are giving is a local instance, a stub will be created on the receiver side if it doesn’t exists yet.

“@ReceiverSide” means that the instance that we are giving is a stub representing a remote instance, so on the server side, the method will be called with a local instance in argument.

Hmmm, is this really necessary? Isn’t it better to do something similar to RMI, that is always to reference objects which implements Remote and serialize all other objects?

For now, the remote calls are made via stubs (I call them “encoders” in my API, since I am using some “decoders” on the receiver side). The calls of the remote instances are made exactly like a local call.

Ok, similar to RMI I think it’s a good idea that remote methods must be declared to throw some checked exception like IOException or similar. Otherwise it’s easy to miss disconnect errors etc.

In the actual Framework of JNAG, the client is using 1 threat with the non-blocking io API. The server is using 2 threats : 1 to accept the connections, and 1 to process the requests and execute the tasks of the server (the actions that the game logic is doing on a time basis rather than on a message-from-client basis, like to update the NPCs, trigger the event of the end of a game, etc …)

Great.

Yes, I need to improve this also. For now, I use a basic 10kb output buffer per client, I fix details and I pray to have my tests running like I want

Buffer handling is big problem, trust me Some kind of dynamic buffer handling is probably needed. Typically you want a small socket send buffer to minimize latency. With small socket buffers you can implement message priorities where a message with higher priority is sent before a buffered message with lower priority.

now that I am thinking … it is really easy to implement synchronous method calls on the top of JNAG. Even if it is inneficient, it may be needed sometime (for easy debug ?). If I see that people need it, I may add it as a auxiliairy util later, but it won’t be a part of the core of JNAG.

Sure, synchronous messaging is easy to implement over asynchronous messaging.

Looking forward to testing your code.

[karmaGfa]

Ok, it would be interesting to hear how you solved this. Will this work even if the client and server have different names for the same class (which could happen with obfuscation)?

Gosh … this could happen ? :o I didn’t know.
I assumed that the same class are obfusced to the same name. Usually, the programmers are able to obfuscate all at the same time, aren’t they ?

I hope you plan to implement something that is a lot faster than normal serialization.

I will do the best that I can, or find the better compromise, or leave the choice to the user with some parameters to specify … I will see this later.

Hmmm, is this really necessary? Isn’t it better to do something similar to RMI, that is always to reference objects which implements Remote and serialize all other objects?

In my example, the annotations are annotating some Remote interfaces only. “CaracterType” is an enumeration type.

Ok, similar to RMI I think it’s a good idea that remote methods must be declared to throw some checked exception like IOException or similar. Otherwise it’s easy to miss disconnect errors etc.

For now I am not sure where JNAG should notify the user. Since the calls are asynchronous, it makes less sense to throw an IOException right after the call (since it doesn’t reflect the precise position of the problem in the stream of the messages : nothing can certify the programmer that the remote host received the previous calls). I am thinking to let the user know the status of the connection explicitly by calling a function. Usually, the programmers don’t need to check it after each call. Instead, they check once a frame or once a main loop.

Buffer handling is big problem, trust me Some kind of dynamic buffer handling is probably needed. Typically you want a small socket send buffer to minimize latency. With small socket buffers you can implement message priorities where a message with higher priority is sent before a buffered message with lower priority.

oky doky

[phazer]

Gosh … this could happen ? :o I didn’t know.
I assumed that the same class are obfusced to the same name. Usually, the programmers are able to obfuscate all at the same time, aren’t they ?

You might obfuscate just the client and not the server, or have different versions of the clients running against the same server. Then you need to supply a class file mapping to the obfuscator to get the same class names.

In my example, the annotations are annotating some Remote interfaces only. “CaracterType” is an enumeration type.

I don’t understand why you need @CallerSide and @ReceiverSide. Isn’t ClientPlayer and ServerPlayer implementations of Remote? The only way a peer can call methods on a remote object is if he got a reference to the object through a method call from the other peer. Then you would already know in which peer the object is stored. Or am I missing something?

Btw, do you have any plans to implement an automatic distributed GC (could be done using weak references) or will you leave that to the user?

For now I am not sure where JNAG should notify the user. Since the calls are asynchronous, it makes less sense to throw an IOException right after the call (since it doesn’t reflect the precise position of the problem in the stream of the messages : nothing can certify the programmer that the remote host received the previous calls). I am thinking to let the user know the status of the connection explicitly by calling a function. Usually, the programmers don’t need to check it after each call. Instead, they check once a frame or once a main loop.

Yes, that’s an alternative, but I don’t think it’s better. Isn’t it natural to throw an exception when you know that the message can’t be delivered (for example when disconnected)? It forces the programmer to take some action.

[karmaGfa]

I don’t understand why you need @CallerSide and @ReceiverSide. Isn’t ClientPlayer and ServerPlayer implementations of Remote? The only way a peer can call methods on a remote object is if he got a reference to the object through a method call from the other peer. Then you would already know in which peer the object is stored. Or am I missing something?

ClientPlayer and ServerPlayer are implementing Remote. JNAG doesn’t cover the general case like RMI does. In the kind of program targeted by JNAG, the method have to be used only with a kind of instance extending Remote in parameter : local (caller side) or remote (receiver side). Now of course we can check at the runtime if the parameter is an encoder or not, but there is 2 disadvantages :

• the encoder use more CPU to do a check that is supposed to give the same result all the time for a bugless program.
• the programmer don’t explicitly see on what side the remote object in parameter should be when he is implementing since it is not specified in the remote interface.

… it is a little like using generics or not …
Vector or Vector ?

We can use Vector with MyType objects as elements, but it is more convenient to use Vector … it is more clear, and it allows some optimisations on the type check casts in future JVMs.

Btw, do you have any plans to implement an automatic distributed GC (could be done using weak references) or will you leave that to the user?

I will let this to the user until I am conviced that he really need it. JNAG is for a well specified kind of program … the client and the server needto think the same so I assumed that they both know exactly when to let the GC recycle a instance and an its stub on the remote host.

Yes, that’s an alternative, but I don’t think it’s better. Isn’t it natural to throw an exception when you know that the message can’t be delivered (for example when disconnected)? It forces the programmer to take some action.

hum … I guess you don’t see the subtile difference between synchronized and asynchronized for this precise case :
If the implementation decides to send the message later (in order to group messages) and can only detect the problem later ? for example after a kind of flush() called sometime ? The programmer that is using JNAG should not rely on a “fail fast” behavior.

[phazer]

ClientPlayer and ServerPlayer are implementing Remote. JNAG doesn’t cover the general case like RMI does. In the kind of program targeted by JNAG, the method have to be used only with a kind of instance extending Remote in parameter : local (caller side) or remote (receiver side). Now of course we can check at the runtime if the parameter is an encoder or not, but there is 2 disadvantages :

• the encoder use more CPU to do a check that is supposed to give the same result all the time for a bugless program.

The CPU time used to check this is negligible. Your solution restricts the programmer to use only remote or local objects as argument to a method. In some cases you want to pass both types to the same parameter of a method.

I will let this to the user until I am conviced that he really need it. JNAG is for a well specified kind of program … the client and the server needto think the same so I assumed that they both know exactly when to let the GC recycle a instance and an its stub on the remote host.

That will work for most games I think. A distributed GC can always be added later.

hum … I guess you don’t see the subtile difference between synchronized and asynchronized for this precise case :
If the implementation decides to send the message later (in order to group messages) and can only detect the problem later ? for example after a kind of flush() called sometime ? The programmer that is using JNAG should not rely on a “fail fast” behavior.

I understand the difference. But what will you do if the program tries to send data over a disconnected socket? Throw a runtime exception or just ignore it? For example:

remoteObject.doSomething();
// Socket disconnected
remoteObject.doSomethingElse(); // What will happen here?
remoteObject.doSomethingStupid();

I think a checked exception is the best solution for this situation.

[karmaGfa]

The CPU time used to check this is negligible.

Yes, depending the context, it can be negligible.

Your solution restricts the programmer to use only remote or local objects as argument to a method.
In some cases you want to pass both types to the same parameter of a method.

That’s right. They might be some cases where the user wants to pass both. Rare, but it may happens …

I think I will enable the user to specify nothing and put the tests into the encoder/decoder. But the users will win clarity to use the annotations … and also the compiler of the encoder/decoder might have later a debug option to test if the parameter is really on the caller side or on the receiver side in order to have a fail fast behavior.

I understand the difference. But what will you do if the program tries to send data over a disconnected socket? Throw a runtime exception or just ignore it?

I will just ignore it.

I don’t think the exception will help the programmer more, since he will have to finish what he is doing (probably he doesn’t need to change his execution flow since the other instructions don’t get any return values from the remote calls), and since he will have to remember that there was an exception thrown until the place where he effectively react to this event. That’s why I am think to quietly ignore it and let’s the user call a method on the connection that will tell him if it is still connected and if there is no problems.

For example:

remoteObject.doSomething();
// Socket disconnected
remoteObject.doSomethingElse(); // What will happen here?
remoteObject.doSomethingStupid();

the function remoteObject.doSomethingStupid() doesn’t depend of the fact that the server received or not the call of the function remoteObject.doSomethingElse() or remoteObject.doSomething() .

[phazer]

I think I will enable the user to specify nothing and put the tests into the encoder/decoder. But the users will win clarity to use the annotations … and also the compiler of the encoder/decoder might have later a debug option to test if the parameter is really on the caller side or on the receiver side in order to have a fail fast behavior.

That will work well I think.

I will just ignore it.

I don’t think the exception will help the programmer more, since he will have to finish what he is doing (probably he doesn’t need to change his execution flow since the other instructions don’t get any return values from the remote calls), and since he will have to remember that there was an exception thrown until the place where he effectively react to this event. That’s why I am think to quietly ignore it and let’s the user call a method on the connection that will tell him if it is still connected and if there is no problems.

One potential problem that can occur by just ignoring the connection error is that unnecessary work will be performed, for example calculating stuff to be sent to the disconnected client. However, unless you have very frequent connections/disconnections this might not be a big issue.

[karmaGfa]

This page describes how the enumerations are serialized in Tiger :
http://java.sun.com/j2se/1.5.0/docs/guide/serialization/relnotes15.html

Now it is clear that if RMI covers the general case, almost no game programmer would like to stay in the general case. :-/

JNAG is serializing an enumation by sending the index of the element in the declaration of the enum, and doesn’t need to send the type since it is implicit.

--------- d(0_o)b --------

Some news : I encountered some problems with the reference handling. I found a solution, but it may ask me too much time to do it immediatly (I am in an hurry, I have to implement a game by using the working features of JNAG only … only 2 weeks before the first deadline).

I prefer to wait a little before I release a preview, because for now it is too much experimental … still a lot of details to fix.