I don’t hang around here very much, but I tend to keep an eye on the jME and Networking sections of the site. I’ve seen a lot of posts from people that have their own networking API. Since I’m the creator and maintainer of JGN this is a question of interest to me.
So, how many other networking APIs are out there besides JGN and Darkstar?
I guess this also raises the question, does there really need to be all these APIs our would it be beneficial to try to collaborate and merge the best features from each into a more universal API? If no one cares that’s cool too.
Both CaptainJester and Riven provide swish looking code/libraries in this thread.
I keep meaning to write something new in networking too, still yet to see anything that implements the existing strategies for game networking.
I’d put DarkStar outside of any of the libraries mentioned here though, it solves a different problem. It doesn’t help much with networking more with hosting IMO.
As to combining efforts to pick the best features from the different libraries, it’s always an honourable goal but who get’s to decide whats a good/bad idea normally falls apart based on different aims and/or experience. I’d love to get involved in a common game networking library for java! but can’t see how it’d be kept on track
I would be happy to host such an endeavor and we could potentially put together a group of us that have a networking API already and/or just a lot of knowledge of networking.
My goal with JGN was to accomplish that and it’s got a modest following, but I would love to the knowledge of such great networking gurus like Kev, Riven, etc. towards a common goal.
If we were to define such a group we could vote on any controversial issue as well as discuss. I know there can be a lot of controversy of topics such as TCP vs UDP and the like, but I think we could hopefully come to some general consensus. I’m game if anyone else is interested. If not, we can just go back to maintaining our own projects and keeping our own duplicated code.
I’m still yet to see anything that defines the existing strategies for game networking.
Now that I have a blog (oh, how wonderful it is to be able to post half-complete ideas :)), I’ll be posting some blog articles about this in the near future, mostly copy/pasting and then improving the research I did for a book a couple of years ago.
Off the top of my head, main categories are:
distributed state management: send-whole-world, send inputs only, send inputs/receive whole world, send inputs/receive inputs/predict future/receive corrections to future, send inputs/receive collapsed inputs (with “impossible” inputs to make it work)
dead-reckoning with direct linear interpolation, exponential interpolation, and spline versions of both
optimizations: send-localized-subset, send-dynamic-QoS-as-separate-pseudo-streams, mark hard-coded QoS onto game-objects at dev time
simulation playback: play fixed-frame-delay, play dynamic frame delay, play exact frame with prediction, play exact frame with prediction but server rewinds time, sequenced time (monotonic increase of packet numbers), distributed time (ping estimations), distributed time (vector clocks)
…but I know there’s one or two missing there, something cunning to do with physics and strange frames of reference, but I’d have to dig out my notes / wake up and have more coffee to remember it.
If anyone’s got any more they’ve heard of but don’t understand, please shout. If I know anything about them, I’ll do my best to explain them :).
Looking at the list from a generic network library point of view only some apply,
Is game specific and down to the data that you mark as synchronized between nodes. I don’t think it’s possible to prempt what a developer might want to use as their inputs/data etc. Essentially all of the above comes down to allowing the developer to define world state.
Totally! The API needs to have a pluggable strategy for describing:
a) How the state received is applied to the local state (for instance, running dual world like X-Wing vs TieFighter)
b) What portion of the data the local node should broadcast (local units only, distance based, team based, type based etc)
This is where I live mostly, seems like the smartest approach to me - lower bandwidth, higher potential for synchronized large unit numbers. I use a variation on lock-step that allows the simulation to proceed by scheduling each steps client contributions at the server. However, I’ve been unable to determine a way of making a library that makes the idea of distributed simulations understandable at the API level. You have to be very aware of the model you’re using all the time. While I can almost see how an API could be defined to support that, I can’t see how it could be made generic for other types of networking.
I think thats really a specialist topology of the above concepts. Who owns what, who validates what.
Got no idea what you’re talking about here, unless you’re just referring to error detection/correction?
I’d also be interested in the library providing protocol level strategies. Take the Quake3 model for instance, the build up of change deltas until an ack. It’s tried and tested and seems to give great performance over the public internet (pretty much the only place I’ve ever seen any sort of problems). It’d be nice to mark different fields of data for different strategies.
I’d love to see it all done with annotations really, but I’m too scared to use 1.5 for games
EDIT: Also quite interesting to see the lack of other language networking APIs in the open source land. RakNet seems to be quite popular according to Google.
Raknets author is good at promotion and the API doesn’t do too much, it’s still simple enough to be useable. There’s also eNet, which is much simpler and easier just to plugin to whatever higher level net strategy you wish to use…
I would like to see a simple network API that could send bytes as fast and efficiently as possible, the trouble with most of the API’s out there (and there are lots done by single people but which aren’t projects) is that
they try to do too much, for example by involving synchronization and/or serialization into the API which you may not need. What’s wrong with just sending byte arrays!
they don’t recieve whole byte arrays, so you have to do your own byte-array accounting to put the chunks together… does anyone want half a byte-array?!
Riven has been graciously ironing out his NIO TCP network API which uses a listener model and a single thread (but the threading model might be flexible). The issues involved in using NIO efficiently seem complex (especially Selectors) and it’s great that Riven and Matt (sunsett) provide these network API’s to hide this complexity from us poor simple game logic devs!
About network API’s in general, it would also be cool to have TCP and UDP – TCP is more common as far as I’ve seen.
Also, having lots of different API’s isn’t so bad since they tend to use threads in different ways which suit different people. In my opinion it would be cool if the API didn’t involve threads at all.
Of the network API’s that I’ve seen, the way they use threads becomes important. For example can you write, read and check for connections at the same time (on different threads) and if you can is it because the API is using synchronized blocks (which affect performance and which you might not need)? Also, can you listen for connections the whole time (in your own thread which blocks on connection.waitForConnection()) or do you have to keep checking for connections?
The way that you’re meant to use the API is also important – do they have a listener model or can you use them like connection.write(byte[]), connection.read(), connection.waitForConnection() – my preference is the latter. This also affects how they report problems, for example by throwing exceptions (I prefer this, and I like it when the API throws the original IOException instead of it’s own thing) or by running a listener-method thereIsAProblem()!
I want to post a response to Adam and Kev’s points but I’m meant to be working so I’ll delay…
This is what I meant above about different aims. To me, just sending byte arrays and making sure they’re complete doesn’t warrant me picking up a depedency on a game. I’m working on a network based game atm the moment and since I can’t find a network API that suits me right now I’m implementing what I need as I go. The ability to host a game, recieve player connections, send and recieve complete byte arrays and provide a message passing framework that the game can hook into was implemented in a hour or so (admitedly maybe because I’ve done it before). Without doing anything more in the API (i.e. maintaining object trees and managing changes - see JGN) there isn’t much worth in the library to me.
The other option I guess would be to allow access at lots of levels - but I think the lack of focus would end up with nothing being really great and tailored to use.
Forgot one of the common network APIs btw, Game Gardens provides the libraries that were use to implement Puzzle Pirates. I found them extremely hard to use but I know there are plenty of people who have had great success with them. It is obviously highly tested through and the API, while I find it hard, is well developed.
Wow, that’s very efficient! But does it use Selectors, because it sounds like they’re pretty important for taking full advantage of the nio API?
About network API’s, and this is related to the techniques that Adam listed, I don’t think that a Network API (by this I mean an API that does more than byte-sending) can do everything since many techniques are actually heavilly entangled in the game logic. An API can’t really deal with this - it’s up to the developer.
I’m working on a network game too and of course I’m using my framework but the really hard thing so far has been trying to get the game-logic code to produce the same results on the clients and server regardless of the latency. I think that if an API tried to do this it would run the danger of being too prescriptive in the way that you code your games.
Sure, it’s NIO with Selectors. There isn’t much point using anything else these days is there?
EDIT: Indicently, that wasn’t written to say “that was so quick”. It was intended as an illustration of where I see the costs of network game development - the bits we should be trying to reduce.
Isn’t that the point, the library provides you with stuff you don’t want to code yourself - i.e. it prescribes the methods it’s going to use and you just use them. Making the different known approaches pluggable means you can try and test them without breaking your game code all the time.
Java prescribes that we use garbage collection not direct memory release. AWT/Swing prescribes how it renders to the screen. A library that doesn’t prescribe anything isn’t providing anything is it?
I’m glad this has inspired some deeper discussion, that’s what I was hoping for.
Let me pose this question: Do you think it’s possible to create an abstract API that can be used for pretty much any game?
I believe it is possible, and though probably a noob in networking compared to many of you guys I believe I’ve accomplished that in JGN. I went for sort of an abstraction of the complexities of networking via a messaging system that allows you to create your own beans that get streamed as bytes very fast and efficiently using my own serialization system and put back together on the other side. This really allows any sort of communication to be done and messages are handled via listeners. That’s the core of JGN and there are several features on top of that to provide simplicity for common networking tasks:
Client / Server convenience classes to encapsulate a UDP server and/or a TCP server and make logical decisions about how to commuincate
RemoteObjects - provides the ability to create an object on one machine and get a proxy implementation on several other connections
Streaming - provides the ability to have multiple InputStreams and OutputStreams between a single connection and stream data. Sort of a convenience for file streaming and other standard IO convenience in the NIO world
Synchronization - provides an abstract mechanism to synchronize “Objects” in a game with only a few lines of code and an implementation designed for that game object type (like jME Spatial synchronization synchronizes position and rotation).
The biggest benefit I see a networking API providing the Java world is simplicity of NIO. I think for most people (including myself) doing NIO correctly and efficiently is VERY difficult and providing a simple abstraction above it that makes life a bit easier for game developers is really the first necessity in making a useful API in my opinion.
I’m not trying to say that JGN is the “right way”, but rather to share what I’ve done and provide that basis as one possible way to accomplish the abstraction necessary.
(NB: this is not intended to replace or compete with the existing full-fat java networking libraries people have written. This is the simplest possible thing I could come up with, and you’ll probably find it more useful for teaching yourself NIO than for anything else. But, that said, I am actually using it in mini games including one I’ve already written (will post that game sometime in the coming weeks).)
