high level network apis

Networking & Multiplayer
#1

hi,

Here comes a flood of questions!!!

  1. what do you think about Jini, JavaSpaces, Java Shared Data Toolkit and such higher level networking?
  2. How usefull could they be for example for turn based games?
  3. Are their eating much more speed & bandwidth or isn’t the difference that big?
  4. Does any higher level networking api specialized in games exist?
  5. What would be the best choice between the 3 mentionned above?
  6. Can they interact with Servlets?
  7. Is remote database access also part of them?
    8 ) Are they compatible with RMI?
  8. Could they easely be used for making multiple chat sessions, storing name and passwords and such things…?

…well, just tell me anything you can! :stuck_out_tongue:

despite of the questions looking advanced, i’ve no idea of those 3 apis! And only very very few about databases and servlets.
But i think it is worth to investigate a little about these alternatives… After all, we’re here to learn!

oh, and what’s about JMS - Java Messaging Service?
…damn, there are so many apis! i don’t know where to go! :o

So, there is another important question coming: which one is intuitive and easy to learn?

cheers ::slight_smile:

#2

Having worked extensively with message queues in my life before game programming, I can tell you this: do NOT use a message queue implementation for a game. These things are generally very geared towards redundancy and guaranteed delivery of messages, which can incur lag like you wouldn’t believe. I guarantee you’ll get it there; I just can’t guarantee how long it will take me, says the message queue.

I know there’s ways to configure MQs to do other kinds of delivery, like publish/subscribe, etc… but here’s the real issue: using a message queue for a game is like using a 36" chain saw on a loaf of bread. MQs are meant for industrial-strength business systems and in my opinion are not geared for the kind of fast messaging we game programmers need. Please note that I am NOT knocking the technology; I think it is VERY good for what it does.

You may wish to look at ARMI, written by Bobby Martin. I’m working on refining it a bit (after speaking with him). It is essentially a type of RMI without certain overhead. Thus far it works well. My only question will come in the tens of thousands of clients and how they will be affected (so hopefully I won’t be worrying about this too soon, but soon enough. ;D )

You may also wish to review the whole TCP vs. UDP discussion here, as well as the warnings on why NOT to write your own protocols (also in the same thread, I think).

Jini and JavaSpaces don’t belong in this context, by the way. Though they contain elements of networking I perceive them to be much larger than that (especially in the case of JavaSpaces).

I’m not going to recommend a technology. I’m not writing a turn-based game, so any answer I give you is not suitable. Even if I tell you what I am doing it might not be suitable. I think your best bet is to take a shot at ANY technology you feel comfortable working in, getting it working, and then decide whether or not it should be replaced. It’s what happened with me… I was using pre-NIO sockets, had problems, ran into a post on ARMI (which is essentially asynch RMI) and investigated it. So far, doing the job.

#3

When you know what you’re doing with them, you can churn out new apps/games based on them VERY quickly.
Unfortunately, some of the specs change quite rapidly, and keeping abreast of the latest versions is often tricky for anyone not working full-time with the tech.
Also many of the associated tools and utility apps are relatively immature - they show great potential, but you need to have plenty of time to learn all the undocumented gotchas and bugs.
This is very generic advice, although it held fairly true for the two of those three that I’ve used, but it’s the only short answer I can give you ;).

As Gergis points out, overkill.

Yes, no, and maybe. The UDPvsTCP thread may give you a better feel for how one man’s “TCP is just as efficient as UDP” is another man’s “I lose 28 bytes in the header!!!” (or a third one’s “…but TCP saves me mucho bandwidth when fragmentation occurs”).

In general, though, it’s a bit of a fancy trick to get any of them working for a real-time game. Turn-based games you shouldn’t have any problems at all.

Many. About 50% are open-source, 25% are commercial, and the rest are both. c.f. the UDP/TCP thread for some examples (or try gamasutra, and look for reviews of netwokring libraries).

!!!

Typically this is a non-question: e.g. if you want to acces an RDBMS, you can do so using java.sql.* - there’s no issue of “compatibility” here. Most high-level networking libraries will “interoperate” with things like SQL - but usually NOT in the way that is of any help to you; they typically use these external standards for their own processes. Best just to understand SQL or etc yourself.

So, there is another important question coming: which one is intuitive and easy to learn?
[/quote]
Ignore all of them, find the design patterns for network programming (google is your friend), and that’ll be as high-level as you need to go.

There’s so much you need to learn and understand to do networking properly, and very very very little of it is made any easier by those API’s (until you understand the networking side…or unless you’ve got other people working with you who can configure and design your app). The API’s you’ve mentioned have an awful lot you have to learn in order to use them, and none of it helps you with basic networking, so start at the bottom, and go as high as you find yourself needing to.

#4

thanks to both of you for your replies.
You’re maybe right when saying it’s overkill, but I am still wondering if those apis could help me again like RMI helped me to do easy stuff simplier/faster. I’ll continue to “investigate” a bit…
In my case, i don’t care about 28 bits more or less over the network, i’m more worrying about making a sort of… hmm… well, a server like yahoo games, therefore, i think such technologies could help to make life easier.
I’m maybe wrong, i don’t know…

Already thanks

Any other comment is still very welcome…:wink:

#5

If you’re talking about stuff where performance has no real limit (e.g. yahoo games), RMI would be ideal. I’ve worked with it many times and it actually quite simple to get going. I’m not quite sure how well it works across the internet, I think you’ll have to bootstrap the server in someway…

Kev

#6

[quote]1) what do you think about Jini, JavaSpaces, Java Shared Data Toolkit and such higher level networking?
[/quote]
Jini is a distributed computing platform, designed to support dynamic service discovery, failover, etc, etc so it really doesn’t fit in this category.

JavaSpaces is a pretty simple, elegant API built on Jini which can be used to build a wide variety of applications, including chat apps, but is especially good for things like compute servers, and loosely coupled, distributed applications. Also, if your not familiar with the programming model it can be a bit of a paradigm shift.

Jini, and therefore JavaSpaces as well, are built on RMI, which is the underlying network communications model.

The Java Shared Data Toolkit was actually designed for collaborative desktop applications such as games, but it is essentially a dead product, so it hasn’t been updated to take advantage of things like NIO, etc.

JMS, as has been pointed out, is a high-level messaging API designed for industrial strength communications between mission-critical business apps.Modern JMS servers have gotten pretty fast, but are unlikely to ever match the speed of raw socket communication. That said, there is at least one commercial game server (TerraZona from Zona, inc) that uses JMS as its primary communications channel.

[quote]2) How usefull could they be for example for turn based games?
[/quote]
It really depends on the latency requirements of the game.

I disagree with the assessment that “JMS is too slow, so don’t use it” as this is going to largely depend on the nature of your game. True, if you’re sending data 15 to 20 times a second, JMS probably isn’t for you. If you’re sending data 2 to 3 times a second, or even less frequently, JMS will give you a lot of power for very little effort. It is very easy to learn and use and has a number of nice features for things like secure communications, message filtering, temporary topics for callbacks, etc, and is designed to reliably scale to tens of thousands of simultaneous connections. If it fits with your preformance requirements I say go for it.

JSDT was designed to be much more efficient without a lot of the armor-plating of JMS, but as I pointed out, it’s no longer actively supported, so you have to take that into consideration.

[quote]3) Are their eating much more speed & bandwidth or isn’t the difference that big?
[/quote]
JMS and JSDT both come with a certain amount of overhead, which may be significant, depending on your app. Once again, unless your trying to deliver real-time simulation updates 10+ times a second, this probably isn’t going to be an issue.

[quote]6) Can they interact with Servlets?
7) Is remote database access also part of them?
[/quote]
This is an orthogonal issue. The answer is yes, in the same way any Java API may interact with servlets or jdbc.

[quote]8 ) Are they compatible with RMI?
[/quote]
Jini and Javaspaces are both RMI-based, but may not address your needs. JMS is message based and JSDT is channel-based so neither really maps directly to RMI.

[quote]9) Could they easely be used for making multiple chat sessions, storing name and passwords and such things…?
[/quote]
Multiple chat sessions, sure. storing names and passwords is beyond the domain of the networking APIs, but there’s no reason you can’t communicate this information over any of these networking mechanisms and then store them any way you like.

[quote]So, there is another important question coming: which one is intuitive and easy to learn?
[/quote]
IMO JMS is extremely easy and intuitive to learn, and probably provides a lot more capability than you’re likely to need. There is a cost in terms of latency, but unless extremely low latency is an issue, which it may not be in a turn-based game, JMS may be your best bet for getting off the ground quickly with a minimal learning curve.

#7

[quote]If you’re talking about stuff where performance has no real limit (e.g. yahoo games), RMI would be ideal. I’ve worked with it many times and it actually quite simple to get going. I’m not quite sure how well it works across the internet, I think you’ll have to bootstrap the server in someway…
Kev
[/quote]
RMI is notoriously problematic over the internet, especially for things like registering callbacks. It’s also not firewall friendly, so you’re going to have to do a lot of work building bridges if you’re planning to use raw RMI for an internet enabled game. That said, the folks on the Jini project recognized this issue and developed JERI, an extensible RMI implementation, along with plug-ins for doing RMI over the internet.

#8

very informative, thanks

#9

If you need something that’s brain-dead easy to learn and you aren’t trying to build quake - JMS is not necessarily a bad choice. MQ scenarios are exceedingly useful for large numbers of clients - but much less so for the average 4 - 8 player game. In a turn based game where you want to make sure messages get there untampered - JMS is a good choice. I wouldn’t spring from IBM’s MQ Series or anything crazy like that, but you do want to find a robust implementation. OpenJMS comes to mind. SonicMQ is also good if you’ve got a little money to spend.

I would definitely (and currently do) use it for a variety of gaming scenarios where near-real time is good enough. I wouldn’t use it for an action game however. it is possible to deal with the latency scenario such that the MQ servers don’t cause you any grief - and simply message selectors and remote queues will make like painfully simple. Essentiall you’re slapping a HashMap or similar together and passing it to people - and the MQ servers are very very good at delivering messages (especially if you DO spring for MQ Series).

JMS is also very good for message driven situations. Lets just say that if you want a large internal world simulation - having a bunch of servers speaking to each other over JMS will be your best friend :smiley:

#10

Has anyone considered using JXTA for network games? It’s supposed to solve all sorts of issues about getting through firewalls and NATs. I was wondering if for things like Quake where anyone can be the server, if a P2P solution like JXTA makes sense?

#11

[quote]Has anyone considered using JXTA for network games? It’s supposed to solve all sorts of issues about getting through firewalls and NATs. I was wondering if for things like Quake where anyone can be the server, if a P2P solution like JXTA makes sense?
[/quote]
JXTA does have some nifty features, such as traversing NATs, firewalls, etc but it does so at a cost. JXTA (or more specifically, the default transport layer for JXTA - more on this in a moment) will get your message from Point A to Point B by whatever route it can find, even if it has to make several hops to do it, which could make latency prediction somewhat difficult, and therefore questionable for use in something like Quake.

Now I mentioned this was the behavior of the “default” transport layer, and this is where the real power of JXTA is realized.

More than simply a P2P platform, JXTA is, at it’s core, a framework for “virtualizing” the network. Think of it as a “write once, run anywhere” solution for the network. You write your application independent of the network infrastructure and decide at deployment time where you’re going to run it. If you’re running on a LAN, plug in a UDP or TCP transport for unicast messages and Multicast UDP or LRMP for broadcast. Going over the internet? Switch the Multicast transport to a Relay for broadcast. Gotta get past those corporate firewalls? Switch to an http transport and send those packets over port 80. Want to add a cell phone client to the game? Add an SMS relay. And the best thing about this is that it can all be done without touching the original application code.

The other cool feature about JXTA is that you get to decide which components of your network app (or game) run where at deployment time as well. Take the Quake example. Over the internet you may want to run client-server with game logic handled in a secure centralized location to prevent cheating. In a LAN setting you want maximum performance and you’re willing to trust the clients with the game logic. With JXTA, you’d simply create a GameLogic PeerGroup and a GameClient PeerGroup. In client-server mode your players join the GameClient group and your servers join the GameLogic group. In the P2P version, your players join both groups. Just like the transport layer, PeerGroup membership is configurable at deployment time, giving you maximum flexibility without having to rewrite your code.

Obviously there’s a price for this flexibility, though the Network abstraction in JXTA is extremely thin. At the end of the day, performance will be more heavily dependent on the types of transport(s) you plug in than on the overhead of JXTA itself.

#12

Ehum, why would anyone use the stuff mentioned above at all? I mean, is there more to it than just using java.nio to do your stuff? As far as I know, you cant write RAW packets in java, can you? So how would you be able to manipulate routers etc and affect the propagation time?

#13

[quote]Ehum, why would anyone use the stuff mentioned above at all? I mean, is there more to it than just using java.nio to do your stuff? As far as I know, you cant write RAW packets in java, can you? So how would you be able to manipulate routers etc and affect the propagation time?
[/quote]
And why would anyone use anything but hand-tuned assembler to write their games? ;D

Seriously, there are any number of reasons for using high-level communications APIs including rapid development, ease-of-use, reliability, scalability, flexibility, etc, etc. all the same reasons we use high-level languages like C++ and Java.

Obviously there’s are tradeoffs, but if performance isn’t your biggest issue, or more importantly, if the high-level api you choose satisfies the performance requirements of your game, then why the h*ll not?

#14

I meant more like; What do the high-level apis have to offer in terms of low-level optimisations? Or, what do they have to offer at all?

#15

[quote]I meant more like; What do the high-level apis have to offer in terms of low-level optimisations? Or, what do they have to offer at all?
[/quote]
In answer to your first question, that depends on the specific APIs, each of which is optimized for a different set of requirements. I know that the current release of JXTA uses NIO and I’m sure at least a few modern JMS implementations do as well. There are also likely to be implementation specific optimizations as well.

As for your second question, I think a number of the previous posts outlined the benefits of each of the high-level APIs described.

Once again, if the only way to meet your performance requirements is through a custom networking solution, and you’re confident in your ability to write robust, scalable high-performance networking code (and have the time to do it) then you’re probably better off rolling your own. But there’s something to be said for using an existing solution that you know will scale reliably to 10^5 or more users while exhibiting constant performance characteristics right out of the box, or that can be deployed across a wide variety of network environments with minimal configuration, especially if those solutions already meet the performance requirements of your game.

I don’t know where I read this but I always thought it was brilliant:

“The First Rule of Optimization: Don’t Do It”
“The Second Rule of Optimization: Do It Last”

#16

For the benefit of Americans: 3G is a theoretically rather good packet-based mobile-phone technology that you can use in Europe and Asia at the moment (that’s why UK mobiles can do video-calls, instead of just voice). If you’re really lucky, your government might decide to regulate the phone industry sometime soon, and you too could get the benefits of single standards that aren’t owned by a single money-grabbing corporate.

(Tongue firmly in cheek…and ignoring the rollout problems ! ;))

#17

[quote]Sadly, with 3G ping times only slowly edging their way below the 1-second barrier (I nearly died when I first heard about the multi-second ping times ;)), that kind of abstraction isn’t an option in many cases today.

I’m guesstimating that it’ll be another 5 years before the various networks I’d WANT to virtualize across are similar enough in “minimum typical performance” that I could afford to virtualize.
[/quote]
Yeah. We got to do something about them phones. >:(

In the meantime, there’s still the option of creating multi-platform games (as opposed to cross-platform) where different kinds of activities can be performed on different devices. Maybe I can’t roam through my RPG world from my phone, but I can still outfit my character, and perform other non-real-time activities.

By abstracting the network within a common framework, it becomes a pluggable commodity. If someone writes a highly optimized JXTA transport for games I can plug it in and try it without rewriting my code. The JXTA abstraction layer is thin enough that this is may not be an impossibility.

#18

I really just hated the idea of encode and decoding XML documents over and over again. Seemed to be a complete waste for gaming - especially in the mobile space where transfering data isn’t free, and its mindnumbingly slow.

#19

…this was the other 50% of the point I was alluding to (i.e. not making clear ;)): because of underlying network problems, you cannot fully virtualize the application, because it HAS to change functions depending on knowledge of what transport it’s going to be using. In the example you describe, it may well have wanted to do that anyway (for gameplay reasons), but you cannot just fire-and-forget; you have to make a decision in advance “what method am I going to call” based upon the transport that’s going to be used.

#20

[quote]I really just hated the idea of encode and decoding XML documents over and over again. Seemed to be a complete waste for gaming - especially in the mobile space where transfering data isn’t free, and its mindnumbingly slow.
[/quote]
While JXTA does use XML for a number of things, these are mostly configuration and discovery protocols, which incur a start-up cost, but can be generally be avoided in your inner loop after everyone’s connected.

In the loop, you’re more likely to use JXTA’s BinaryDocument over a transport which uses the Binary Message Format at which point XML never even comes into play.