[quote]systems language
[/quote]
No - not a systems language. No language support for realtime tasks and interrupts and separately managed resource spaces which I think it would need; and very complex and inefficient when working with DirectByteBuffers (vote for Structs!).
Surprising how relevant some of those things are for gaming: separately managed resource spaces - fairly important for auto-managing graphics card resources independently of the Java heap; and of course my old favourite Structs which is so terribly useful for efficient operation with external systems… such as graphics cards once again.
[quote] If you’re getting them for some other reason then they can point to a fundamental mistake in your concurrent threading model
[/quote]
Could just as well be a bug. Assume that you have input threads that add messages to queues for each character in the game world and processing threads that read the messages. Also assume that you optimize away synchronization for the characters by using non-overlapping ranges for the threads and minimize contention between readers and writers using the exchange pattern on the queues. The fundamental principle is solid, but a bug in the range-specification code may result in multiple reader threads concurrently accessing the queue of a character.
This bug may result in really odd consequences if there isn’t a mechanism there that makes it fail immediately. It may also only occur when the server is heavily loaded and have lots of logins and logouts going on (things that screw with the, possibly distributed, range specification), making it tricky to catch in testing.
[quote]Who cares about hacking the servers (high profile, likely to get you a lot of unwelcome attention)
[/quote]
Well, the Shadowbane hack seemed to be about server hacking. The DAoC hack that I witnessed personally, where someone had written stuff on the ground (" rulez") by inserting lots and lots of non-player created loot bags into the world also seemed to be server-side.
[quote] Right, this is the only thing I disagree with. In fact, I’m suspicious that the whole topic is a bit misleading. I can’t actually see any strong argument for using Java for a particular genre AT ALL.
[/quote]
Which part of it do you disagree about, that distributed applications are harder to debug and that bugs have worse consequences for distributed applications than for local applications, that the cost of debugging and consequences of bugs are severe enough to influence the technology choice for MMORPGs, or that programmers tend to write fewer bugs with Java than C++?
Like I’ve written, I don’t think Java is a niche language either, but I evaluate its strengths against what the established competition offer. I think it is dubious to say that Java is more productive than the established players for most other genres than RTS games and MMORPGs due to the ready-made solutions available to the established alternatives in most other genres (solutions that will hopefully be available for Java too, but is currently somewhat of a chicken-and-the-egg problem).
Yes - but all the time-consuming bugs in server development are like that (all the others are so simple to fix, relatively speaking, that they usually don’t impact the development time by a significant percentage).
Sorry, I didn’t mean to say that no MMOG servers had been hacked - but bear in mind that a typical MMOG ought to be receiving approximately 200 hacking attempts EVERY DAY (a relatively pessimistic estimate based on my knowledge of how many attempts are made on typical web servers, and on how many attempts I’ve seen on game-servers I was running).
There have been public statements on something like 100 hacks of MMOG’s (I’m guessing, and aiming at an upper limit - most of the announcements get withdrawn quickly so it can be hard to keep up with them :)). So, where are the other 20k attempts that have been made on EQ, UO, etc? Surely we’re not looking at a one-in-ten-thousand success rate?
OTOH, if you look for evidence of client-hacking, it’s like a plague - it’s everywhere. And, in most cases, it’s successful. I have tons of info on this, but look around and you should find plenty of different tools, cracked clients, MMOG trainers, etc.
Well definitely not the first, since I’ve written a Game Programming Gem on precisely that topic :). Actually, none of the above…
Assuming an expert Java/C++ programmer, and a properly managed MMOG dev team, Java does not reduce the number or severity of bugs, averaged over time.
Unit tests do. Regression test do so even more. Automated idiot tests like the ones Larry and co used in TSO seem to be the best of all (AFAICS this is mainly because it’s too hard for normal devs to write great unit tests for distributed apps - the idiot tests are much easier to write).
Re: other games, and the 3rd party libs…The point about immaturity of 3rd party and lib support is good. But from my POV, so many commercial games studios write all their own tools - including custom Maya + 3DS + etc importers and exporters - that I can’t see any difference in their development process if they switch from C++ to Java. I’m being a bit theoretical here, forgive me :), but it seems to me that in theory there’s no problem now that Java has decent, tested, supported OGL access [even if not perfect] (and commitments for ongoing support, particularly from Sun). Sorry to have excluded individual developers from that POV
Personally, I don’t include RT tasks as a make-or-break for being a systems language :).
I see your point, and I think I have a blind spot here because for most of my working life I’ve never needed RT capability - nor worked with anyone who has (at the time) needed it. I have yearned for more detailed control of resource spaces, and in fact for lots of refinements/specification for what the world of data is like outside the basic/low-level of your current VM (which is where the JLS concentrates: for instance, inter-VM communication via shared memory has never been satisfactorily implemented ).
Personally, I see these as oversights in Java’s current design + libs…things that will be corrected sooner or later (like the eventual appearance of List, then regexps and ByteBuffers…we’re going in the right direction ;)).
I keep reminding myself how long C++ took to dominate C (partly because of the need for C++ to mature).
(PS sorry for the long posts. BT only allows on in every 50 phone calls to connect from this line, so I can’t go online much this week )
[quote] Yes - but all the time-consuming bugs in server development are like that (all the others are so simple to fix, relatively speaking, that they usually don’t impact the development time by a significant percentage)
[/quote]
Exactely, and I think Java’s design does help turn many subtle, time-consuming, bugs into fast-failing ones. Not a magic bullet, but definitely an aid.
[quote] Assuming an expert Java/C++ programmer, and a properly managed MMOG dev team, Java does not reduce the number or severity of bugs, averaged over time.
[/quote]
Is this anecdotal experience or do you have any study to back this up? Even the productivity studies that are critical of Java’s overall productivity do seem to acknowledge that Java programmers tend to write fewer bugs than C++ programmers. It will be very interesting to see what the Magicosm and the Wurm developers will be able to accomplish on their spare time in Java compared to the full-time efforts done in C++ that are on the market today.
[quote]The point about immaturity of 3rd party and lib support is good. But from my POV, so many commercial games studios write all their own tools - including custom Maya + 3DS + etc importers and exporters
[/quote]
Ok, I was looking for someone to comment on how wide-spread the use of 3rd party tools in AAA-level titles really is. I just know that a number of high-profile games do it (recently GTA3, Morrowind, DAoC etc.), and I do know, from talking to producers at publishers and developers over here, that Director dominates the children’s games market.
I’m afraid I just can’t see it! How does java make any difference for the bugs you really care about here? (i.e. the long painful difficult ones).
Basically, they are ones caused by: distributed state, unsynchronized microchips (on different circuit boards) - hence problems of distributed time, chain-reactions (one small error is unnoticeable, but is compounded elsewhere and 6 hours later causes a crash in an apparently unrelated module), inability to use debuggers (they don’t work in a distributed environment), etc…
There are others, too, but those tend to be the most important. e.g. Others include all the problems that are fundamental to protocol implementation (and also to the implementation of any non-trivial “standard” - e.g. even MSIE v6 does NOT implement HTML 4.01, which is rather sad really, and means that certain (theoretical) legal HTTPserver implementations won’t work with MSIE).
(nb that was meant to be applied only to mmog dev projects, I’m not sure I made that clear enough)
The problems that occur in MMOG development seem to be unaffected by any of java’s core differences from C++. I have lots of factual data on the problems, the symptoms, and the causes - but no info on fair experiments of c++ vs Java in a controlled MMOG dev environment. There are lots of small things - like the advantages of a cross-platform asynch IO lib, for instance - that are a clear plus…but they don’t really impact the major problems.
N.B. I spend most of my working life building tools to make MMOG development easier and less inclined to catastrophe…given how much work it takes to alleviate the problems even slightly, I really can’t see java on it’s own making a great difference. Perhaps I’m just not seeing the wood for the trees here?
Although I would recommend it every time as the dev platform for MMOG servers :), as I said above this is because I find it generally superior for a very wide range of development tasks.
Well, the worst bug I’ve ever encountered in a distributed system had to do with a language-independent detail of the transaction system, so you may be right. Still, through the jobs where I have worked on enterprise systems, I have always been happy to not have to deal with pointer problems, multiple deletions, and other problems that don’t cause immediate crashes in C++. The concurrent modification exception has also saved me from ugliness a number of times by failing quickly on concurrency errors (though mostly when I started doing GUI development with Swing).
I do think Java’s safety features are useful for non-MMORPGs too. I’ve also seen some reviewers attribute the failure of Sin, a game that competed with Half-life for the position as the first story-based FPS, to its poor quality (I think many people had to download a patch to even get the game to run). I just figured that the added safety would be much more critical for MMORPGs (to the point where the additional safety outweighs the usefulness of third party content-pipeline tools), but now I’m not sure.
I just can’t believe that the premature releases of MMORPGs like Anarchy Online can be blamed on poor configuration management. Anyone in beta would have seen that the game wasn’t ready (it really had all kinds of bugs), and the game flopped because people didn’t pay beyond the free month (with massive layoffs at Funcom as a result). I suspect that the higher costs of beta tests and the need to squeeze MMORPG releases into the periods between the releases of expansions for other established games give developers less time to do debugging of all kinds of bugs and Java may offer some relief.
As complicated and speculative as this has gotten, it might not provide an overt enough argument for Shawns book though.
I just thank eveyone who posted in responce to my question. There is always much good discussion on this forum. We will definitely use this as input into our explanations of Java gaming and genres
[quote]I just can’t believe that the premature releases of MMORPGs like Anarchy Online can be blamed on poor configuration management. Anyone in beta would have seen that the game wasn’t ready (it really had all kinds of bugs), and the game flopped because people didn’t pay beyond the free month (with massive layoffs at Funcom as a result).
[/quote]
Well… I played (and paid) for longer than the initial free month… Only one extra month though, as it soon became apparent that not only was it buggy, many of the core features listed do not work at all. Even simple things like team quests just plain didn’t work, which destroys most of the incentive to team up and work together unless you’re playing with someone you already know.
This wasn’t at release either, it was about a year later.
I know, I subscribed to it after about a year too (didn’t pay past the free month when I bought it right after the release), but stopped again shortly afterwards. Still, Anarchy Online is just the worst example and the recent releases don’t show improvement.
Star Wars Galaxies has serious problems with something as basic as its chat channels and it wouldn’t surprise me if guilds have broken apart or switched games due to the frequent downtime of the guildchat. Group chat also tended to break frequently, but it has gotten a little better now after two months into the release or so.
I don’t think this is purely a capacity problem either since the guildchat frequently breaks for individual guild members and when the whole chat server seems to go down, all the chat possibilities except the spatial /say are unavailable (including tells, groupchat, guildchat, in-game mail, and the friends lists that lets you know who of your friends are online).
The game also bears the signature mark of memory-leaks, nightly restarts (night in the US, daytime in Europe, always prime-time somewhere). They wouldn’t be so serious if they did not put such heavy load on the login servers when people try to get back on, forcing people to sometimes wait for hours (especially on weekends). They provide a great example of how MMORPGs can turn a normal bug into a much more serious problem than it would have been for a single-player game.
I’m writing a report at the moment on AO (and other MMOG’s) that will be used by developers of new MMOG’s to help them understand what causes the most pain to users.
I’d love to hear from anyone with significant bad experience (e.g. like OT’s above) of an MMOG who doesn’t mind being interviewed (by email) for the report.
To date, there are three sources for studios wanting this kind of info:
[] Gut feeling / instinct / guesswork
[] Asking other developers who have some MMOG experience
[*] Trawling forums, reading every single post
The problems with the first are obvious :). The second fails because there are so few devs with first-hand MMOG experience, and of those few most still don’t understand what they experienced (witness SW:G - it is lead by the most experienced modern MMOG developers, and still makes big mistakes).
The third means reading something of the order of a million posts. And then you have to examine each one and decide to what extent the author had a hidden agenda (a high percentage of complaints on forums are false - the users are hoping to get free stuff in-game: “My character just lost a level!”, or complain so loudly they get bought off with a free account, or just trying to achieve fame and notoriety).
I’m trying to use my experience and understanding to filter these into some useful conclusions, but that involves a lot of analytical work…
[quote]The reason I think MMORPGs might be a good genre for Java is that they seem to be complex enough to really make Java’s safety benefits outweigh the benefits of most productivity solutions.
[/quote]
A recent opinion piece:
…although the focus is “Java’s security model is just better for all apps” rather than “more appropriate for some apps, not others”.
[quote] …although the focus is “Java’s security model is just better for all apps” rather than “more appropriate for some apps, not others”.
[/quote]
Specifically, the article pushes the point that Java reduces the number of bugs/vulnurabilities for all applications, something that also appears to be verified by most productivity studies. That does imply that Java is more appropriate for applications with less tolerance for bugs and security problem than for applications where those do not matter as much.
MMORPGs are a lot more expensive to beta test than normal single-player or multiplayer games, it is also more important to stop people from hacking them than from hacking single-player games, and bugs in MMORPGs generally have worse consequences than in single-player games. I think those properties make Java’s benefits especially useful to the genre. I am sure it is also no accident that the author uses so many connected applications as examples…
Sure. And, if as I do, you believe that means all production-quality applications you write, then it means it’s just a better dev language (like C++ is “generally better” than C, to grossly generalize ;)), without being “particularly suited” to any game genre.
I feel you have a good point here, in that MMOG’s spend more money on beta-testing, so a percentage increase in reliability/eae of development saves them more absolute cash than a typical non-MMOG game.
I only disagree when you try and extrapolate backwards and say that java particularly suits the genre. Please excuse the analogy :), but that seems (to me) like saying “horses are particularly suited to long-distance travel because they are faster than walking”. To me, you’re making a faulty logical step by observing an outcome and inventing a reason…horses are simply a better form of travel than walking if you want to move fast: they aren’t “particularly suited”. OTOH, a plane is “particularly suited” because:
it cannot fly short distances
fuel consumption is extremely expensive
average fuel consumption is inversely proportional to distance (i.e. fuel consumed per mile is LOWER the further you fly, because of the high static cost in takeoff and landing, and also because the higher you can fly the lower your in-flight consumption due to aerodynamics; you can’t fly high on short journeys).
Well, in a short article he had to choose a single main example, and the one he chose was buffer-overruns in client-server apps. So, yes, the apps he cites tend to be client-server :).
[quote]And, if as I do, you believe that means all production-quality applications you write, then it means it’s just a better dev language … without being “particularly suited” to any game genre.
[/quote]
If I had to choose between using Java to prevent a buffer overflow attack and a memory leak that requires a daily restart of the game or to use C++ and be able to port my game to consoles and integrate with ready-made engines, my choice would be heavily influenced by the genre of my game.
Security (as in defense against active attackers, rather than just robustness against accidents) only affects multi-player games, so other demands would have greater influence on the decision for single-player games. Extreme robustness, like being able to run for 24 hours without running out of memory, is also a non-issue for single-player games. One could even argue that the game should put a stop to it if someone plays for 24 hours straight
OTOH, portability to consoles and integration with Renderware would be useless for any multiplayer game server, so I just think Java wins hands down there.
[quote] Well, in a short article he had to choose a single main example, and the one he chose was buffer-overruns in client-server apps.
[/quote]
I am pretty sure that the length of the article had nothing to do with why it doesn’t discuss strictly local applications, such applications just don’t need the security (or at least they shouldn’t need it, as the principle to not trust the client in distributed applications applies to all local applications as well).
I really don’t understand the middle section of your post. You claim that “horses are simply a better form of travel than walking if you want to move fast” and I claim that “Java is simply a better technology than C++ if you want security and robustness”, something that especially applies to multiplayer game servers but quickly drops on the priority list for most other genres (a basic level of robustness is always needed, but far from same as for servers).
Java won’t prevent the second, and my point about the nature of MMOG bugs is that the first really isn’t your big worry.
How many games are not multiplayer these days, in percentage terms? How many games do not, at some point in their lifetime, need to become multiplayer (e.g., now that tombraider is on mobiles, a multiplayer version is suddenly much more important)? How many good games developers would design a complex game and say “Hey, lets hardcode this so it’s difficult to make multiplayer because it makes our jobs a bit easier now”? (hint: that kind of attitude greatly increases the failure rate of small studios - when their publisher demands late changes to the game design, they’re stuck).
I stated outright that IMHO all serious applications should be written to a high-level of quality; I have also (in the past) been VERY badly bitten by the kinds of bugs in single-player games that you say are not an issue. It’s hard to see in each case to what extent they were due to how long they were running for, as opposed to more mundane bugs, but it’s certainly a major suspect.
For instance, my brother completed 85% of FF7, but then a bug in the save/load system for a certain combination of completed quests/equipment/etc meant the game crashed everytime that save was loaded, after the load completed.
For another example, I never completed Ultima7 because the game removed a key from my backpack I needed to complete a quest and wouldn’t allow me another one (although they did fix that bug eventually with a patch)
All those examples are adventure games, where restarting beyond a certain point is enough to make you lose the will to live ;), and they are merely the most extreme example (other genres are less effected in general, but the same problems exist). Experiences like this lead me to be intolerant of slapdash coding on any serious project I’m on.
One reason I’ve not been tempted by friends encouraging me to move to MS is that all evidence points to a major lack of quality engineering at that company; I wouldn’t ask MS to change, but I wouldn’t want to work there.
It seems you haven’t been watching the virus lists for MS windows. Buffer overruns also affect local-only programs, and can have equally devastating effect, although the vectors are usually different (e.g. MSIE rather than IIS).
The point I keep making is that “the aspects of security and robustness that java improves inherently are not the ones that keep you awake at night if you’re writing an MMOG”. Security, for instance, is barely touched upon by Java’s sandbox - as far as MMOG dev is concerned. How does java make your Threat Model easier to write? How does it improve your Security Policy (I mean the real one, not a set of compiled classes)? What does it do to guard against impersonation, DDOS attacks, and accidental data corruption (a method leaves the DB in a non-consistent state, but it’s not detected immediately)? How does it prevent item-duping (one of the biggest recurrent security problems)?
For example, things like TP do inherently improve these problems.
I think at this point we’re not going to reach any agreement :). If you want to continue this discussion, mail me privately. I’ve got nothing more to add on it in this thread.
[quote] Java won’t prevent the second, and my point about the nature of MMOG bugs is that the first really isn’t your big worry.
[/quote]
You don’t know if Java will prevent a serious bug or not. Tricky bugs wouldn’t be tricky if we knew beforehand what combination of problems that cause them.
I’ve already provided a potential scenario for a problem in a test server architecture that I wrote where Java’s underlying security principles could have saved me from a real tricky problem (the scenario with the concurrent modification exception).
[quote]How many games do not, at some point in their lifetime, need to become multiplayer (e.g., now that tombraider is on mobiles, a multiplayer version is suddenly much more important)? … (hint: that kind of attitude greatly increases the failure rate of small studios - when their publisher demands late changes to the game design, they’re stuck).
[/quote]
Another problem for all software development is the time that gets wasted through unnecessary generalization. Methodologies like Extreme Programming, developed to manage projects with dynamic requirements, specificly include practices to avoid it.
[quote] I stated outright that IMHO all serious applications should be written to a high-level of quality;
[/quote]
Who can disagree with a general statement like that?
The problem is when you get specific. Nasa’s quality assurance procedures for software that it sends out into space could probably not be justified for a game (or games would cost $500 rather than $50). The same relation might well apply between MMORPGs and other games.
[quote] For instance, my brother completed 85% of FF7, but then a bug in the save/load system for a certain combination of completed quests/equipment/etc meant the game crashed everytime that save was loaded, after the load completed.
[/quote]
These are the bugs that I would include in the basic quality requirements for all applications as it affects the normal user experience. I wouldn’t put a memory leak that requires a restart every 24 hours too high on the priority list for a single-player game though.
[quote] It seems you haven’t been watching the virus lists for MS windows. Buffer overruns also affect local-only programs,
[/quote]
I’ve already claimed in this thread that MS Word is a connected application (though the connection is indirect, through Outlook). The same applies to apps like the Windows Media Player because it is often used to view downloaded content. How would it affect a single player game though? Will people spread viruses through save games? It’s definitely an issue for games that support player-made content though.
[quote] For example, things like TP do inherently improve these problems
[/quote]
Absolutely, but most of the attacks in the world in general, not necessarily against MMORPGs, aren’t against the theoretical security concepts, but against bugs in their implementations. After all, a buffer overflow problem can allow an attacker to crash the server with a simple DOS attack rather than to force him to hijack a bunch of Windows boxes and arrange a DDOS attack.
Maybe MMORPG developers aren’t losing any sleep over the low-level bugs, maybe they should… (from having observed both games, I think the developers of both Anarchy Online and SWG probably should)
I agree that this discussion has reached full circle though, always interesting with new perspectives.
i cant see where java cant make any game u can come up with. i also cant see any hassle in making something like CounterStrike in Java now we can draw with JOGL and we have the likes of Xith3D- it would just take time to make the models (ok i havent spoted a skelital model loader yet). as for sims well ive made 2 or 3 so it cant be that hard ;). even did a hellcats port for a giggle, added mulitplayer so we could mess around playing PCvMac action… was a snap the networking.
the only down side of java and games is sound. sound so far is a joke- u cant do anything good at pressent, javasound is very poor. joal is ok but openal is not that good anyway. but its not a massive hassle to wrap a jni to fmod so thats cool. i guess getting things to draw first it the top of the list- that fine with me :).
but with java’s networking framework and class loading, the speed of computers and videocards. i cant see the point to codeing C/C++ apart from the wrappers.
comeing from playstation2 development- java is a god send! along with jogl, xith, joal, etc etc… our next big title is java. simple as that.
A possible problem with java for some kinds of multiplayer online gaming is the level of abstraction. People love to cheat and I seem to recall that there were a bunch of methods of doing this that involved creating customised OpenGL drivers that made walls transparent and so on in a way that was very hard to detect from inside the game. It occurs to me that java would potentially be more susceptible to similar attacks on the runtime in addition to the more low-level stuff.
