Sim Server Press is great...  but wrong.

General Discussions
#1

Hey Guys,

I wasnt sure where to post this. I don’t want to cross post so I think I’ll post here and then point to it from networking and on-line games.

If some of you are reading the press coming out of GDC there isa lot of press around our new Sim Server Technology. Abotu half of it totally misses the point and the rest of it is woefully incomplete. This is my baby, the thing I’ve been saying I couldn’t talk about so I’ll try to clear some of it up now. There will be a whitepaper up here as soon as it passes leagal (grr) but here is the jist:

The Sun Sim Server technology is an execution environment for the server side of online games. It has the following porperties:

(1) Massively horizontally scalable. This means, among other things, the need for regions and shards and such are gone. (More on that further down.)

(2) Fault Tolerant. The same mechanism that makes it scalable means any processor can handle any user at any time. If the user disconnects from their current processor (eg server crash) they just reconnect to another processor and go rigth on playing with little to no noticable lapse in game play. (How little depends mostly onyour games latency handling to cover the reconnect time.)

(3) Orthogonally persistant. It uses what sim programmers call a "real object’ paradigm. An object ocne created esists, and exists in only oen state, until destroyed. Your world is buitl of Sim obecjst thus the entire game-world is persistant and can change, evolve, take damage, etc. Referential integrity is 100% gauranteed (no dupe bugs) and you can cut power on EVERY machine in the back end simultaneously and when you power back up the system coems back online witha state thats within a few moments of when it all went down.

(4) It has a pluggable communciation layer that abstracts the method used to communicate to the user from the rest of the system. This allows PCs, who might be using Gamespy GT2 transport, and MIDP1.0 cell phones for instance (who use HTTP) to connect into the same game system and inter-operate.

(5) The system load balances across all available equipment at all times (no wasted CPU in back-up servers sitting ideal) and in fact allows multiple games to be installed into the same backend and will load balance across ALL of them. This potentially allows low-usage games to say up indefinitely and remain profitable as long as there are enough of them to pay for the total capacity of your single back end.

Okay in a nutshell thats what it is and what it does. For the “hows”, the whitepaper is coming soon…

#2

Awesome! Can I download an eval version? :wink:

#3

he he he

You aren’t the first to ask that.

So let me talk a bit about its state.

The Sim Server exists today as a Sun Technology. A technology is something we are actively developing and talking about but for which there is not a product plan in place yet.
On the engineering side we have what I would consider an “alpha” or engineering prototype. It works, it performs okay, but there are many fatures I wish to add and it hasn’t had even a first performance tuning pass yet.

A few things have to happen to move it forward. The first is that Sun management wants feedback from the industry that this is a wanted piece of technology. This is because, even though I came from the online game development world, Sun never trusts its own people on this sort of thing (after all we Do have some bias) and wants outside confirmation. We did a lot of generating that feedback at CGDC.

The second is that we are currently lining up a couple of pilot projects. These will be hand in hand development efforts with one or more big publishers on big MMOLG projects. The reason we are going with big guys is, frankly, Sun being a big compnay just knows better how to talk to big companies. While I came from the garage, and love that part of the industry, Sun management really dosn’t understand it very well.

The reason for the pilots is very simple. Having come out of the game developmetn world I would not release a piece of middleware that had not been through the fires of real game development. AT likes to say (and I like to quote) that game development is the Space Program of software engineering. Its needs are intense and not totally predictable, its deadlines critical, and failure is disasterous (it kills companies.)

Once this system has proved itself as the back end for a real game THEN I’d feel comfortable putting it in a box and selling it.

One last thing that will happen at some point, in re evaluation/development. The currest system is designed for a multi-mahcine back-end. It is however highly modular. By repalcing the third tier (the data layer) I can create a stack that may not be fully fault tolerant and is restricted to one server but can be used as a develo and test environment for developers. Its on my list of things to do…

#4

sounds similar to a replicated, clustered, ejb solution using 3 architectural tiers ( tier 1 session facade, tier 2 entity objects, and tier3 database).

it is great to see sun flirting with this problem area.

#5

In theoreticla architecture yes, its is 3 tier system

Communication (“edge” in enterprise terms)
Stateless Logic (“business logic” in enterprise terms)
Objectstore (“data” or “database” in enterprise terms.)

In implementation though its vastly different from something like a J2EE stack having been rethought from the ground up to meet the demands of near real-time simualtions (eg on-line games.)

Im hoping to have that whitepaper available soon…

#6

Isn’t this GrexEngine in another guise? Or rather, doesn’t this promise very similar things?

Kev

#7

looks similar to blah^3 thingy, something like J2EE for games.

I’m not fan of database backend (I prefer to keep 4GB of data in mem) it will be possible ?

#8

yep, it looks similar in function to grex. is sun planning on publishing the interface and system semantics to encourage the development of the server side sim framework marketplace?

sun could effectively leverage its expertise in creating the j2ee marketplace, with the games framework industry. this would be a great way to create a healthy, competitive ecosystem that drives down costs and increases overal productivity of java game developers.

#9

[quote]yep, it looks similar in function to grex. is sun planning on publishing the interface and system semantics to encourage the development of the server side sim framework marketplace?
[/quote]
We first tried to talk to Sun in winter 2002 about just such an effort; we were prepared to throw a lot into the ring. We were repeatedly told we’d have to sign their NDA before they could talk to us. We couldn’t get a copy of that NDA in 12 months of trying. I’ve asked many times about such collaboration, and generally got the cold shoulder. We’ve had no feedback from the GTG on why this has happened other than statements about their need to compete with us.

I don’t know about you, but I don’t see any sign of a lack of competition in this arena. I know that Chris said their solution is unique, but that isn’t true.

In terms of lowering costs, there are 2 major open-source projects, one which has been going for 7 years with over 120 contributors. There are countless others, but mostly those are complete rubbish. If you want free or cheap stuff, it’s there.

The vast majority of games developers have much higher reliability, support, and performance requirements; that stuff is economically impossible to provide cheaply - and, if you speak to the purchasers (e.g. finance directors of games companies) you’ll find they generally prefer to pay more, since it increases the chance that their supplier won’t go out of business, and leave them with an unsupported non-upgradeable system (we’ve spoken to two companies who had this happen to them with MMOG systems! This can mean having to cancel the entire game; the studios generally don’t have the skills to build their own MMOG system even if they had the source to someone else’s).

#10

commoditization of services can only happen when interface and semantics are well defined. all that i am saying is that the developer would be better served (by having to learn less APIs, be able to migrate from free->middle of the road->high end service or framework based products. the market is then able to set the price of a product based upon verifiable differentiators (apples to apples benchmarks).

i’m sorry to hear sun being so picky with their strategy, but i’m sure at this point they are really just sticking their feet in the water before they even think of taking on the very expensive investment involved in a standardization process. but, who knows… i personally like to give sun the benefit of the doubt that they are benevolent corporate organism.

#11

[quote]…sun the benefit of the doubt that they are benevolent corporate organism
[/quote]
Scene: jolly giant whistles tunelessley as he traipses through the forest to show his fantastic new cakes at the pixies’ faire. Underfoot, countless pixies are crushed to death by his clumsy feet as he blithely blunders along.

Cas :slight_smile:

#12

Okay, so let me set a few thinsg celar.

This is a project of mine that I started when I was at Total Entertainment Network 5 years ago and has been in off-tiem development for 5 years straight, full time at sun for about 9 months.

I gave it to Sun in return for which they gave me the resources to complete it.

The concept of gane backend middleware is hardly original to anyone. Anyone with any eye towards what has been happening shoudl have been able to see it coming 5 years ago, as i did. (See ZONA, Butterfly, etc) IMO this system has some unique proeprties and, as my third actual attempt at such a system, is probably as close or closer to ‘right’ then any of the other systems out there.

If you want an analogy this oen is a whole lot closer…

Scene: Mouse coems along and pulls thorn out of giants foot, in return for which gaint protects mouse.

I really can’t be responsible for anyone elses “might have been.” I’ve worked damn hard on mine to make it an “is.” If someone else pushes and pulls and takes 5 years of their lives trying to make their dream real, doign whatever is necssary (including working on only vaugely related thinsg for a big company that can eventually help them reach their dream) I hope they are as successful.

Okay, now second points. There is a reason I said its only like an enterporize system in theory. It has soem huge differences:

(1) The entire vertical stack is integrated in one process for speed.

(2) The database layer is a memory resident database with secondary storage as a second asynchronous teir. This gives me phenomenal access times. (And is why I chose TimesTen as a base technology.) I can do an SQl query on priamry key, get the object back and deserialize it all tenths of ms. (Thats right .2 MS or so.)

DON’T confuse it with J2EE. It is worlds apart. Its just the the fudnemental 3 tier architecture is a good way to structure scalable fault tolerant systems. The logical seperation is valuable and used but the implementation is abotu as far from any J2EE app server as you can get.

#13

cool, thanks for sharing the history of your endevour.

sounds much more like a distributed object cache with guaranteed transactional semantics.

i can see where you could make some optimizations based on the fact that MMOGs have different types of persistent information: character based (temporal), location based (spatial), common meta data (semantic networks).

does your framework actually handle running the server side logic?

if so, does it provide quality of service guarantees over the computational resources consumed by the autonomous entities of the simulation?

(i’m sorry if my question is way off base, i’ve never written a mmog, just simple client-side sims and a few 2d games.)

#14

[quote]cool, thanks for sharing the history of your endevour.

sounds much more like a distributed object cache with guaranteed transactional semantics.
[/quote]
Thats a great one sentance description of what we call the “objectstore”, the data tier :slight_smile:

For now, in an attempt to keep the programming interface as simple as possible and avoid thinsg like the nigthmare XMl file that goes along wtih EJBs, we are only using heuristics that can be devined from run time characteristics. In particular we are currently using an object migration scheme were objects remain close to their last user. This is based o nan observation that the vast majority of time in an online game what you are dealing with is “your stuff” and only ocassionally do you actually have a direct interraction with someone else.

But this is just the first cut. Determining what heuristics we end up with long term is part of what those pilot projects are all about. The design approach I’m taking comes out of my other prrformance work and can be described as “keep the design simple and clean and then tune the heck out of it” as opposed to trying to predict the need for fancy and complex mechanisms.

In theory yes. In practice thats an area that I’m still doing development on. Today each stack has a list of the apps installed on it so not every Sim will consume resources on every stack. Withina stack a given task is time limited on how much time it can consume and all tasks are labled with the simulation that spawned them. At the moment the stack logic kernel has a simple single task queue but we have a number of imrpovements in mind including prioritized queues, seperate queues per simulation and such.

I’m letting the needs of the projects drive which features get implemented first as explained above.

Nope excellent questions. Much better then any I’ve gotten from the press so far :slight_smile:
[/quote]

#15

[quote]At the moment the stack logic kernel has a simple single task queue but we have a number of imrpovements in mind including prioritized queues, seperate queues per simulation and such.
[/quote]
would this be able to correctly handle thousands of discrete automota that have widely varying time requirements to complete processing of their “calculate next action” function, such that in the context of “lack of acting fast is detrimental”, the faster processing “calculate next action” algorithms prevail over the slower ones?

#16
Somehow its works out like other Sun-things in the past. There are some excellent, bigger and smaller efforts out there, driven by some non-Sun-people with a lot of energy. Think of GL4Java, LWJGL, JXInput, grexengine, HeadQuarter, ...

Now it seems that one day there is a (highly talented) guy working for Sun thinking: wouldn’t it be fun to make a OpenGl binding/network infrastructure/joystick API/… and starts of. 1 year later, Sun makes big press releases and conference presentations about that new and cool technology. Of course, announced by Sun, the master of the Java universe, it gets all the attention it deserves.
A standard is born and set and the others may go home now.

BTW, is TimesTen related to TEN?

#17

no disrespect to jeff, but herkules makes a good point. i am very new to java gaming community, but from my perspective there is definately some friction here between the new official sun java apis and products and java community efforts.

i think it is import however to remember that for licensing, platform support, agenda, or maintenance guarentees, sun cannot always start with the best open solution. and forking it would not be an option either - then they would be seen as intellectual robbers even though the license might clearly permit this.

from jeff’s explaination of his project, it clearly sounds like he was developing and researching his technology way before sun acquired his intellectual property. thus, jeff conceivably would have a product anyhow. it is a strength on jeff’s part, not a weakness that he was able to find a corporate partner to help him expedite his vision.

on the other side of the coin, it probably would have been nice to hear that sun made the effort to meet leading open solutions in the middle. frankly, i am not sure how they would do this. hire away lead developers from open projects? respectfully ask for permission from the leading community project to let sun fork it? anyone have ideas here?

the thing that totally blows my mind is that xith3d looks to be what java3d should have been. i also gather that sun let java3d get rusty and is now trying to revive it. that makes no sense to me. that looks like the perfect case for sun to reach out and make some negotiatory effort to escalate xith3d as java3d’s successor.

#18

[quote]the thing that totally blows my mind is that xith3d looks to be what java3d should have been. i also gather that sun let java3d get rusty and is now trying to revive it. that makes no sense to me. that looks like the perfect case for sun to reach out and make some negotiatory effort to escalate xith3d as java3d’s successor.
[/quote]
Right on!
When LWJGL was initially developed there was nothing to “borrow” from or cooperate with (GL4Java was not an option, and JXInput was a bit too flexible for what we wanted) so we had to do it from scratch.
Then comes along JOGL/JOAL/JINPUT. Now JOGL is a bit of an odd fish in that it somehow wants to have an awt/swing dependency so they go about developing it all from scratch with a weird ass backend that about 3 persons on earth actually comprehend. However JOAL could just have used the OpenAL part of LWJGL, but no - they again develop it from scratch (borrowed some snippets from LWJGL though). Next up is JINPUT which basically IS JXInput - and again, rewritten from scratch.
Now the Not Invented Here syndrome is rather characteristic of Sun - but the part that really pisses me off, is that they took NO contact whatsoever to the “community” while doing this duplicate work, and they never even told why! So much for JCP being an open process :frowning:

#19

I reakon you need to be careful here, if I went out tomorrow and wrote another binding to OpenAL, I wouldn’t get slated (well, I might, but thats a different matter).

Would we all be so annoyed if these new APIs had come from less powerful origins. Could it be that the only really annoying part about this is that new developers (especially in Java) first look for a SUN implementation and only then if they can’t find one look for something else.

So, while opening up a new project to do the same thing as an existing one is “bad form” and “impolite” its not actually technically bad in that it does encourage people to rethink ideas they originally had.

However, where Java3D is concerned the statement above does seem to be slightly short sighted. Java3D isn’t and never was intended solely for gaming. On the gaming board we might consider it thus but its simply not the case. Xith3D aims squarely at gaming, and does it very well. Java3D is trying to be more things to more people. They are distinct and it makes sense they should both exist. Infact, coming back to the “what if it wasn’t SUN” point, why is it there are at least 3 other Java3D liked scenegraphs based on an indepedant rendering layer on the go (yet as a community we accept this quite happily)?

Don’t get me wrong, I don’t personally agree with the duplication of effort or the waste of developers time and money. I do however think we need to wary of being critical of SUN just because they are a big company who can command the attention of the community.

Maybe we just need to educate people to make a technical decision instead of a blind decision…

Kev

#20

The big problem IS when Sun blows its own trumpet!

Sun should be working WITH the community - not against it. And before you say they aren’t, may I remind you of how Sun more or less invalidated Log4J with its own logging api? Sun should be forthcomming and sensitive to community work!

Then they nuke their own J3D implementation, and as soon as someone picks up where they left off - they re-enter the field. Though Xith3D may not be oriened towards cad/cam - I am fairly certain that it could easily be modelled into something quite usefull. I couldn’t find any info - but isn’t J3D closed source? - that doesn’t exactly help either.

Join hands and minds, instead of duplicating efforts and getting half finished implementations.

/me slightly annoyed