I am at work on an MMO implementation in Java. Whereas other MMO implementations feature multiple “shards” that host a limited number of players due to technical limitations, ours requires scalability of one world across multiple servers.
How do I develop an application that replicates its processes across multiple computers on a network? Is a tutorial available?
There’s no magic, so long as you have something running on the servers you can send new code and tell your server to run it. Just like Applets run in browsers, or Microsoft distributes viruses via outlook.
How? If I write a chat program that supports a dozen clients on a single server, how do I rewrite the program to replicate itself across several servers on a network, in order to support hundreds of clients in a single chat room?
What API would I use? If the program could replicate itself, could I simply hook up more computers to the computer with the program, or must I install the program on each computer and write code so that the computers recognize each other? (I am new to the world of networks.)
[quote]I am at work on an MMO implementation in Java. Whereas other MMO implementations feature multiple “shards” that host a limited number of players due to technical limitations, ours requires scalability of one world across multiple servers.
[/quote]
This is a very, very, very, very hard problem, unless you have a very simple MMOG (e.g. I could write a distributed version of Microsoft Hearts in a few days, or maybe weeks).
(disclaimer: I’ve been working on this problem for almost six years, the first 3 as research at university, the last 3 as head of a company devoted to this technology, and we’re still not done. We do have production code that solves the basic problem you’ve highlighted, available for licensing right now, but we’re still working on productising the advanced stuff for use by other companies. You may wish to distrust me on this because I have a vested interest in making people realise this is hard enough they should instead use specialist expertise / libraries)
The easiest possible way to make this work is to use CORBA, or another DOA (Distributed-Object Architecture); note that RMI is a DOA based upon CORBA, and both are part of the java standard libraries (or, at least, of Sun’s distribution). You have a lot of reading to do if you’ve never used either before, but once you know what’s going on, both are simple to use.
Unfortunately, they’re far too simple - they burn performance and for an MMOG probably won’t give you much scalability (NB: I have no idea how complex your MMOG is; if it’s quite simple, they could be perfect for you).
Do you want to distribute software/functions or just divide your world?
For the first, you have the best opportunities with Java bc. it is somehow made to send code over the net.
For the latter, you’ll need some more problem-oriented code.
E.g. in my system, I normally use a simple ‘distribute everywhere’ piece of code to start with:
public class Broadcast implements BusTicketListener
{
/**
* A ticket has arrived on the given line.
*/
public void arrived( BusLine l, BusTicket t )
{
l.getBusStation().broadcast( t, l );
}
}
This is executed on a server for a certain type of message, e.g. a chat message.
The next step could be to evaluate other information from shared data to decide wether to forward data or not:
public void arrived( BusLine l, BusTicket t )
{
BusLine[] lines = l.getBusStation().getLines();
for ( int i = 0; i < lines.length; i++ )
if ( checkFilter( lines[ i ] ) )
lines[ i ].enter( t );
}
checkFilter() could ask wether the given line (==connection) is assigned to chat partner in a specific group.
To refine that, make distribution dependant on the actual data that has to be transmitted.
In any case, I agree with blahbla, a suitable base library is very helpful!
PS Herk’s lib is pretty cool architecturally (nb: I’ve never actually tried to use it). Although, Herk, in some recent IP work (part of the filing process for one of our patents) the UKPO dug out a patent which might patent what you’ve done (they were asking us to explain how one of our patents differed, which was fine, but I noticed strong similarities to your work).
If you mail me privately (ceo at grexengine.com), I’ll send you the URL to a site where you can view the relevant patent. It may be too dissimilar to count (and you might not care anyway :)), but it might at least be interesting to you…
[quote]PS Herk’s lib is pretty cool architecturally (nb: I’ve never actually tried to use it).
[/quote]
If you did, you’d know that is cool ONLY architecturally :-[
Err… our MMOG is merely a MUD that supports an unlimited number of users. And I want to distribute software/functions, not divide the world. For example, the chat program should allow several servers to share one chat room, instead of one chat room per server. That is my idea of replication (unless my research is bad).
My research into RMI indicates that that API supports only distribution, not replication. However, if a tutorial were available on how to implement replication in RMI, I would love to see it! Or is a better API available?
If a chat program supports replication, I assume that I cannot simply install it on a computer on a network and connect other computers to the network to enable replication. (That would be too easy.) What steps are involved? Do I simply add a few lines of code to a program to enable replication, or must I rewrite it from the ground up?
Why not duplicate application code on each server, and use data caching/replication to keep everything consistent? Tangosol (http://www.tangosol.com/index.jsp) have a clustered caching product that could be used in this sort of scenario. Failing that, there may be open source alternatives out there.
Replication, literally, means the synchronizing of the same piece of data into multiple locations. Usually applied to having two copies of a database on two different machines (often servers), and ensuring that a change to one DB causes the exact same change to be made to the other DB.
The main uses are:
Replication takes considerable time, and is usually not done as part of a live system because it would slow things down far too much. NB it’s usually done as a background process, e.g. triggering every 5 minutes, or every day, etc.
[quote]I want to distribute software/functions, not divide the world. For example, the chat program should allow several servers to share one chat room, instead of one chat room per server. That is my idea of replication (unless my research is bad).
My research into RMI indicates that that API supports only distribution, not replication.
[/quote]
If our previous posts seem to have been off-topic for your problem, you’ll need to describe it better (and perhaps make it clear how / why you think you want to replicate?). Basically, it’s not clear what problem you think you are solving, nor how you think you are doing it.
True replication (as described above) is indeed not supported by RMI, but it’s also not supported by anything except databases. Yes, you could replicate your MUD db to multiple servers, but not in real time. And that wouldn’t help chat rooms, unless you have DB-backed chatrooms. Or you might be wanting to do multi-source multicast (a chat-specific feature), but I thought you wnated something more general than just for your chatroom? I’m not sure…
I suggest you explain your chat room example in more detail, since it’s not clear what you want to happen…?
I want to write a chat program. The program should maintain a persistent account (password, profile, etc) for each client. If the program resides on a single server on a network, then the program supports a hundred clients off the Internet. The program can support over a hundred clients, theoretically, but at the expense of increased lag and decreased performance.
If, however, I add more and more servers to the network, then the program should support more and more clients, without lag. Furthermore, if one server fails, then the program should be unaffected. The purpose of a fully scalable, distributed program is to decrease lag, increase performance, and increase reliability.
So… how do I implement the above? Does it make any sense at all?
The big question is what is the client app? Are you writing it, or is it a standard MUD telnet-style client?
Assuming just a chat system, look at IRC networks (google). Also check out multi-source multicast chat systems (again, use google). Once you have an idea of what’s already out there, you can decide if it suits you. If not, you’ll probably have to go to the difficult stuff.
Assuming anything more than chat, then I was right in the first place. This is, in general, very very very hard. There is no tutorial on it just like there is no tutorial on “Writing a new Operating System from scratch”. CORBA and/or RMI is a low-level system (a DOA, as described previously) that can provide some of the low-level support you need to build such a system. Alternatively, you could learn J2EE and use something like JBoss (although this doesn’t solve most of the problems for games / MUD’s - you’ll probably find it hardly helps you at all).
Fundamentally, it’s only easy if there are few (e.g. < 10k) clients or if the different clients don’t need to share any (much) data - i.e. if each player’s actions don’t affect any of the other players (e.g. not even appearing on each other’s screens). For some MUD’s this is the case, but it generally makes for a dull MUD.
Yes, Google leads me to conclude that a distributed IRC network exemplifies exactly that which I struggle to describe.
http://www.newircusers.com/images/networks/meshed.gif
So, how do I do this? If someone can summarize it in an image, surely someone can summarize it in a tutorial. 
You could do that with HeadQuarter easily if you find a mean to tell your client which server to connect to.
On each server, run a HeadQuarter instance with the simple ‘Broadcast’ server from above. In a setup script, make sure that each server is connected to the others (in a way that there is only ONE route between each 2 servers) and you are done.
Chat messages will automatically be forwarded to each server and from there to each client.
But not only chat messages. Position updates, taxonomie, properties …
Maybe - in your sense - HeadQuarter is a replication system.
To develop a distributed IRC network in Java, for example, which API should I study? What is HeadQuarter?
IRC builds chains of severs. The servers act like routers on an ethernet propegating messages to their peers.
There are many limitations of this model:
(1) Synchronization. Data arrives when it arrives. Some particpants see data befoer others do. This works okay for chat but does not work for game logic wher you are trying to coordinate acting players.
(2) Reliability. The chain is fragile, A server goes down in the middle and it can “split” the converations.
(3) Bandwidth and the N-squared prolbem. If everyoen can hear anythign anyone else sends you have an n-squared bandiwdth problem. Chat text can get away with thsi up to a point becuase its bursty and not rellay very verbose. The IRC tree strtcuture further helps because it splitst he problem into layers, but as mentieodn above this introduces latency issues.
For a small numebr fo nodes on a thick back-plabne you might try JRMS reliable multi-cast. For game logic youu will still need to deal with issues of synchronization and such thata re not trivial.
Fact of the matter is that EQ does not have regions and shards becuase they liked it that way. Its because these problems are VERY hard to solve ina reliable manner.
There are entire textbooks written on destributed simulation techniques, you might want to go to the bookstoreor library and read a few if you really want to try to do this.