Peristence layer

Networking & Multiplayer
#1

I’m still quite new to Darkstar concept, so forgive me if I’m asking some obvious questions.

Is persistence layer pluggable ? Is there any other option than using serialization for long term storage of objects ?

From what I can understand atm, it seems that all objects are just serialized to opaque database table, using default java serialization. This strikes me as very dangerous for long term maintainability of data. Also, ability to query large amounts of objects seems to be very limited (outside of deserializing them one by one and checking their contents by java code).

Is there a way to replace serialization with custom routines marshalling data from/to normal relational tables ?

#2

So yes it is pluggable though not currently at the app confugration level.
(That coudl be changed.)

We purposefully keep the actual data storage format private to the object store because that allows it to implement
however it wants to for maximum efficiency. The nature of the system also requires that arbitrary objects be martialable
with no external processing or apriori mapping knowledge.

We’re looking at externalization tools for doing more standard database operations such as data mining. Im curious to know what
kinds of uses you invision as it will help us with that tool design. My current gut level says id rather do conversion to external relational tables
for such mining uses then force that on the objectstore…

#3

My main fear is maintainability. At some point in future, you may want to do some serious changes in the way data is structured - possibly even changing the relationships/class hierarchy etc. With relational backend, you are already separated a bit (because of explicit mapping to OO structures), but, what is most important, you can as well write some migration procedures for data. With serialization, it can be a major pain.

I have used serialization for long term storage in the commercial project and while it has speed up the development probably twofold (and allowed us to finish the project in required time with given resources) we started to be hit by it in major way already few months after going into production. We ended up with classes which had like 3 alternative slots for the same data, expecting nulls in every possible place even if it was set to reasonable value in every possible constructor, etc, etc. All this within 1 year after going to production, I’m worried what tricks they have to do currently to maintain it… As a side note, we had a major problems with customer data security department - after project was put to production, they have requested they should be able to read every single value in their SQL tools…

For data mining, I’m not really thinking about anything in particular - I have never run MMORPG, but I know that with old time mud, I used to grep through player files for amount of gold/level :slight_smile: I suppose that any reasonable MMORPG has to run a lot of statistics to get the feedback for balancing process - class/race distribution, amount of quests done per day, wealth levels for players, most often equipped items, etc. While some of the events can be probably logged separately to dedicated file/database, persistent data like equipment and players have to be queried from live data.

#4

[/quote
For data mining, I’m not really thinking about anything in particular - I have never run MMORPG, but I know that with old time mud, I used to grep through player files for amount of gold/level :slight_smile: I suppose that any reasonable MMORPG has to run a lot of statistics to get the feedback for balancing process - class/race distribution, amount of quests done per day, wealth levels for players, most often equipped items, etc. While some of the events can be probably logged separately to dedicated file/database, persistent data like equipment and players have to be queried from live data.

Okay, I suspect for most of this then round-trip extenalization is probably adaquate. (Pusha button to dump tables, push another button to suck tables back in.)

#5

It would be great to have it done through some public interface. Something like getting a all-the-objects-in-the-world iterator (per data type probably) and leaving the messy details for the implementor. Other way could be more tricky…
And of course, reference implementation would be handy :wink: