Hello, back again, I got a question, Me and some friends are creating a Game where there (hopefully) will be 1000 user’s online.
I was thinking that the database would be on heavy load (our game is sort of mmorpg-ish).
and researched a bit, and found no sql.
But is this really worth the effort?
or is mysql(or h2) + hibrenate enough for that amount of people?
thanks!
You really must know why you don’t want SQL, to replace it with NoSQL.
SQL is likely the best to start out for your project - maybe even ditch the database backend for the time being. It will be hard enough to develop the game, let alone handling the persistence.
The number of users doesn’t really say anything about scalability. You can completely trash a server with the load of 10 or 1,000,000 users. As always, it depends on your code / queries / design.
Thanks, will keep that in mind 
FWIW, we use a combination of SQL and NoSQL databases to handle persistence. Most of the game content and player data is stored in relational databases, and we put stuff like event data in NoSQL. It is sort of difficult to make a recommendation without a lot more detail. With either solution, the big challenges mostly lie in caching, and how you handle growing beyond a single database.
I have made a similar design & development journey and from my point of view I’d definitely recommend this:
You’ll want to end up using an SQL-DB. That is superior once you’re in production (and large scale tests), it’s a mature technology and there are so many tools to inspect and work with the database directly.
You might not want to start out with writing the SQL persistency though. This adds a lot of overhead while you’re still engineering the game, changing game entities and redesigning. On the other hand, it may not be wise to postpone the persistency layer entirely unless you’re quite experienced in writing database persistency. You risk having to rewrite parts of the server design once you actually design the persistency layer if you haven’t thought about those issues beforehand.
The above can be an argument for using JPA such as Hibernate or EclipseLink. But I have used both and I really regret it! They can perhaps be good for business applications but they really have the wrong performance profile for game servers (or I dare say any simulation application). They are good for getting ‘some’ DB objects out from the DB, modifying them, and then saving them again in a reliable manner. They are not good for applications constantly manipulating a large set of world data. And while JPA makes the actual translation of individual object field members to DB columns much easier, in spite of JPA’s objectives to insulate developers from DB issues you will end up having to really think about DB modelling while doing your class modelling. In short - for the very early, small prototyping JPA can be neat. But don’t use it for the full game. (And if you must: Use Eclipse-Link rather than Hibernate.)
If I were doing my design journey all over again, I’d start with a clearly defined persistency layer with a small API to load, store etc game data. The first implementation would be to simply write to files as simply as possible, perhaps using Java’s serialization. Then later when the game design doesn’t change that much anymore, I’d replace it with a custom JDBC back-end.
Try myBatis for messing with SQL. If you want NoSQL try Voldemort.
If it’s a hobby project, I’m a fan of Redis because it’s lighter weight and you can do pretty much anything you want with it. It’s basically a giant key-value store. If you write stuff intelligently you can also make your DB fairly object-oriented, like “entity.id#.hp” gives you somebody’s hp, etc. It’s easy to set up, and very easy to debug too.
Redis is in-memory only (though you can snapshot periodically) and doesn’t have clustering, so is hard to scale. I like Kyoto Cabinet, which can be in-memory or to disk, but it also doesn’t have clustering. Couchbase, Cassandra, MonogDB and friends are big monsters, I wouldn’t touch them if you paid me (literally). There are other really bad ones like HyperDEX. Riak might be ok. Voldemort is the most sane of them all though, IMO.
Redis is competition for memcached, so it might be useful for fronting a nosql store, but not replacing one. Mongo has really grown up lately, though its clustering isn’t “elastic” so I didn’t consider it further for my own bigdata projects.
I found Cassandra’s complexity is only there if you choose to use it. You don’t have to predefine column families, or use supercolumns or any of that stuff if you don’t want to. And speed-wise it’s just bananas. I hammered the thing with millions and millions of rows and queries and the garbage collector never even ran. Not exactly real-world conditions, so we’ll see when it actually gets to that.
Last I looked, Voldemort didn’t support configurable quorum policies, which was a huge minus for me. Using vector clocks is a nice touch (something riak also does) but all the data we write is immutable and is associated with its own timestamp, not just the internal one, so a little clock drift doesn’t really hurt anything for me.
Riak is developing something of a MySQL problem, internally fragmenting itself with its choice of storage implementations (bitcask and eleveldb), where some features only work with one or the other. It also lacks range queries and bulk inserts, and considering I needed to insert over 200 million rows to populate the thing, inserting them one at a time was quite literally never going to happen.
I believe I read craigslist and others use Redis like NoSQL, they do their own sharding and have lots of machines with lots of RAM. I imagine it is fast (everything in RAM!), but it is expensive and doing your own clustering is a maintenance nightmare.
Even if you are right that most of the Cassandra nonsense could be ignored, it still scares the hell out of me. It makes both the API and maintenance complicated/difficult to figure out. I want keys and values to be bytes, a simple API for that, and servers that are easy to scale and maintain since I’m doing it all myself. So far Voldemort seems to be working out well for me, though I haven’t released and don’t have any users yet. 
Of course I’ve wildly over engineered, but that is part of the fun I guess. I have a simple interface over the database, so I can swap if needed. Maybe Cassandra is the right tool if you need the column features or a different quorum policy makes a real difference for you, but do you really want to be the DBA for that monster? Plus, Thrift is ewww!
I’ve used Redis on a 300,000 user / day app. MySQL stored most user data, Redis was used for time-related stuff like push notifications, when events would happen, etc. because it was hooked with Resque. That worked totally well. Like I said, Redis is only a viable option as a replacement for MySQL if you’re doing a hobby app and don’t need that many users. And face it, he’s not going to need something that can support more than 100 users / day.
I’ve also used Mongo, which was awesome once we made a bunch of wrappers for it to make it less annoying.
If you’re using a database you can check here:
We are actually giving a talk at http://qconnewyork.com/ QCon tomorrow on java persistence and multiplayer gaming. I’m not yet sure if there will be a live stream of the talk, but I can try to provide the info if anyone is interested.