Hi all,
I am wondering if any out there is using JDBC or similiar in a game project or design.
In fact, I would love to hear ideas of how you might think you would use JDBC in any kind of game on any scale.
If you have a project using JDBC I would like to reference your project in our game programming book as well.
[quote]In fact, I would love to hear ideas of how you might think you would use JDBC in any kind of game on any scale.
[/quote]
Of a 2-3 year development cycle game we’ve been releasing this spring/summer, we used several thousand of ini files to maintain parameters like game balance, AI control values, animation control, etc.
While this worked, the balance and data manager guys didn’t have an easy job (neither did we have). We learned that next time we would use a database (maybe a file datbase like dBase would do).
However, to use datbases within Win32 is not very nice, neither. Compared to the ease and elegance of JDCB. So on a similar project in Java I would use JDBC.
(The step from painfully E-SQL C on Unix to JDBC Java on a mainframe has been a dream 
Maybe some guys working on a Java (server?) game use JDCB now.
So you are saying the game configuration/customization is in 1000’s of external files, that could be game run-time accessed as JDBC records? Sounds good.
Like for example all RPG character and stats perhaps?
[quote]So you are saying the game configuration/customization is in 1000’s of external files, that could be game run-time accessed as JDBC records?
[/quote]
Yes.
Furthermore: during the development cycle it would be nice if the developers could alter the database cells at game runtime, and write back the changes.
So the designers, artists, etc could do an ideal game balance / animation control at runtime. With external files this isn’t a nice and secure tasks (no table/row locks, etc).
[quote]Like for example all RPG character and stats perhaps?
[/quote]
For example, yes.
With a nice design of 1…n tables a DB approach is handier, more robust and more efficient. We had to use tons of GAWK scripts to maintain these 1000’s of external files. Not again. 
For a stand alone game I think JDBC is overkill ressource and performance wise. You would need to distribute a database, the JDBC driver and the whole set of JDBC classes. Just too big.
For that I would recommend some small OO database with speed in mind and some minor restrictions compared to “real world” databases. Examples are Prevayler (www.prevayler.org) or Smyle (www.drjava.de/smyle) which are a few thousand times faster than JDBC against Oracle or MySQL.
You should never forget that JDBC implies mapping records from database tables to Java objects and vice versa. Those conversions take time, are error prone and always imply code refactoring if you change tables or classes…
Only for huge server applications (MMORPGs for example) I could imagine storing client data (accounts, character information) and world data in some database on the server.
But under all circumstances I would try to avoid distributing JDBC to the client side.
Just the 2 cent of a day by day JDBC and server app coder and wannabe game coder…
Tommy
[quote]For a stand alone game I think JDBC is overkill ressource and performance wise. You would need to distribute a database, the JDBC driver and the whole set of JDBC classes. Just too big.
[/quote]
At game runtime just reading the file database should be sufficient. Maybe using the built-in J2SE JDCB-ODBC bridge driver to read some file database like dBase would do.
[quote]You should never forget that JDBC implies mapping records from database tables to Java objects and vice versa. Those conversions take time, are error prone and always imply code refactoring if you change tables or classes…
[/quote]
Not neccessary Java objects. Simple types like resultset.getInt(…), .getFloat(…), .getBoolean(…) and that like could be OK. Main point: to avoid 1000’s of external files to control your animation, AI etc. However this data cells have to be editiable during development by non-developers, too.
Since these data cells are being read at game startup and then cached time shoudn’t be an issue.
I wouldn’t use JDBC to say MySQL to store Java “objects”, but rather Java data And I do mean client-side only.
It seems to me like games like Morrowind could be managed much easier with a standard DB than some one-off game system.
Speed is another question. But then I would be counting on some caching going on as well.
Do people think that JDBC is this really over built for a huge game like Morrowind?
[EDIT- typo]
Hi
I’m currently planning for larger scale versions of my game, i’m no where near implementing the plans, but it’s in my nogin. Any data that is shiped across to the client over a network, when being persisted (at backup/logout times) will be done with JDBC or the likes, this includes the account information, characters and all the information on stats location, sector, friends lists etc etc, data only of course, i’m thinking a table per class probably, and store just the data. I’ve done smaller tests using xml and it’s ok for levels etc that are stored on the client, but for persisted data nothing beats a good database (they have spent more time on performance than I have on high performance file reading :)).
To start with I’m thinking postgresql/MySQL (as I have them running already). Even for medium sized data storage (in game terms) like score tables in non persistant worlds it makes some sence.
Endolf
P.S. I’m not sure where that post was going, but I thought i’d throw in my 2p worth as it were
I agree with shawnkendall. An ad hoc solution can be tough, introducing bugs, etc. I’d say it’s safer (more cost-effective?) to leverage something like JDBC, which is a proven and mature technology. JDBC has a great track record, running in mission critical enterprise software.
Also, using a database is MUCH FASTER than accessing flat files! And if you need, JDBC can access flat files, too.
My quick thoughts are:
The Brackish Water
For trivial things things using JDBC may be slight overkill with the setup and maintainance of the database and the memory overhead of actually bring up a JDBC driver (from what I remember, the implementations I’ve used MySQL, SQLServer don’t seem to be that light).
Although I suppose IF you’re happy to expose your database to the outside world it does stop you having to write a high-score server for client only games. However, if you’ve only got trivial data to store and you’re going to have a server anyway I’m not its really worthwhile.
Kev
[quote] Although I suppose IF you’re happy to expose your database to the outside world it does stop you having to write a high-score server for client only games.
[/quote]
Yes if you use JDBC, wouldn’t you then be forced to put the DB login information in the game? Feels like from a security perspective it would be easier to make a server. Or maybe a server that sits between the DB and the user. A DB driven highscore list for example would require the user client to be able to both read and write to the DB, which would make it quiet easy to cheat. The highscore is probably a bad example though, since it pretty easy to cheat anyway ;D. Lets take an example in an persistent rpg, what would stop the user from adding skills to the DB if the user client talks directly to the DB?
In a persistant RPG (I assume you mean like MMORPG) you’d have the game server element validating the updates… that’d be fine.
Kev
[quote]For a stand alone game I think JDBC is overkill ressource and performance wise. You would need to distribute a database, the JDBC driver and the whole set of JDBC classes.[…]
[/quote]
You don’t need to distribute it. Most companys use databases for developing/designing only… and at the end the stuff wich is really needed gets “extracted” and repacked.
I think it’s the best way to manage tousands of files and parameters. However, it would be kinda overkill-ish for most projects here
But think about it… it’s for sure a nice way. You can add comments to your textures… attributes… and you can categorize 'em. Autochecking of depedencies… no way to forget a file
Still talking client-side here.
In near future, we are planning on using JDBC or something functional similiar Here is our reasoning.
During content production the content production tools use the database for asset management. It makes it easier to back-up and restore if need be, and we can extend any existing file formats with meta data that is game important, as oNyx suggests.
For large 3D worlds that would otherwise need a custom paging database system away (continuous worlds, no level loading) so why not use JDBC instead.
Massive character and item attribute data is simply a stupid pain in the *** to make 100s (1000s?) files and directory management for and even more cumbersome to prototype with. Of course we can make it One Big File ™ but then we have to be ready to make an access system that can handle efficient data inserts, which means the One Big File would have to have a page system just like disks anyway.
Using a database allows a bit more developmental and design freedom.
At GDC last year one of the lectures I went really struct a cord. The speaker said that when creting your content storage design (file format) make sure you can import, export, and edit portions of the final content and not have to re-export the entire asset set. Using a db really solves this problem as well as handling the whole multiple people in one file/area issues and if the run-time engine can access the exact same db without conversion then we really have a nice system
Does it sound like JDBC would still be to “heavy-weight” in this context? I definitely want to identify weaknesses and counter reasons.
I think the caveat on the negative posts above is simply scale. You’ve outlined points that refer to fairly huge scale projects/development. If we’re talking 1000’s of elements of data then the logical choice is always going to be a database (even if its file based like hsql). However, most of the projects round these parts arn’t actually on that scale…
I know we’d all like to build truely scalable systems but there is an overhead involved in using JDBC (runtime and dev-time). However, the trade off between using and not using is going to be based on the personal experience of the developer. If I’d used JDBC everyday of my life for 10 years (impossible?) then even for the most trivial game its going to be worth thinking about.
Kev
JDBC is a damn good way of storing data reliably with no fuss - the techniques are well known and you can pick up any number of books on the subject. Also, most Java developers in the world today are J2EE developers, who already know JDBC.
The biggest problem with regards games is that J2EE API implementations aren’t exactly optimised for either speed or memory consumption! Or at least not for reducing memory consumption…
But as a quick and easy way of persisting state, reference-counting, handling crashes, sorting objects, querying object trees etc, it’s hard to beat a DB. JDBC is simple and really easy to integrate with an application. You can write your ultra-fast low-memory DB implementation and swap it in at a later date if required.
(Shawn, if you’re after a simple example DB I recommend HSQLDB. It’s free, 100% Java, small, fast, and supports in-memory tables.)
[quote]Also, using a database is MUCH FASTER than accessing flat files! And if you need, JDBC can access flat files, too.
[/quote]
That’s not true for many moderate sized data sets (and for small datasets DB’s are usually particularly slow). You can arrange your flat files to be extremely fast, and feature-rich DB’s can never do more than be “no worse than flat files”. The major reason for using DB’s is that you get many advanced features WITHOUT losing speed - you don’t get an increase in speed with a DB if your original data architecture was good.
Really big DB’s are a separate issue - e.g. when they are large enough that a non-DB implementation has to work around the hardware architecture and such like.
[quote]Still talking client-side here.
In near future, we are planning on using JDBC or something functional similiar Here is our reasoning.
[/quote]
On the client-side, your proposal is a good one. We do something similar (in a parallel situation; our clients are servers) but we have a complex software layer on top of the JDBC stuff - I suppose similar to a complex 3D engine on top of OpenGL.
In our situation, the database stuff is used for extremely rapid development of game-logic (and we’ve got a patent pending on some of the layer above which saves devs a lot of time), and then we either snapshot that for distribution or allow clients direct read-only access - but we work on MMOG’s, so distribution is to servers in a cluster rather than home PC clients.
Frankly, anyone who isn’t doing this already can’t claim to be a serious, professional game development studio. For anyone to whom this is not obvious, there are actually 3rd party tools that have been around for a long time that are specifically marketd to games devs for this kind of stuff.
…but you still occasionally see post-mortems where it wasn’t done! 
This is a situation we do have, and have found that (unsurprisingly) JDBC is fine but needs a heck of a lot of work on top of it (c.f. analogy to 3D engine and OpenGL above; it’s pretty obvious, but I’m also trying to give an idea of how much code and dev needs to go into making that layer work…it’s not a trivial affair).
Yup, as I said this is an area where we’ve done a lot of R&D. We were only going to license that part of our tech, but it reduces the dev time and cost by so much we’re now investigating developing and publishing our own games! (greatly reduced risk…)
As an extension to all the server-focussed distribution of “development code as DB” above, we’ve done some very vague investigation of distributing single-player games with a DB - very much as you describe, except that we would be interested in re-leveraging our DB layer for single-player game dev, so in our case it’s less of a case “should we do this?” than of “we’ve already got this; what problems will crop up to make it hard to use in clientside code?”.
The only major problems we saw were:
fast DB’s cost serious mahola to distribute to your end-users. Unfortunately the likes of Oracle have still not experienced a shakeup of their industry, don’t work too hard as a company, and haven’t really cottoned on to allowing cheap (preferably free) distribution of a read-only runtime or similar for this kind of thing.
MySQL is outpaced by a blind snail tied to a ten-tonne weight. Even “OR” was not allowed to use indexes up to at least 4.x (I cannot IIRC whether 4 fixed this yet). There are no stored procs (!) etc. Give MySQL another 3 years and it should be quite fast.
However, that’s from our perspective; if I understand correctly, you’re only really looking at using it for simple data, and could use the snapshot approach like us (so that clients don’t need a copy of the dB software to read the data), in which case I see almost no problem at all…
PS JDBC is IME not at all heavyweight. Poor DB performance and/or people who don’t know how to write SQL queries properly completely disguises the overhead of JDBC. If you’ve got people with decent DBA skill you should be fine.
All IMHO, of course ;).
I’m not sure how you see scalablility of games having much to do with JDBC usage/avoidance?
JDBC (or, more importantly, SQL) is very easy to use, and it takes only a few months max for even a novice developer to be writing “good” SQL (manual optimization is still a VERY big issue in DB work…).
I would suggest that it’s worth learning basic SQL just because it’s so incredibly simple (note: IIRC it was one of those designed to be readable by illiterate managers back in the 70’s…?). Just look at how many incompetent programmers easily manage to write big PHP websites with tonnes of SQL!
I'm not sure how you see scalablility of games having much to do with JDBC usage/avoidance?
I guess I was coming at it from a hobbiest point of view, if you don’t already know SQL (which as it happens, I do) and you’re about to store a trivial amount of data then its probably not worth you going through the (possibly minor) hassle of learning it and setting up the DB.
However, if you expect to scale your game later on to need a hell of a lot more data then its worth the up front work. Thats all.
Kev