unsure, i have never used slick… though i would imagine that it would be similar.
The LZMA.jar is part of the LWJGL distribution shipped with Slick, and provides considerably better compression than GZIP. It is rather more memory hungry and (I suspect) somewhat slower though.
Cas
I’ll have to give it a try! I’m curious to see the compression comparisons between it and the other compression methods. Thanks, Cas!
Colton
Try 7-Zip out on some files for a quick comparison. It can do both lzma and gzip.
Cas
Will do, thank you! :]
Colton
You said your levels are procedurally generated and that you are more concerned about loading times than size, so for that reason I am going to ignore your first post and suggest something different. The best way to “compress” procedurally generated content is to only store the seed of your PRNG. Use java.util.Random’s Random(long seed) constructor and save the 64-bit value you pass to it. In terms of storage size that’s as good as you can get, but if it takes a long time to generate your level you may want to plan ahead.
I advise against storing the level as a file during run time. It would be appropriate if you wanted to let players save mid game, but there are too many drawbacks and not enough benefits. It’s slightly inconsiderate, too. Your users will either have a hard drive and get annoyed by the constant whirring and clicking caused by the game, or have a solid state drive that suffers unnecessary wear. Or they might use some form of ram based file system and then you’re just wasting your time. Store your current level and whatever levels that are accessible from that location in RAM. As long as you can preload those levels in the background faster than the player can get to them then the player won’t experience any extra pauses between levels.
Another good way to save space is to use delta encoding. Track whatever changes the player makes to the level and only store the data necessary to recreate that level from a previously known state (such as the original procedurally created level.) For example, in a Legend of Zelda type game that didn’t regenerate monsters and obstacles after you left the area… store only the relevant information, not a copy of the entire level. Store the damage dealt to each monster instead of the entire Monster object. Store boolean values (in bit mask form to save even more space) to record which doors/chests are unlocked, instead of storing Door and Chest complete with constant meta data such as references to Key or Treasure objects. This could be in the form of a list of changes (monster 4 move left, right, up, left, up, down, left) or only the change from start to finish (m4.x+=-1; m4.y+= 1;) For most games the final changes will be easier to calculate and faster to execute.
Finally, if you decide you need to save data in a different format because a feature requires it or because you change your mind and want to incorporate manually designed levels, then look into other simple methods. Some of the best compression methods have time/memory trade offs just like the potential time/memory trade offs between static and procedurally generated content. You might have surprisingly good results using images to store landscapes and meta data as plain text and using existing general purpose compression algorithms. Or you can write binary data using a data stream like Orangy Tang suggested (+1 for DataOutputStream). However combining both methods probably is not worthwhile because your binary data may already be compact. Try putting a .mp3 file in a zip file vs putting an xml file in a zip file. The latter will be much smaller because there is a shorter way to express the same data, but the former may just get you a zip file that’s larger than its original contents because MP3s are already compressed to the point where they sacrifice quality to save storage space.
Wow, what a long post! :]
The thing is, the levels are procedurally generated before the player enters them and then they persist after the player reaches them so that he/she can revisit them, thus making them sort of a “static” procedurally generated content. Thus, I would need them stored on disk inevitably, and they don’t operate on a seed, per se, at least not the same one for any given level because it’s a function of time. I am aware that it’s not the smartest idea to have things stored on disk during the level’s state in RAM; either I would implement an auto-save feature that would save to disk periodically or just save to disk when the player exits the level; either way, the level gets stored in very sparse intervals of time, and this is what I’ve originally had intended for this, but I think altogether that aspect of things is fairly incidental.
File loading itself is very fast in the engine right now, since the levels are relatively small and only really consist of tiles and entities (right now, level loading is instant, even from files on the fly, though with some changes I’m making to my code, I’m going to have all of the zones of a level loaded at once when the player enters rather than have an on-the-fly method like I currently have).
The way you described saving monsters and the like is smart, but it’s unnecessary with the way my game is structured; all of the entities are determined upon entering the level, and this information essentially is lost when the player exits (though if a level is cleared, all of the enemies will take a long time to respawn, which is stored as a small flag value in the level file). Since enemies are dynamic, there are no delta values to be concerned about, and this information isn’t even stored on disk anywhere. The locked and unlocked doors aspect will be saved on disk, but those will be, as you said, stored in simple values like booleans, but I may heed your advice and store them in bit mask form, though this may not be necessary given that they will already be stored as something simple, like one digit or character in a given location, which seems like it could end up being more trouble than the optimization is worth.
You’re right about the compression, and I do intend to explore serialization methods to express all of the information in raw bytes because it sounds very tantalizing and efficient; however, this will take some getting used to and refactoring of my save methods.
Thanks for your advice! :]
Colton
(Glad I could help) I was afraid I would need to be more explicit on the seed thing. A lot of game programmers I’ve spoken to in just the past few weeks misunderstand how procedural content works. I think people automatically associate it with “random” content instead of a deterministic process. Everyone says “I can’t do that, the numbers are different every time,” or “The seed is based on time,” or “No, the seed is a function of a random number and what keys the user has pressed so far.” One of those programmers used an initialize_seed() function and didn’t seem to grasp the idea that they could get the program to produce the same set of numbers without saving an ever expanding table of old numbers. Only one of those people shared their code with me, but it sounds like they are all using magical global/static random number generator sources. By any chance are you doing something along those lines? For example, calling Math.random() instead of using the Random class? (If not the spoiler text is a digression.) (Edit: The spoiler tag doesn’t work how I thought it would.)
Internally, Math.random() uses the Random class. Here's the excerpt from Math.java ``` private static Random randomNumberGenerator; private static synchronized void initRNG() { if (randomNumberGenerator == null) randomNumberGenerator = new Random(); }
public static double random()
{
if (randomNumberGenerator == null) initRNG();
return randomNumberGenerator.nextDouble();
}
The Random class's no argument constructor in turn creates a seed derived from a timer.
private static volatile long seedUniquifier = 8682522807148012L;
public Random()
{
this(++seedUniquifier + System.nanoTime());
}
Conclusion: No matter how you derive your seed or what interface you use to access a [i]pseudo[/i] random number generator, there is always some seed value even if it's hidden. (End digression)
<hr/>
It sounds like in your game the only non-constant persistent data only includes the locked/unlocked state of each door and level clear time. Those things need to be saved on a disk (128 bits should be enough.) But it doesn't sound like you are changing tile data for example. Why does that need to be saved? If you have deformable terrain you would need a modifiable and persistent set of data (that's why it's necessary for Minecraft but not other types of games.) On the other hand, if your terrain data is constant then it is inherently persistent.
Procedural content generation is deterministic because the underlying RNG is also deterministic (see the introduction to the 'Pseudorandomness' Wikipedia article.) The first 8 ints generated by a new Random(123456) object is always [i]1774763047, -506496402, -41169962, 2018695370, 1083428877, 298499967, -1264165101, 1316144193[/i] the same way [i]"String value".hashCode()[/i] is always -1213577022 and the same reason a cryptographic hash of a password is always the same and the same reason 2 + 2 always equals 4. Am I missing any reason why even if you derive your PRNG seed from the time the user first visits an area you cannot save that time or the seed derived from that time? Even procedural level generators can be made to create the same output given the same input just like arithmetic should always produces the same results given the same expression in a given number system. Using the same input and the same recipe means you get the same results. All you need to save is the ingredients list because you already know a recipe.
Best Username Ever,
I do have deformable terrain; should have probably mentioned that
And yes, I do understand that underneath everything there is a seed value no matter what; in Java, the Random class just takes the current time in (I believe) milliseconds since the creation of the UNIX operating system or some event like that, and I use the Random class for all of my randomizations; I kind of forgot this fact in my last post! However, it wouldn’t matter anyway, because not only is the terrain deformable, but the levels themselves will contain other data that may tie into, say, quests given from other locations, all of which is procedurally generated, and I could see it growing unfeasible to regenerate the level every time when there are multiple variables involved which can not only be changed from the game level but from external sources in the overall game world (my game is a fantasy RPG, which is comprised both of an overworld and many levels scattered about it). If I were shooting for a basic platformer, however, then your idea would be excellent and very disk-space effective!
I definitely do see what you mean, though, and I apologize for not fully grasping everything you were saying in your prior post, because it’s absolutely true. Thanks for your time and the information!
Best regards,
Colton