Custom resource caching?

Newbie & Debugging Questions
#1

Hey JGO, I was wondering if there’s already a useful library out there that can pack my resources into a .dat file or something?

Example project layout:

  • src
  • cache/icons.dat

Than in my project I’d be able to call something like:


static StreamSystem streamSystem = new StreamSystem("cache/");

static void load() {
	Image shopIcon = streamSystem.getImage(
		"icons.dat", // resources packed into .dat
		"ShopIcon.PNG" // our image to load
	);
}

Thanks for the feedback guys :slight_smile:

#2

Yes, .jar files.
What are You trying to do? Image atlas ? Or something else ?

#3

You can use the built-in zip file api in the standard java api and rename the .zip file to .dat :slight_smile:

#4

I mainly don’t want my resources ‘bluntly’ out in the open :smiley:
Now I know I can’t fully stop people from grabbing/editing the resources but I’d like to remove the ‘bluntly’ part and possibly store everything in “Cache files” :slight_smile: (.dat etc)

Never thought of that :stuck_out_tongue:
Wouldn’t they just need to rename the .dat to .zip though? XD

I’ll fiddle around with some ideas tonight in my IDE :slight_smile:

EDIT:
How would I ‘casually’ go about storing a .png in a .dat file (Encoded, see below)
Beautiful charset output: 3cÿrBR·ÒçÓW~ò$ãÔ€ä5Al½ô

Feedback’s still welcome guys =D

#5

I really advice against doing this because it is much more trouble than it is worth, but you can do something really simple, like this:

  1. Compress your resources (loosely) into a ZIP archive
  2. Apply a simple encryption algorithm (really simple, since the purpose here isn’t to ‘keep them out’ so much as it is to making it not so blunt to acquire access to the resources. Also, the encryption algorithm is entirely local so there isn’t much in the way of someone tearing apart your usage of the Java encryption API and grabbing the key if they really wanted it anyway…) So (in this example) XOR encryption.
  3. Decorate a FileInputStream (to the Zip Archive containing your resources) with an XorDecryptionInputStream (basically just applies an XOR to every byte you read from the stream to decrypt it. A simple one byte key is entirely sufficient for these purposes) and pass that to the ZipInputStream
  4. Use the ZipInputStream as you would.

XOR encryption isn’t that great, and there are flaws (you have to avoid NULL bytes etc) but assuming you just want to make the archive unreadable\unrepairable, it should work fine for your purposes.

Also, with XOR Encryption

Original XOR Key = Encrypted

But

Original XOR Encrypted = Key.

So if they know what the data is supposed to be, your key is exposed. And they can figure out some signature in a Zip Archives Filesystem.

But all that is besides the fact, since security isn’t an issue here.

#6

This idea in its rawest form is:
Export a .png to a .txt file.
Encode / encrypt the .txt file.
Change the format to from .txt to .dat.

How would I go about doing that?


// get .png
// pack .png into .zip
// I'm now lost.

Sorry, maybe I should’ve stated my progress / familiarity with encryption / decryption + exporting haha.

The most I’ve done:
DES Cipher encryption / decryption of strings.
Exporting a images pixels to a .txt file with encryption applied.
Importing a image via encrypted pixels.

Thanks for the feedback :slight_smile:

#7

Thanks mate :slight_smile:

I made a program that:
Takes the resource (.png) than imports the raw .png data for encryption prior to exporting it into the cache.dat file :slight_smile:

Next thing I need to think about is how to retrieve a certain object in the .dat cache file, should I use resource headers?
(The headers will be encrypted also ^_^)

Here is the cache.dat file contents (pseudo):

Note:
Since I can’t post the encrypted data on the text area on JGO and successfully click post, here are the links :slight_smile:
I’m happy with the output, there’s no tenfold on the encryption files size.

Old encryption output for 87kb png: 500KB.
New encryption output for 87kb png: same size.

Raw imported PNG data: http://pastebin.com/jyZ8VuuC
Encrypted PNG data: http://pastebin.com/L7fLrSD7
Source code: http://pastebin.java-gaming.org/ab4d1673771

#8

Simply using a non-standard file extension is going to cut out most people. You could always not include the first few header bytes (which makes the format obvious) and that would knock out a bunch more. You’re now down to programmers than can be bothered to decompile your code…which no matter what you do they can walk around.

If you want to kick-it-up to the next level I’d suggest using a 7zip like compression library…at least that way you get something productive (better compression ratios).

Encrypting is simply a waste of your & the CPUs time and doesn’t reduce the number of people that get to your data, so don’t bother.