Anyone ever done creating a new binary file format for your own special uses?

I made a wacky file format called the AAA file. The binary file stores 1 65 times.

Literally:

FF FF FF FF FF FF FF FF … FF FF 80.

It was used for storing Boolean values using the size of the ASCII character value.

I do not understand that so I’m not sure if this is a serious thread or not.

If it is: I made a binary format for storing large groups of bitmaps (OpenGL type bitmaps) optionally with integer (generally character) bindings for them. That was back when people still used OpenGL bitmaps. I still do but to my knowledge no one else does.

Not in Java, but I’ve, if not invented, so atleast made use of unconventional file formats for storing application data. In C I wrote a custom *.pak file format using ZLIB and TomCrypt and I also played with RLE encoding of game data, but in the end it was more of a little fun project, didn’t end up using it. I did use RLE on *.obj files in a C project, but that was just to keep file sizes down - this was done when people actually used 3.5" 1.44 Mb discs :slight_smile:

I also coded an IFFL packer/reader in 6502 assembler, but that’s like a long, long time ago :slight_smile:

Guilty. 8)

I created a Binary file format to create a font out of binary for my 4K game. I created a new binary format for storing maps in this game I’m creating. I dunno, I like binary… it is very very good for making things take up a lot less space (and also to make things a lot more confusing and abstract).

It is becoming a lost art in gaming with all these libraries running around though…

I’m planning on doing so shortly for Paint.JAVA…

But other than that, I use text formats for most thigs because then I can easily edit by hand.

A: Quite frequently.

Sure, but then again, all file formats are binary and there are infinite amounts of different ones.

What I don’t get is what you mean by “It was used for storing Boolean values using the size of the ASCII character value”. But hey, if it works, it works.

There is value in what HeroesGraveDev says though, most of the time plain text is just fine and, since it is in plain text, it’s much easier to debug - which is a huge advantage.

[quote=“Rickmeister,post:3,topic:46421”]
Ny first programming “language” was 6502 machine code on an acorn system 1. You had to convert your code into long lists of hex numbers and then enter them all painstakingly by hand, writing even a 1K program was very hard work. Oddly enough I still look back on that computer fondly. My next computer had an assembler built in which made life considerably easier. Would I ever go back to entering long lists of hex numbers to program? you must be crazy!

That’s something completely different. (Been there…done that, then I wrote my own assembler)

Hi

I use my own binary formats with care. I do my best to allow the end users to rebuild a file in my format by using standard file formats in the worst case. I create a new file format for a special use only with a small scope in mind. For example, in JFPSM, the model converter accepts various file formats, it doesn’t force developers to use Ardor3D binary format. You can (and you should) keep your models in Collada, OBJ, MD2 or MD3 before converting them into Ardor3D binary so that if a change of Ardor3D breaks the loading of its binary files, you won’t waste your models. I wouldn’t use a binary format to store something both lasting and subject to changes. Moreover, I plan to store an format version number to ensure backward compatibility which will be easier to manage with XML + XSLT if the data are “complicated”.

I use Kryo to get binary without thinking much about.

Food for thought. All these text based formats are fine IMHO for interchange and/or production time but are kinda sucky for runtime. Let me give an example. My hunk of junk android phone is faster than the Cray-1 (that was a supercomputer for ya youngsters). A Nexus 4 is rated at 2.8 GFLOPS compare that to the Cray-2 @ 1.9 GFLOPS (the top dog in 1998) (SOURCE). Now I have some basic/casual games on my phone that take minutes to load. So I don’t play them that much. Text formats are crazy phat. If you’re going to use one at runtime, then please think about using compression. A few years back I hacked in compression into a Dungeon Master clone and it reduced the size from megabytes to a tens of kilobytes (if memory serves). And parsing text is a slow operation. Don’t misread my comments…use a text format if you want but check the low-end performance and think about a binary format (at least cached) if load times are not reasonably fast.

Hm…

I created some binary formats a long time ago.
One of them (called ‘BDT’) is still being used by myself all over the place.
It’s nicely tested, fast, and allows for arbitrary information in tree-form.

It cost me two weeks to make it work so good. : )

I will soon have to invent some new binary format for a prototype I am working on.
And this time I will actually publish the prototype, instead of just posting images/videos.

TIL:
Posting just images and videos never really gets feedback.
Put up a prototype, its way more effective.

Are you done planning it out?
I probably have some more ideas for it…

Have a nice day!

  • Longor1996

I made an archive format called .IDA for my MMORPG to prevent reversing of its resources, basically what it does is you can build an IDA file but theres no way to extract an IDA file back to its original structure. How it achieves this is by hashing entrynames converting all entrynames to 32 bits therefore you can only locate an entry by its hash, not by name, and encrypts the resources with AES-128 using its hash as an IV.

I made my own format for skeleton animated 3D models and animation sequences, because the ones that exist shouldn’t. Either they don’t support skeleton animation or they support everything. No, I don’t want a 3D format that requires me to implement a parser for a full programming language that is then used to define how the actual model data is stored. No, I don’t want to parse a complete 20MB 50 000 line XML file to extract 1% of it.

you should probably use a text file with readable commands, and when you go into production, write some tool that converts it over to binary, and then write a reader. it helps so much more when you’re trying to add different kinds of commands, parameters etc

IMO: XML != human readable. On the whole even with a readable format, most data set I don’t want to edit a text file. But it’s all a question of scale and what the data is. Animation data? Use a proper tool. Tweaking animation data by hand isn’t going to either very understandable nor time effective…unless we’re talking about uber-basic animations.

I did when I messed around with rolling my own image compression, but it was mostly just a header and then blocks of data from a DeflaterOutputStream.
I was just happy that I could beat PNG without much perceivable quality loss :stuck_out_tongue:

I’ll probably do it again if I make a game where I want to protect assets or config info though.

EDIT: and don’t get me started on XML…

Yeah, a long time ago when I was still messing around in C and assembly. Doing binary files in a custom format is actually not that hard. However, today I just tend to use serialization and other more high-level approaches.

Which are binary formats…just the library/language is doing the basic lifting for you.