I have a thread in General Discussion where I post general updates to the project. I wanted to start a thread here containing project-specific things that don’t involve updates. I.E. discussion, but I am sharing the code of course. I want to break the old thread.
As most of you know, I have been making file formats. X-Media is what XM stands for. I have an image file, mipmap file, and a packed file which contains multiple XM files. (well any file that is smaller than Integer.MAX_VALUE) That is a general run down of what I have and it is pretty much complete!
I would really like if someone were to test out my library and benchmark the loading times against your current file loading method. This would be an amazing help to the development. I am also open to suggestions on the project. I am talking practical use. It’s not hard to use this library at all.
The library is self explanitory, but I have dedicated a few hours to writing wiki pages… which turned out great! The code is fully commented. Check it out
The consecutive post limit was designed to stop people WIP-spamming when nobody else was interested in discussing the project. If you find yourself in this situation then you need a blog rather than a forum thread.
It was not designed so people would instead spam a bunch of new threads once they hit the limit.
There are plenty of free blogging options that will be better suited to your project.
So, the big question is, why would anyone want to use a new file format ?
From looking at your Wiki, I don’t get it. Are you saying its better to download uncompressed files to save decompression time ? Then, you do seem to have compressed file formats as well ?
Adding your own benchmark results would help as starting point.
I will try and reword the wiki more direct. I just spewed information.
- The goal of the file formats are to be decompressed at loading time.
+ The goal of the XMI, XMM file formats are to not be decompressed at loading time.
This saves the decompression overhead a PNG file would have to go under. There is optional compression. The only time you compress these files are to transfer them over, say the internet. There is a utility that packs and unpacks the files. The result is an XMP file.
See the Quick Start Guide on readme.md
The reason to use my file format is just that - speed of loading. It is DEFINITELY better than using PNG files. You can see a list of advantages by file format on the wiki called Advantages. Please note I kept XMM file out of the bunch, because it is basically a pack of XMI images to be used as mipmap levels.
The Advantages article touches on DDS files - the biggest competitor. DDS files are stored on the GPU then decompressed and read there. The bandwidth is shorter, but there is still the overhead of decompression. DDS files have lossy compression methods, which are terrible for gradients… so I read. While the GPUs are powerful, there is no need to do compression.
If not use XM files, use Direct Draw Surface files.
In time, XM is going to feature a heightmap file and as of about 10 minutes ago, a moving image - gif style.
So let me get this straight. The only thing that your format does better than PNG is that it’s slightly faster to load and decompresses at loading time?
This may only be true for me but I’ve never had to wait for a PNG to load in my game, so speed doesn’t matter. I don’t get how the second one makes it better.
So at the end of the day you really just cloned PNG.
My file format is always uncompressed. When you transfer over the network the files are compressed to save data. Otherwise, they are not compressed. PNG files are compressed, loaded in then uncompressed then used. XMI files are uncompressed, loaded in and used.
Its not a PNG clone. There are 5 different file formats here, too. XMI is the base for the images.
The whole reason why many popular image formats use compression is to save space. When it comes to games, imagine a modern AAA game that takes up about 6GB of disk space with compression. With your formats, you’ll end up using probably 8-12GB depending on how many images were in the game. They might as well use a BMP file because those are not compressed.
Know what? It doesn’t matter. If you don’t want to decompress, use plain TGA or BMP. In your Wiki you reason that the BMP format has a brazenly large header of ~50 bytes? You just insulted my 4 GZ PC with a SSD and plenty of RAM, and he demands an apology.
My mobile phone told me that it happily trades a few milliseconds of load time for less SD storage any day.
You wrote of checks about face culling (what?) and translucency. There is some context which dictates whether an image must be rendered with alpha-blending or alpha-masking enabled. This is not required to be part of the image data. If your game needs to scan pixels to determine if some of them are non-opaque, you are doing it wrong.
Also, mipmap levels. Please describe a use case where each mipmap level needs its own “transparency” flag. Moreover, last time I worked with mipmaps, the was simply /2. So, the “minimal” data you would actually need to store is plus .
Endianness: There are two major choices, and you take the one 99% of the consumer hardware out there does not use. That’s ok, doesn’t matter much if you handle it carefully. What also doesn’t (should not) matter is whether image data is stored RGBA, ARGB or BGRA. You just need to be consistent. In case of doubt this is - at the latest - just sorted out by some vec4 swizzling after texture sampling in the pixel shader.
One day, you’ll not be working in a vacuum anymore. Artists draw pretty pictures. They want to update and get them into the game pretty fast, and pretty often. They want to do that in a tool they know, with exporters and options they know.
They also want to browse and look at them comfortably, be it with their nice gallery app or just in a file explorer.
They do not want to run arcane bash commands or batch files.
<rant /off>
So where are we now?
As it happens, I can see a reason why one would use such a “family of file formats”: as some form of intermediate data, e.g. for caching, or (pre-)transforming image data into something friendly to the target platform. E.g. some asset build which takes an image and converts to DXT5 for PC, and ETC2 or whatever they use nowadays on mobile. Strictly automated, no front-end of any kind, hands-off for any non-programmers.
But even in that case, I’d rather happily trade some milliseconds and use a more flexible solution.
You guys have to chill. He’s working on a cool project that he obviously enjoys. No point in telling him nobody will ever use it, he’s enjoying coding something. That should be enough for you guys.
It’s like saying “Hey, don’t show me or continue that art project! Nobody will ever buy it, geez.” Who cares? If you don’t care about the project don’t post on it.
The advice that doesn’t say that is great though. CoDi^r’s response was great.
The beef this entire community has with Hydroque is that he’s saying the exact opposite of that. He’s saying he puts the benefit of the masses as his top priority.
I don’t think that Hydroque should neccessarily discontinue the project. I’m just saying that if you’ve posted 5 times in a row and nobody’s responded, then maybe these forums aren’t the right place to discuss the project. And even in the situations where you’re not getting responses but people seem interested, there are proper ways to go about bypassing that limit (message a mod, or find someone who’s interested and ask them to comment). Creating a new thread to continue the WIP-spamming is somewhat rude.
You over exaggerated. It has a large header of 54 bytes, 4 bytes should do. Its old. Thats one part of the argument that goes with color table look up, supporting all BMP file formats, and the crappy storage of the pixels thereof. Not to mention BGR. BMP shouldn’t be used, and you complaining about your precious HDD space is a little odd in your following text.
I would gladly choose a PNG file for phone… or DDS. Yeah DDS.
To elaborate, when you want face culling on plants and objects, using face culling is something to think about. When you enable culling for plants, you get a funky 2/4 face plant with culling on. This simple little boolean allows checks for just that. Using OpenGL’s alpha enables isn’t a good thing to do. Discarding pixels in shaders so it goes.
You are right. Each mipmap level doesn’t need a transparency boolean on its own. There is the global one after all. So I guess I will rip that right out of there I am not sure what you were referring to with the size. You never negate one when multiplying to figure out how many bytes are used. This isn’t array offsets.
That is if you use all one image file format indeed. That argument doesn’t mean much.
True. You do realize I will be writing plugins and such for these formats?
Lol funny. XM is just a title. The real formats aren’t even XM. XMI, XMM, XMP, XMH, and XMV.
Actually, I am an entirely selfless person with no more interest than to give my everything to people. ._.
…and I never said that.
Thanks for letting me know that WIP threads have a post limit before you can’t post anymore. I didn’t know this, and I won’t be posted any WIP threads. I made this thread in a more specific place. No conversation sparked over there because I typed a well essay. It wasn’t my intent to make another WIP thread.
Nonetheless the project is good for at least me. It is of benefit to me. Therefore I shall use it and expand on it as I go. Its not a bad format, and once I write a few different things for editors all will be cool.
Please read again. I do not complain about HDD space at all. I complain about trying to solve a problem I don’t agree we even have. CPUs are fast (decoding at load time), SSDs are freaking fast (seeking over a few extra header bytes), and there is plenty of RAM (if some is even needed).
Again, wrong problem space. Face culling is a geometry property, not one of the texture painted on that geometry.
A mipmap is a series of images. It starts with the original image, level #0. Level #1 contains the same image at half the size (width/2, height/2) of the original. Level #2 contains the same image at half the size (width/2, height/2) of level #1.
Pack Raw Data to zip is you’re format?)
sorry but you not create not Zip algorithm, not PNG - you can’t сall them Yours
Framework - yes, Format - no
One story: Team Lead from my old work - Copy Past VP8 codec, rename output extension to “crapextension”
And gloriously talking everybody that hi create mega cool Video packing format.
and Another one: when i looking for job one company talking that they have mega cool own language like php like php – means they rename php files to “VAV” extension and fools all clients =)
offtop:
I find them
looks like they finaly rename “vav” to “htm”, but look on blog post aligns
and this ppl make Web page for president and banks XD
[spoiler]
If you have question why them? – because corruption, mafia and many-many lies ^^
Nah… noting special in many countries same (if not even all :()
[/spoiler]
Hydroque - No one is jealous of your file format, no one is gutted they didn’t think of it before. No one is going to use this format.
People are just surprised of the arrogance you have in thinking you have created something new that the masses will want to use because you haven’t. Please don’t stop developing your format, you obviously feel very proud for what you have done and you should continue having this enthusiasm when furthering your project.
Hahahahaha. @Longor1996 beat you by almost 3 years, and he used this unoriginal format for a legitimate purpose, rather than advertising it as better than PNG.
Please just quit the arrogance. You’re only making yourself look bad.
