GZIPInputStream

Performance Tuning
#1

Hi everybody !

I’m trying to make my own texture format for lwjgl. Basically, it’s very simple : the image is in raw format, and I gzip the image to reduce its size.

My goal was to reduce the loading time (because using awt image loader is not very fast…). But i’ve made some benchmarks and here are the results :

  • When using the awt loader : 1530 ms
  • When using raw format without gzip compression : 450 ms (3 times faster !!!)
  • When using raw format with gzip compression : 3560 ms

So I deduce that using a GZIPInputStream is very very slow…
Does someone know where to find a faster “ungziper” or how could I do to make the decompression faster ?

Thanks for your help
++
Chman

EDIT: Oh, and forget about my english :slight_smile:

#2

no idea really. The only thing I can think of is to look at using java.util.zip.Deflator directly if it is just that the input stream is slow. But that is ZLIB (GZip uses ZLIB but is somewhat different), and I don’t know how much faster you could make things run that way. You could also look for other gzip decoders (I can’t remember any, but a search should find something).

#3

Hum… There are not so many gzip nor zip decoders for java on the web… The ones I’ve found were very slow, like in the JDK…

Any advices for my image loader ?

#4

I don’t know about any fast unzippers… but what I would suggest is unzipping in the background as you need textures. That is predictively determine what textures are going to be needed before too long and start loading it.

#5

http://www.jcraft.com/jzlib/index.html this is the one other java compression lib I know of.

Ant knows how to do bzip2 but that is more CPU intensive than gzip http://ant.apache.org/manual/CoreTasks/pack.html

#6

You’re coming at it from the wrong angle Ch*man!
Zipping is only important when you’re downloading stuff over the net. When it’s local - store it in unzipped form! Then it’s fast. And when you download it - well, it’s zipped, of course. So it’s fast then, too.

Cas :slight_smile:

#7

Hmm princec beat me to it.

Well keep your images gzip in your distribution. Then you can unzip all the files the first time the game is launched or when it is installed. HDD storage is cheap and most people have to much of it. I prefer a fast loading game over some 10-50mbs of HDD space anytime.

#8

Yes, I understand, but zipped image uses 80ko, whereas unzipped it uses more than 700ko…

#9

Disk space is hugely plentiful and abundant, and bandwidth ain’t. So who cares if it’s half a meg when it’s unzipped? I only care that it takes 10 seconds to download!

You ought to be more worried about squeezing it into graphics card memory than the speed it unzips at! Look into S3TC texture compression to shrink your textures a bit.

Cas :slight_smile:

#10

Ok, thanks for your help men !
I will use S3TC… it seems to be good ! Do every graphics cards can handle this ?

#11

No, only modern ones. You’ll have to check for the extension.

If I recall, it doesn’t save on disk space, but when you upload the texture to the card, the drivers convert it to the compressed format of your choice (in glTexImage2d)

Cas :slight_smile:

#12

yeah I know :slight_smile:

Ok, now my texture loader works well with S3TC…
Thanks for your help !

#13

Actually the best thing about the extension is the ability to ignore specific formats. I can upload a texture with the GL_COMPRESSED_RGB(A) format and it will simply choose an appropriate format. That’s the real spirit of OpenGL and comes as core in version 1.3 or 1.4 (don’t remember exaclty). You will only need the more specific S3TC extension if you need to update your textures on the fly.

  • elias
#14

Isn’t IO the slowest operation though :S

Loading 20megs of compressed data should take rougly half as long as 40megs of uncompressed.
If you can then decompress the 20megs in less time than it would take to finish reading in the 40megs - you should end up with a faster loading, and smaller program.

Compression is going to get more and more important as the discrepancy between Storage and processing speed increases.

#15

But we’re talking compression as an internal texture format, so the texture will take up less of the precious video memory. The extension has nothing to do with disk format (even though you can retrieve the texture in the driver specific compressed format).

  • elias
#16

oh my response was relevant 5posts earlier :smiley: