Loading loads of textures the fastest way...

Performance Tuning
#1

Hi. Say, for your 2d OpenGL game (which will be finished by 2010 or so) you’ve to load many 2d textures, stored as compressed PNGs on disc/Jar/whatever. This even applies to a single level, so you’re very keen to use the fastest way to feed these PNGs to your OpenGL card. Which is a really fast way?

Currently I load them via ImageIO to a BufferedImage whose raster.getDataBuffer() I feed to the Ogl texture bind call o.glTexImage2D(…).
However, this takes a long time.
Maybe I could cache those PNGs when they’ve been uncompressed for the first time and write them to plain files (into a cache folder) and then at the next run the game would load them via (mapped) “new I/O” or such?
However, maybe loading large uncompressed files from disc takes even longer than loading smaller compressed ones plus their cpu time needed to uncompress them…?

#2

That’s right. Especially on a laptop!

Cas :slight_smile:

#3

[quote]That’s right. Especially on a laptop!
[/quote]
Ok, then again I’ll take the simple (and faster) way: just load compressed PNGs. :slight_smile:

#4

All the way through ImageIO, into Images, into rasters… is pretty slow. Loading TGAs is more than 10 times faster on my machine. Loading TGAs through a GZipInputStream should be still faster than PNGs through ImageIO (and you get about the same compression - with my test images it was actually a bit better in each case).

However, gzipping em isn’t really necessary if you want to use webstart. You can gzip the whole thing there, it gets decompressed on the client side.

Well, yea decompressed… it’s bigger. If you have a rather slow hdd which can read ~30mb/s, you would need 0.15 seconds for loading a six 512x512 images (eg a skybox).

#5

[quote]All the way through ImageIO, into Images, into rasters… is pretty slow.
[/quote]
Yes. It matters when you start to load many images at once…

[quote]Loading TGAs is more than 10 times faster on my machine.
[/quote]
Uncompressed TGAs?
How do you load them? Do you use a TGA loader (which one?), or do you load the bytes directly and examine its (mini) header for bpp, height, with, etc? (I guess so.)

[quote]Loading TGAs through a GZipInputStream should be still faster than PNGs through ImageIO (and you get about the same compression - with my test images it was actually a bit better in each case).
[/quote]
Using GZiped TGAs could be a good idea; we would have to do some tests. Indeeds, it’s even slighty smaller.
However why should they decompress faster compared to compressed PNGs? (Because Java’s gunzip is faster than ImageIO’s PNG decompressor?)

However, then I couldn’t control the tons of images laying around on disc with a picture viewer anymore… which is nice when using PNG.
Jedit supports GZip txt files, but Xnview doesn’t show GZIped TGAs. :expressionless:

[quote]However, gzipping em isn’t really necessary if you want to use webstart. You can gzip the whole thing there, it gets decompressed on the client side.
[/quote]
You mean the usual Webstart JARs, so the TGAs inside it would be decompressed on the fly at loading time?

#6

Do you use a TGA loader (which one?)

I’m using a modified version of JCD’s tga loader. I’ll make that one public sooner or later.

However why should they decompress faster compared to compressed PNGs?

Because you move less data around and gzip is simpler than png.

You mean the usual Webstart JARs, so the TGAs inside it
would be decompressed on the fly at loading time?

Yes and no. With 1.5(*) you can use 3 flavours of jar:

  1. bla.jar
  2. bla.jar.gz
  3. bla.jar.pack.gz

#1 = [max] zip archive
#2 = [stored] zip archive inside a [max] gz
#3 = [stored] zip archive inside a pack inside a [max] gz

Only the first one get stored in compressed for on the client. However, 2+3 are smaller (through the net) and get decompressed by webstart once (and get stored decompressed in the cache).

So you wouldn’t use compressed images at all. Just plain TGAs.

(* It’s backward compatible to 1.4 etc. If the webstart client doesn’t ask for gzip or p200-gzip compression it will get a jar instead)

Applying additional gzip compression on the TGAs would be slower without any benefits (Except saving space on the user’s hdd - which is rather pointless nowadays. No one cares if your game takes 5 or 17mb on a 120gb hdd).

Well, the topic is a bit complicated… ok actually it’s pretty simple, but you need to write quite alot of text for explaining it. For keeping it simple: just use plain TGAs right now… the packing is the very last step and by that time I’ll have finished that article :wink:

#7

Could someone PLEASE post a TGA TextureLoader that is working?

I have converted my textures from png to tga (photoshop) but I now I seem to have to lost my alpha channel (and the colors are looking very strange as well) ???

Could someone help me out? ::slight_smile:

Thanks in advance,

chris

#8

I just adapted a TGA Loader I found on the net. It works fine now with 24bit and 32bit translucient textures. If someone needs the code, let me know.

chris :smiley:

#9

I could use that :slight_smile:

#10

[quote]I just adapted a TGA Loader I found on the net. It works fine now with 24bit and 32bit translucient textures. If someone needs the code, let me know.
chris :smiley:
[/quote]
I was about to point out that if you were writing TGAs out from photshop 7.0 then it loses the alpha channel enroute even if you have one in your image. You have to get a patch from the Adobe site which fixes it up. Looks like you’re not having that problem but i figured i’d post this anyway incase anyone else gets the same problem. Took me a while to cotton on to it when i encountered it first.

On another PS TGA note though, Do you find that photoshop flips your image vertically (but not horizontally) when you save it ? Theres a flag in the TGA spec (forget which, haven’t looked at my TGA loader in a while) which dictates whether the image is stored with the first pixel at the ‘top left’ position or the ‘bottom right’ position. Photoshop gets it half right :slight_smile: On the other hand maybe its a bug in my loader code … I really must look at it again …

D.

#11

It’s the imageDescriptor byte. The one that comes after pixelDepth. Bit 4 is horizontal flipping (left/right), bit 5 is vertical flipping (top/bottom). Last time I checked Adope photoshop and paint shop pro stored the image differently with regards to vertical flipping. So checking the flag is a good idee :wink:

#12

[quote]It’s the imageDescriptor byte. The one that comes after pixelDepth. Bit 4 is horizontal flipping (left/right), bit 5 is vertical flipping (top/bottom). Last time I checked Adope photoshop and paint shop pro stored the image differently with regards to vertical flipping. So checking the flag is a good idee :wink:
[/quote]
well, sometimes I can only shake my head in despair, had a gander at my TGA loading code which I’ve had around for quite a while originally in C++ form before i ported it across, and what do i find :


        loByte = inStream.readByte();
        //TODO sort out the flipping indicated by the above byte.
        //  from the spec 
        //  Bits 3-0: These bits specify the number of attribute bits per pixel.
        //  Bits 5 & 4:These bits are used to indicate the order in which
        //  pixel data is transferred from the file to the screen.
        //  Bit 4 is for left-to-right ordering and bit 5 is for top-to-
        //  bottom ordering as shown below.
        //  Table 2 - Image Origin
        //  Bits 7 & 6:Must be zero to insure future compatibility.
        //  Image Origin  bit 5 bit 4
        //   bottom left      0 0
        //   bottom right     0 1
        //      top left      1 0
        //      top right     1 1

I suppose i’ll have to get around to ACTUALLY TODO-ing it one of these days :slight_smile:

D.

#13

Where can I find more information on the p200-gzip stuff, I couldn’t find anything about it on either google or java.sun.com. I didn’t even find it referenced in the docs for web-start or jars.

Can someone point me to a tool that can create the p200-gzip files. I’m currently using the defaul jar tool, but I’d like to get better results if possible.

Thanks

Andy.

#14

[quote]Where can I find more information on the p200-gzip stuff, I couldn’t find anything about it on either google or java.sun.com. I didn’t even find it referenced in the docs for web-start or jars.
[/quote]
Indeed:
http://java.sun.com/j2se/1.5.0/docs/tooldocs/#deployment
contains hardly any information.

[quote]Can someone point me to a tool that can create the p200-gzip files. I’m currently using the defaul jar tool, but I’d like to get better results if possible.
[/quote]
In your JDK’s Java/bin folder there should be an executable program named
pack200 (.exe on Windows)
which could be what you’re looking for.

Probably Onyx knows more about this topic.

#15

contains hardly any information.

There is everything you need to know.

But…

…some of the diagrams are incorrectly labeled. For that reason I had to read the whole thing several times until it was clear (until I was able to spot the mistakes ::))

Probably Onyx knows more about this topic.

Yes, I do. I could go ahead and write everything down, but that would take like… 1-2 hours and I rather put it into an article (which is on my todo list for quite a while :-X).

Btw the way I do it uses php and mod_rewrite.

Here is a little demo:
http://kaioa.com/src/tmp/BounceDemo.jnlp

With 1.5 you get “I was a .jar.pack.gz” and with an 1.4 client you get “I was a .jar”. Both are the same application, but I put different label.txt files into the packages for demonstration :wink: