Core stability

[eMKa]

I’m thinking about the best way to integrate Yuri’s texture loading way in TextureLoader2… I saw 3 potential places to put it in :

1. Create a “generic” TextureStreamLoader
2. Create a generic ImageComponentLoader
3. Replace this call by something better :

else {
                    try {
                        BufferedImage img = ImageIO.read(in);
                        
                        if (img != null) {
                            tex = createTexture(img, flipVertically, format, mipmap);
                        }
                    } catch (Throwable ex) {
                        ex.printStackTrace();
                    }

Other question : aren’t TextureStreamLoader/ImageComponentLoader added automatically ?

[eMKa]

I choosed solution 3…

Getting errors with some JPEGs…

Texture [src/Textures/gelo.JPG] has not been found in any locator
java.lang.IllegalArgumentException: ImageReadParam num source & dest bands differ!
	at javax.imageio.ImageReader.checkReadParamBandSettings(ImageReader.java:2746)
	at com.sun.imageio.plugins.jpeg.JPEGImageReader.readInternal(JPEGImageReader.java:907)
	at com.sun.imageio.plugins.jpeg.JPEGImageReader.read(JPEGImageReader.java:864)
	at com.xith3d.loaders.texture.TextureLoader.read(TextureLoader.java:397)
	at com.xith3d.loaders.texture.TextureLoader.loadFromStream(TextureLoader.java:349)
	at com.xith3d.loaders.texture.TextureLoader.getTexture(TextureLoader.java:788)
	at com.xith3d.loaders.texture.TextureLoader.getTexture(TextureLoader.java:458)
	at com.xith3d.loaders.texture.TextureLoader.getTexture(TextureLoader.java:443)
	at org.xith3d.ui.hud.widgets.Image$Description.<init>(Image.java:85)
	at org.xith3d.ui.hud.widgets.Image.<init>(Image.java:257)
	at org.xith3d.ui.hud.widgets.Label.setBackground(Label.java:312)
	at game.Game.createHUD(Game.java:561)
	at game.Game.<init>(Game.java:1314)
	at game.Game.main(Game.java:1349)

What does this mean ?

Yuri is the approach fast because a DirectBufferedImage is used ?

The current code is as follow :

                                BufferedImage dst = null;
				if(format == RGB) {
					dst = DirectBufferedImage.getDirectImageRGB(iw, ih);
				} else if(format == RGBA) {
					dst = DirectBufferedImage.getDirectImageRGBA(iw, ih);
				}

but with that code :

dst = reader.getImageTypes(0).next().createBufferedImage(iw, ih);

I never get any errors

[eMKa]

Okay, so I don’t know how to make DirectBufferedImage for each image now so I stay with BufferedImage but Yuri if you have an idea of how to fix that take a look at the CVS.

[yvg]

Hi,

After reviewing the code in texture shader, ImageComponent2D and DirectBufferedImage, I believe that we sould introduce some changes also there.

The general point on loading textures is that we copy them way too much - at least 3 times when DirectBufferedImage is used, and 4 times when using plain BufferedImage. Textures often are big, so this consumes memory and increases the startup time.

For regular BufferedImage, procedure is following:

A1. Read image and store it to data array allocated internally by ImageIO
A2. Allocate new byte[] in ImageComponent2D and pull data out of BufferedImage from step A1 using pixel processor
A3. Allocate DirectByteBuffer in ImageComponent2D and put data from byte[] from step A2 into it
A4. Pass a reference to DirectByteBuffer created at A3 to OpenGL context, which will allocate one more array to hold texture data in driver space

Step A4 we can not avoid anyway… (there are some ideas on that, but I am not so sure they will work).

Now what is happening with DirectBufferedImage.

B1. Create new DirectBufferedImage of appropriate type - it will create the byte[] (the Backing Store) similar to one created on A2 and hold reference to it
B2. Read image DIRECTLY into DirectBufferedImage - this will store pixel data into Backing Store byte[] and will avoid of extra byte[] allocation inside ImageIO
B3. Allocate DirectByteBuffer in ImageComponent2D and put data from byte[] from step B1 into it
B4. Pass a reference to DirectByteBuffer created at B3 to OpenGL context,

…and how it should be at the end:

C1. Create new DirectBufferedImage of appropriate type - it will create the DirectByteBuffer (the Backing Buffer) similar to one created on B3 and hold reference to it
C2. Read image DIRECTLY into DirectBufferedImage - this will store pixel data into Backing Buffer DirectByteBuffer. This is way more tricky, but I have few implementations of this which are fast enough (I did that for progress-trackable texture loading which shows the progress of ImageIO decoding the image.
C3. Create ImageComponent2D such a way that no new DirectByteBuffer is created but one from step C1 used
C4. Pass a reference to DirectByteBuffer created at C1 to OpenGL context

You can see that we can get rid of a lot of pixel processing this way, but this only fits Power-Of-Two-sized textures, and others have to be loaded conventional way.

Yuri

[yvg]

Hi,

In general, you should support images of up to 4 bands:

1-band (grayscale) - for Alpha or Luminance textures
2-band - For Luminance+Alpha textures
3-band for RGB
4-band for RGBA.

There are methods for creating DirectBufferedImage for cases of 1, 3 and 4 bands. We probably should also support 2-band. Check com.xith3d.image.DirectBufferedImage for details.

Yuri

[yvg]

Hi,

[quote]Step A4 we can not avoid anyway… (there are some ideas on that, but I am not so sure they will work).
[/quote]
I just checked the state-of-the-art OpenGL extensions, and there are ways how to avoid this and make texture download to the driver even faster.

Yuri

[eMKa]

Okay, I guess this issue is assigned to you? (understand : I won’t work on it anymore)

[yvg]

Hi,

[quote]Okay, I guess this issue is assigned to you? (understand : I won’t work on it anymore)
[/quote]
Not really, I just explain how things work.

What I can (and probably will) do is to change DirectBufferedImage implementation so it is using DirectByteBuffers. Integration with TextureLoader is your part.

A4 step acceleration is tricky, but it after may make use of DMA transfers directly from user space to AGP/PCI/PCI-e memory or even decode image data directly there.

I am not so sure if we should use this because of side effect is neccesity to put all textures in single DirectByteBuffer (i.e. implement a kind of texture memory allocation ourself).

Yuri

[eMKa]

@Yuri Vl. Gushchin

Not really, I just explain how things work.
What I can (and probably will) do is to change DirectBufferedImage implementation so it is using DirectByteBuffers. Integration with TextureLoader is your part.
A4 step acceleration is tricky, but it after may make use of DMA transfers directly from user space to AGP/PCI/PCI-e memory or even decode image data directly there.
I am not so sure if we should use this because of side effect is neccesity to put all textures in single DirectByteBuffer (i.e. implement a kind of texture memory allocation ourself).

Hmm ok but then what about the not-same-number-of-bands error ? Some changes in DirectBufferedImage also? (maybe add a “bands” params in createDirectRGB(A)() methods?)

[yvg]

Hi,

I would like to get images that produce these errors for further investigation.

I had only a problem with bands when was trying to load RBG texture into RGBA destination, and Grayscale texture in RGB/RGBA destination.

Yuri

[darkprophet]

I just checked the state-of-the-art OpenGL extensions, and there are ways how to avoid this and make texture download to the driver even faster.

Texture download? I think you mean texture upload ? (since the graphics driver is the server and the application is the client). In anycase, im wondering what that “state of the art” extension is? If its PBOs, i dont think they’ll help you much with the glTexImage2D call…

DP

[yvg]

Hi,

[quote]Texture download? I think you mean texture upload ? (since the graphics driver is the server and the application is the client).
[/quote]
I meant transferring texture from the encoded state (jpeg/png/*** image or stream) to the ready-to-render state (in video memory).

OK, depending from which side to look. “Texture Download” wording taken directly from NVidia’s “Fast Texture Downloads and Readbacks using Pixel Buffer Objects in OpenGL” technical brief, so I just re-used the term (OK, this brief is around one year old, but I was checking this topic way earlier). So if you don’t mind I will continue to use the same term, besides I agree 100% that from the Application Side it looks like “Upload Textures To The Driver/Video Memory”.

Yes, you are right about PBOs, but I can not 100% agree ok that they will not help. With faster texture download I target not only the FPS reached, but faster application startup and lower memory usage during startup. And the latter is a point where PBOs may help I think.

Another application of PBOs I am targeting personally is a video streaming to texture - I intensively use this feature, and if I can parallelize decoding and texture streaming plus utilize DMA transfers using, say, PDR (Pixel Data Range) (which I personally don’t like too much), then I can reduce CPU load for video playback by avoiding at least one memory-to-memory transfer.

Yuri

[darkprophet]

for video streaming, PBOs will help you, definetly.

That paper only shows how different texture formats affect sending data to the server and how readback can be affected. The only hint of using it to upload textures is this paragraph:

“To download data to the GPU, bind the buffer to the GL_PIXEL_UNPACK_BUFFER target using glBindBufferARB(). The glTexImage2D() command will then unpack (read) their data from the buffer object.”

This unpacking, could mean a copy, you dont know…its up to the implementation. If you carefully look at the figures in the table, using BGRA instead of RGBA affects transfer “downloads” by 6.75% if you used BGRA. So the card will store BGRA in video memory, its just the transfer to memory. FPS will not be changed, but like you said, startup is changed.

In conclusion, PBOs are going to help the normal game citizen, they help readbacks. Using the optimal texture format will only help transfer rates initally, FPS during gameplay remains unchanged.

_{All figures are quoted from the 7800GT}

DP

[yvg]

Hi,

I agree with one minor comment: I easily believe that this unpacking procedure made by glTexImage2D may perform quite different if there is no real data transformation changed (they call this “swizzling” in that paper), so if you observe 7800GT performance for 8-bit Fixed Point RGBA and BGRA (columns 1 and 2 on 7800 GT line on Download), you will find quite big difference - 483 MB/s for RGBA vs 2238 MB/s for BGRA, which is, say, a kind of difference.

Yuri

[darkprophet]

Ooops, i was reading the wrong section; so BGRA8 is infact a heck of alot faster than RGBA8…im not familiar with texture formats, but I heard that TGA is practically raw with a header, if it is raw, your going to have to do the swizzling in CPU (since the data is RGBA packed) and that might offset the speed of transfer…

Also, you might be interested to know, that a 512x512 DXT1A texture with all its mipmaps takes less space (and that correlates to transfers too) than mipmap level 0 of the same 512x512 texture in RGBA format…

At the end of the day, unless your doing alot of readbacks, texture compression is going to be better…

DP

[yvg]

Hi,

[quote]Also, you might be interested to know, that a 512x512 DXT1A texture with all its mipmaps takes less space (and that correlates to transfers too) than mipmap level 0 of the same 512x512 texture in RGBA format…
[/quote]
Yes, I agree, and even more - we have support for comprssed texture formats already implemented and functional. But if you consider also Internet-downloadable applications, then you should also consider the transport size and - as you can guess - JPEG or JPEG2000 images will be way smaller than DXT1 compressed ones. But, yes, use of DXT1 compressed images saves you from doing decompression - it all is done by hardware, and it is efinitely the best choice. Even for smaller textures DXT1 image may be smaller than JPEG, and this saves A LOT of performance.

…as always, there should be a balance…

Yuri

[darkprophet]

A 512x512 DXT1A DDS texture takes 171KB with all mipmaps (so that includes 256x256, 128x128, 64x64, 32x32, 16x16, 8x8, 4x4, 2x2 and 1x1). If you zip it (with winRAR), it becomes 125Kb…

try doing that with JPEG and having the same quality as DXT1A. Also, the texture that I have has an alpha channel, JPEG dont have alpha channels, so its not a real 1-1 comparison, but close enough

Then you have to think of the amount of RAM that uncompressed JPEG is going to take, all in all, JPEG is a horrible solution for games, no matter which way you look at it.

DP

[yvg]

Hi,

[quote]JPEG dont have alpha channels
[/quote]
False. It is tricky to produce, but it does. I use “custom build JRE” to be able to save such a jpegs, and it even lets me to create textures where quality and subsampling of alpha and color channels are different.

Yuri

[yvg]

Hi,

[quote]Then you have to think of the amount of RAM that uncompressed JPEG is going to take, all in all, JPEG is a horrible solution for games, no matter which way you look at it.
[/quote]
Yes, you are right in some extent, but at the end compressed textures can also be downloaded (uploaded - you nearly converted me to this term) using PBOs, which [may] save some RAM on startup.

Yuri

[Qudus]

@Yuri Vl. Gushchin

[quote]JPEG dont have alpha channels

False. It is tricky to produce, but it does. I use “custom build JRE” to be able to save such a jpegs, and it even lets me to create textures where quality and subsampling of alpha and color channels are different.
[/quote]
The GIMP says, that JPEGs don’t support them. But you could of course use a specific color like BLACK, to indicate some kind of user-defined-transparency.

As I mentioned in another thread, Java-ImangeIO reading is not sayd to be the fastest way to load textures. Maybe we should consider porting some open sourced image loading code instead of using ImageIO.

Marvin