png reading

Performance Tuning
#1

My game allows scrolling of large maps. Because the entire map doesnt fit into memory, I have broken the map into smaller tiles, as the user scrolls the map, the tiles are loaded into memory (I do some fancy stuff to pre load tiles the user may scroll to next, and I keep previously loaded tiles in memory with a soft reference in case they are re used).

When I load a tile, I convert the tile to a BufferImage that is compatable with the screen. The process is,

  1. Load the .png file with ImageIO to a BufferedImage
  2. create a new bufferedImage, and copy the tile from 1) to the new image
  3. cache the image from 2)
  4. draw the cached image to the screen when needed

Steps 1,2 and 4 are very fast. Step 1 takes about 7ms, and step 4 takes about 1ms (256x256 png images).

Step 3 takes about 30ms, using the code below,

                URL imageLocation = ...
                BufferedImage fromFile = ImageIO.read(imageLocation);
                BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_4BYTE_ABGR);
                Graphics g = image.getGraphics();
                g.drawImage(fromFile, 0,0, null);
                g.dispose();

It seems very odd that it takes longer to transform the image in memory than it does to read it compressed from disk.

My question is, can I read the file from memory into a BufferedImage of arbitrary Type, skipping the intermediate step?

I tried using ImageReadParam and setDestination, with the code below (stolen from code in ImageIO),

    InputStream istream = null;
    try {
        istream = input.openStream();
    } catch (IOException e) {
        throw new IIOException("Can't get input stream from URL!", e);
    }
    ImageInputStream stream = ImageIO.createImageInputStream(istream);

    Iterator iter = ImageIO.getImageReaders(stream);
    if (!iter.hasNext()) {
        return null;
    }

    ImageReader reader = (ImageReader)iter.next();
    ImageReadParam param =  new ImageReadParam(); 
    BufferedImage destination = new BufferedImage(width, height, BufferedImage.TYPE_4BYTE_ABGR);
    param.setDestination(destination);
    reader.setInput(stream, true, true);
    BufferedImage bi = reader.read(0, param);
    stream.close();
    reader.dispose();
    return bi;

But this throws,

Exception in thread “Map panel background drawer” java.lang.IllegalArgumentException: ImageReadParam num source & dest bands differ!
at javax.imageio.ImageReader.checkReadParamBandSettings(ImageReader.java:2746)
at com.sun.imageio.plugins.png.PNGImageReader.readImage(PNGImageReader.java:1357)
at com.sun.imageio.plugins.png.PNGImageReader.read(PNGImageReader.java:1530)
at games.strategy.triplea.image.TileImageFactory.createImageDirectly(TileImageFactory.java:243)
at games.strategy.triplea.image.TileImageFactory.startLoadingImage(TileImageFactory.java:179)
at games.strategy.triplea.image.TileImageFactory.getImage(TileImageFactory.java:141)
at games.strategy.triplea.image.TileImageFactory.getBaseTile(TileImageFactory.java:110)
at games.strategy.triplea.ui.screen.BaseMapDrawable.getImage(IDrawable.java:307)
at games.strategy.triplea.ui.screen.MapTileDrawable.draw(IDrawable.java:238)
at games.strategy.triplea.ui.screen.Tile.draw(Tile.java:126)
at games.strategy.triplea.ui.screen.Tile.getImage(Tile.java:88)
at games.strategy.triplea.ui.BackgroundDrawer.run(MapPanel.java:853)
at java.lang.Thread.run(Thread.java:595)

Thanks for any help.

#2

When you read in the image, it gets loaded in the format it is stored on disc.

Its not surprising the converting format is taking so long. That means pulling appart and repackign every single pixel in your source.

Id suggest you either store the image in the format you are going to use at run-time or you make your run-time format match your disc format.

#3

I also agree that you should try some kind of raw format instead. Size wise it wont be a big difference actually. Once its jarred the size will be about the same, but loading it will be way quicker. Java’s zip (jar) stuff is highly optimized so decompression will be pretty fast.

#4

Thanks for the help.

How does one read/write images that are of type BufferedImage.TYPE_4BYTE_ABGR?

#5

There is something else you could try:

http://www.java-gaming.org/forums/index.php?topic=9200.0

basically its a very fast texture-transforming algorithm. It’s written to produce ^2-size 4BYTE_ABGR images given an arbitrary image. I have a gut feeling it will be alot faster than Java2Ds internal transformation methods. I think you could hack it to do what you want. (disclaimer: I wrote that thing :P). If you decide to use it, please let me know your benchmark results, I am quite interested.

#6

Well, you want a file format that saves 32 bit pixels with a byte for each component and the components in the
order Alpha-Blue-Green-Red (thats usually high bit to low bit.). The easiest thing mihgt be to make your own format. Do the conversion as above then open a file, write the width, write the height, and then scan the BufferedImage and write out the bits.

To load you’ld just reverse that.

#7

Looking at your orignal post, it seems the link to the get2dfold algorithm is broken,

  "you will need the "get2fold" algorithm, as mentioned in this post:

http://www.java-gaming.org/cgi-bin/JGNetForums/YaBB.cgi?board=share;action=display;num=1117186907"

#8

http://www.java-gaming.org/forums/index.php?topic=9198.0

#9

I am not sure whether this helps at all but i did similar thing a while ago for a game which progressed quite far be fore i put it on hold: http://goldenstudios.or.id/products/games/index.php#ANACONDA%20NET

While it is designed to be used with the GTGE library, it should be easily modified to use any other library, or just plain Java2D :).

I had to base it off of a decompilation of the com.golden.gamedev.object.background.abstraction.AbstractTileBackground class found in the GTGE engine so the render method’s paramters are not easily understandable, however the java docs for the AbstractTileBackground can be found: http://goldenstudios.or.id/products/GTGE/index.php#documentation

The main features of this code:

-images are loaded on a seperate thread and thus do not interrupt the main game thread
-can pre-cache the the file data to avoid disk access (but uses more memory)
-semi intelligently looks ahead depending on a given heading.
-automatically centres the screen.

#10

ProgressiveTileBackground.java part 1


package com.moogiesoft.AnacondaNet.render;
import com.golden.gamedev.object.background.abstraction.AbstractTileBackground;
import com.golden.gamedev.util.ImageUtil;
import com.golden.gamedev.GameObject;

import java.awt.Graphics2D;
import java.awt.Transparency;
import java.awt.image.BufferedImage;
import java.util.*;
import java.util.Iterator;
import java.io.*;
import javax.imageio.*;
import java.net.*;

/**
* ProgressiveTileBackground only keeps a limited number of tiles in memory at any point in the game. 

* This allows use for an arbitrary sized background image that has been tiled previously giving the 

* ability of a massive background with no-repeating tiles.

* 
  
* Only the tiles that are immediately needed are kept. These tiles consist of the tiles involved in the rendering of the view area

* and as well as tiles which are identified as needed to be pre-fetched in the case of a scrolling game.

* 

* There is the option of pre-loading the tile images. When set, this will make the ProgressiveTileBackground load all of the

* COMPRESSED tile images into memory but not yet decompressed into a BuffedImage. Only the tiles that are requested are then decompressed.


* If not set, then the tiles are loaded and decompressed at run time.
* 
* The pre-loading of the tile images will increase the memory requirement somewhat, however it will improve performance.
* 
*/

public class ProgressiveTileBackground extends AbstractTileBackground implements Runnable
{
    private HashMap tiles; 						// The cache of tile images 
    private int cacheCounter;					// The current number of tiles in the the cache.
    private int cacheCounterLimit;				// The maximum number of tiles allowed in the cache.
    private BufferedImage blankImage;			// The default blank image.
    private LinkedList imagesToLoad;			// A list of tiles requested to be "loaded"
    private LinkedList destoryTileLinkedList;	// A list of tiles requested to be destoryed and garbage collected


    private int tileWidth;						// The width of a tile
    private int tileHeight;						// The height of a tile
    private int horizontalCount;				// The number of horizontal rows in the background
    private int verticalCount;					// The number of vertical columns in the background
    private int heading;						// The direction hint for pre-fetching images

    static final int BUFFER_ZONE_SIZE=1;		// The number of rings of tiles around the viewable area (for scrolling purposes)

    private int leftBoundary;					// The number of columns to the left of the viewable area to pre-fetch;
    private int rightBoundary;					// The number of columns to the right of the viewable area to pre-fetch;
    private int bottomBoundary;					// The number of columns to the top of the viewable area to pre-fetch;
    private int topBoundary;					// The number of columns to the bottom of the viewable area to pre-fetch;
    public boolean alive;						// A flag to destroy the "image loading" thread. 
    private boolean preLoadCompressedImages;	// A flag determining the pre-loading of compressed images or on-the-fly loading
    private ByteArrayInputStream[] compressedImageStreams; // An array containing the compressed tile images
    private GameObject game;					// The instance to the GameObject which owns this background
    private ProgressiveTileRelationship relationship;	// The tile Relationship class assosiated with this background.

    

    
	/**
	*  Default Constructor, private as you can only get an instance of ProgressiveBacktground via the static createBackground() methods
	*/
	
    private ProgressiveTileBackground()
    {
        super(0,0,0,0);
    }
    
    /**
     * 
     * Gives the background render hints as to which sides of the screen to pre-fetch.
     * 
     * The heading is a angle between 0-360 degrees 
     * 
	* @param degrees an angle between 0-360 degrees
	*/
	
    public void setHeading(int degrees)
    {
        heading=degrees;
    }
    
    /**
     * Stops the tile image loading thread.
     * 
     * This should be called before loosing the reference to a ProgressiveTileBackground instance
     *
     */
    
    public void stop()
    {
        alive=false;
    }
    
    /**
     * A factory method to create ProgressiveTileBackground instances.
     *  
     * @param game the instance of the GameObject which creates this ProgressiveTileBackground.
     * @param relationship a class which uses the ProgressiveTileRelationship interface.
     * @param preLoadCompressedImages when set the ProgressiveTileBackground will load all the compressed images into memory and only de-compress the images as needed.
when not set the images are loaded and decompressed on the fly.
     * @return the ProgressiveTileBackground instance.
     */
    
	public static ProgressiveTileBackground createBackground(GameObject game,ProgressiveTileRelationship relationship,boolean preLoadCompressedImages)
	{
		return createBackground(game,relationship,100,"Progressive Background",preLoadCompressedImages);
	}
    
    /**
     * The standard constructor for the ProgressiveTileBackground.
     *
     * @param game the instance of the GameObject which creates this ProgressiveTileBackground.
     * @param relationship a class which uses the ProgressiveTileRelationship interface.
     * @param cachelimit the maximum number of images to keep in a memory cache
     * @param name the name to give the ImageLoading thread associated with this ProgressiveTileBackground.
     * @return the ProgressiveTileBackground instance. 
     */
    
    public static ProgressiveTileBackground createBackground(GameObject game,ProgressiveTileRelationship relationship,int cachelimit,String name,boolean preLoadCompressedImages)
    {
		byte[] buf=new byte[32*1024];
		URL url;
		DataInputStream dis;
		ByteArrayInputStream[] compressedImageStreams=null;
        try
        {
            
            int tileWidth=relationship.getTileWidth();
            int tileHeight=relationship.getTileHeight();
#11

ProgressiveTileBackground.java part 2


            int horizontalCount=relationship.getHorizontalCount();
            int verticalCount=relationship.getVerticalCount();
            
            if (preLoadCompressedImages)
            {
	            
	            compressedImageStreams=new ByteArrayInputStream[horizontalCount*verticalCount];
	            for (int i=0;i<horizontalCount;i++)
	            {
	                for (int j=0;j<verticalCount;j++)
	                {
	                    
	                    ByteArrayOutputStream baos=new ByteArrayOutputStream();
	                   
	                    String fileName=relationship.getTileFileName(i,j);
	                    url=game.bsLoader.getBaseIO().getURL(fileName);
	                    dis=new DataInputStream(url.openStream());
	                    int read=dis.read(buf);
	
	                    while (read>-1)
	                    {
	                        baos.write(buf,0,read);
	                        read=dis.read(buf);
	                        
	                    }
	
	                    dis.close();
	                    baos.close();
	                    compressedImageStreams[j*horizontalCount+i]=new ByteArrayInputStream(baos.toByteArray());
	                    compressedImageStreams[j*horizontalCount+i].reset();
	                }
	            }
            }
            return new ProgressiveTileBackground(game,relationship,compressedImageStreams,tileWidth,tileHeight,horizontalCount,verticalCount,cachelimit,name,preLoadCompressedImages);
            
        }
        catch (IOException e)
        {
            e.printStackTrace();
            System.exit(1);
        }
        return null;
    }
    
    /**
     * Constructor
     * 
     */
  
    private ProgressiveTileBackground(GameObject game,ProgressiveTileRelationship relationship,ByteArrayInputStream[] compressedImageStreams,int tileWidth,int tileHeight,int horizontalCount,int verticalCount,int cachelimit,String name,boolean preLoadCompressedImages)
    {
        super(horizontalCount, verticalCount,tileWidth,tileHeight);
        this.game=game;
        this.relationship=relationship;

        blankImage= ImageUtil.createImage( tileWidth,tileHeight, Transparency.OPAQUE);
        tiles=new HashMap();
        imagesToLoad=new LinkedList();
        destoryTileLinkedList=new LinkedList();
        this.cacheCounterLimit=cachelimit;


        this.tileWidth=tileWidth;
        this.tileHeight=tileHeight;
        this.horizontalCount=horizontalCount;
        this.verticalCount=verticalCount;
        this.compressedImageStreams=compressedImageStreams;
       
        new Thread(this,"background: "+name).start();
    }


	/**
	* overwrites the AbstractTileBackground renderTile Method 
	*/
	
    public void renderTile(Graphics2D graphics2d, int tileX, int tileY, int posX, int posY)
    {
            graphics2d.drawImage(getTileImage(tileX,tileY), posX, posY, null);
            
    }
    
    /**
     * Retrieves the image of the tile from the tile cache given the x and y co-ordinates.
     * If the image is not in the cache then it is created and added to the cache.
     * If the cache is full and the image is not in the cache then a default blank image is returned.
     *
     * @param i the X-tile co-ordinate
     * @param j the Y-tile co-ordinate
     * @return the image retrieved
     */

    public BufferedImage getTileImage(int i,int j)
    {
        Tile tile=(Tile)tiles.get(new Integer(j*horizontalCount+i));
        
        if (tile==null)
        {
            tile=new Tile();
            imagesToLoad.add(tile);
            tile.i=i;
            tile.j=j;
            tile.used=2;
            tile.key=new Integer(j*horizontalCount+i);
            tiles.put(tile.key,tile);
            return blankImage;
        }
        else
        {
            tile.used=2;
            return (tile.image==null)?blankImage:tile.image;
        }
    }
    
    /**
     * Converts the stored compressed byte stream for a tile into a buffered image.
     * 
     * @param i the X-tile co-ordinate
     * @param j the Y-tile co-ordinate
     * @return the decomrpessed image;
     * @throws IOException
     */
    
    private BufferedImage loadImage(int i,int j) throws IOException
    {
		if (preLoadCompressedImages)
		{
			compressedImageStreams[j*horizontalCount+i].reset();
	            
	        BufferedImage image = ImageUtil.createImage( tileWidth,tileHeight, Transparency.OPAQUE);
	        BufferedImage tempBuffImage= ImageIO.read(compressedImageStreams[j*horizontalCount+i]);
		    Graphics2D g = image.createGraphics();
		    g.drawImage(tempBuffImage, 0, 0, null);
		    g.dispose();
		    return image;
		}
		
		BufferedImage image = ImageUtil.createImage( tileWidth,tileHeight, Transparency.OPAQUE);
		BufferedImage tempBuffImage= ImageIO.read(game.bsLoader.getBaseIO().getURL(relationship.getTileFileName(i,j)).openStream());
		Graphics2D g = image.createGraphics();
		g.drawImage(tempBuffImage, 0, 0, null);
		g.dispose();
		return image;		

    }
    
	/**
	* overwrites the AbstractTileBackground render Method 
	*/
    
    public void render(Graphics2D graphics2d, int i, int j, int k, int l, int i1, int j1)
    {
        
        if (heading<90 || heading>270) 
        {
            rightBoundary=BUFFER_ZONE_SIZE;
            leftBoundary=0;
        }
        else
        {
            rightBoundary=0;
            leftBoundary=BUFFER_ZONE_SIZE;
        }  
        
        if (heading>180) 
        {
            topBoundary=BUFFER_ZONE_SIZE;
            bottomBoundary=0;
        }
        else
        {
            topBoundary=0;
            bottomBoundary=BUFFER_ZONE_SIZE;
        } 
        

      
        
        int l1 = l - getOffsetY()- getTileHeight()*topBoundary;
        int i2 = k + i1+getTileWidth()*rightBoundary;
        int j2 = l + j1+getTileHeight()*bottomBoundary;
        int l2 = getTileY()-1 - topBoundary;
        do
        {
            l2++;
            int k1 = k - getOffsetX()- getTileWidth()*leftBoundary;
            int k2 = getTileX() -1- leftBoundary;
            do
            {
                k2++;
                if (k2>-1 && l2>-1 && k2 < horizontalCount && l2<verticalCount) renderTile(graphics2d, k2, l2, k1, l1);
            } while((k1 += getTileWidth()) < i2);
        } while((l1 += getTileHeight()) < j2);
        clearOldTiles();

        
         
    }
    
    /**
     * The tile loading thread.
     * 
     * 

     * 

     * This thread progressively "loads" the requested tiles from an list of tiles identified to load and adds them to a tile image cache.

     * Each tile in the cache has time to live which is decremented each render cycle. Tiles that are not on screen or being requested are eventually removed from the cache.

     * 

     * The loading is done in a manner to minimize the effect on gameplay.
     */
    
    public void run()
    {
        alive=true;
        Tile tile=null;
        boolean loaded=false;
        while(alive)
        {
            loaded=false;
	        if (imagesToLoad.size()>0)
	        {
	            tile =(Tile) imagesToLoad.getFirst();
	            if (tile.used>0) 
	            {
	            	try
	            	{
	            		tile.image=loadImage(tile.i,tile.j);
	            	}
	            	catch (IOException e)
	            	{
	            		System.out.println(e.getMessage());
	            		e.printStackTrace();
	            		System.exit(0);
	            	}
	            }
	            imagesToLoad.removeFirst();
	            loaded=true;
        
	        }
	        else if (destoryTileLinkedList.size()>0)
	        {
	            tiles.remove(((Tile) destoryTileLinkedList.getFirst()).key);
		        destoryTileLinkedList.removeFirst();
	        }
	        
	        if (cacheCounter>cacheCounterLimit)
		    {
		        
		        Iterator iterator=tiles.values().iterator();
		        
		
		        while(iterator.hasNext())
		        {
		            Tile tile1=(Tile) iterator.next();
		            tile1.used--;
		            if (tile1.used==0)
		            {
		                destoryTileLinkedList.add(tile1);
		            }
		                
		        }
		        
		
		        cacheCounter=0;
		        
		    }
	        else if (!loaded && tile==null || tile.used>0)
            {
	            try
	            {
	                Thread.sleep(20);
	            }
	            catch (InterruptedException e)
	            {
	                e.printStackTrace();
	            }
            }
            tile=null;
        }
    }

	/**
	* reduces the time to live of tiles in the cache.
	*
	*/    

    private void clearOldTiles()
    {
        cacheCounter++;
    }

}
#12

other supporting classes:


package com.moogiesoft.AnacondaNet.render;

import java.io.File;

/**
* Provides a means of converting a tiles X and Y co-ordinates into the tile's filename so that the ProgressiveTileBackground can load the tile's image.
*/

public interface ProgressiveTileRelationship {

	/**
	* returns the corresponding image's filename for a tile with the given X and Y co-ordinates
	* @param x the tile's X co-ordinate
	* @param y the tile's Y co-ordinate
	* @return the tiles's image's filename
	*/
	String getTileFileName(int x,int y);

	/**
	* returns the Tile's width
	* @return returns the Tile's width
	*/
	public int getTileWidth();
	
	/**
	* returns the Tile's height
	* @return returns the Tile's height
	*/
	public int getTileHeight();
	
	/**
	* returns the number of horizontal rows of tiles in the background
	* @return returns the number of horizontal rows of tiles in the background
	*/
	
	public int getHorizontalCount();
	
	/**
	* returns the number of vertical columns of tiles in the background
	* @return returns the number of vertical columns of tiles in the background
	*/
	public int getVerticalCount();
}



package com.moogiesoft.AnacondaNet.render;

import java.awt.image.BufferedImage;

public class Tile
{

    public Tile()
    {
    }

    int used;
    BufferedImage image;
    Integer key;
    int i;
    int j;
}
#13

Actually you should save your file in the recommended formats like jpeg/jpg or gif… cause if you save it as a png file… file compression is very basic…

so I think to optimise your performance you should save your pictures in the “FINAL” format…

just recommeneding no harm done… :slight_smile:

#14

jpeg is lossy with no transparancy mask
gif is 256 color only with a 1 bit mask

png is far more advanced than either format. How exactly are they ‘recommended’?

also, the problem was about dynamic loading of map portions, not about image formats.

#15

lolz… opps… guess my teacher must have taught my class the wrong thing then… :-X

haha… nvm… i have noted it le… :wink:

#16

PNG is in some ways much simpler than JPEG, not more advanced.
It is more advanced than GIF for suer though.

But they serve different purposes. JPEG was optimized for “real” images… the lossy bits are designed to make the loss less noticeable in natural pictures while trading for much greater compression ratios.

If you want advanced that would be JPEG2000 which supports lossless or lossy compression, and tons of fancy features. JPEG2000 really outperforms JPEG, particularly at low bitrates. Unfortunately the JPEG2000 codecs are much slower as a result.

Incidentally the PNG image reader in ImageIO has been improved in Mustang according to the notes from recent weekly builds.

#17

[moan]yay, finally it will work as it should - only took 2 Major revision releases![/moan]

#18

I know how you feel. They still haven’t fixed JPEG. Can you believe the JPEG codec isn’t using SIMD code! Intel had a free library for reading/writing JPEGs that was highly tuned for the intel processors that you would think could have been used… they don’t make it available for free anymore though.

Maybe if I ever have some free time (i.e. never) I will submit a patch to Mustang :slight_smile: