Texture loading on seperate thread

I’m trying to load my textures on a seperate thread so that I can have a progress bar rendered while they load. If I don’t use a seperate thread, then the current thread will block until all the textures are loaded and the progress bar will not get a chance to update.

The obvious solution is to load the textures (and sounds eventually) on a seperate thread which will tell my progress bar to update after each file is loaded and to continue with the game once all have finished. Whenever I do this, all the textures end up being completely white and opaque.

So what I’m wondering is if there is a problem with loading textures on a seperate thread from that which the game runs on. While writing this I actually thought of a couple solutions, however neither would be preferrable to my previously described method.

I could keep a total number of files that need to be loaded and then only load one during each frame so that I can update/render the progress bar afterword and move on to the next file during the next frame (decrementing the total number of files to be loaded after each frame). This will probably work fine if all I want to display is a static screen with a progress bar, but what if I want something else on the screen to be updated at the usual 60+ fps (rotating game logo or something…)?

The other possible solution would be to update/render my progress bar without using my game’s main loop, so that it would be on a seperate thread instead of the texture loading. However, I have a feeling this will still be a problem since I would have to call Display.update() from a seperate thread (and I would like to only have the rendering/updating done in one place in my code).

Any explanation of what I’m experiencing here as well as any solutions (or ideas for solutions) would be appreciated.

Loading textures in a separate thread is fine, but to upload the texture to GL, you’ve got to have a current context for the thread. This can be pretty hairy and inefficient so the trick is to poll your disk-loading thread for newly loaded textures in your rendering thread, and if you spot any new ones uploaded, do something in the rendering thread instead.

Cas :slight_smile:

I have a class for loading resources exactly as you described, I’ll post it tonight when I get home from work. I have a splash screen that displays the progress of the loading while the images/sounds/models are being read from disk.

Watch this space!

Andy.

Here is the code as promised, if you use it “as is” please give credit where it’s due.

Andy


/**
 * $Id$
 * 
 * (c) 2004 Swizel Studios.
 */
package com.swizel.utils;

import com.swizel.exceptions.ResourceNotFoundException;
import com.swizel.exceptions.ResourceNotLoadedException;

import java.awt.Image;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.util.Enumeration;
import java.util.Hashtable;
import java.util.Properties;
import java.util.Arrays;
import java.util.Comparator;
import javax.swing.ImageIcon;

/**
 * Resource loader and managment class.
 * 
 * @author $author$
 * @version $revision$
 */
public class ResourceUtils  {

  private static final float ONE_HUNDRED_PERCENT = 100.0f;
  
  private static Hashtable<String, Object> resources = new Hashtable<String, Object>();
  private static int resourceCount = 0;
  private static int resourcesLoaded = 0;

  /**
   * Load the list of specified resources ready for use.
   * @param resourceFile The file that lists the resources we want to load.
   * @throws ResourceNotLoadedException If there is a problem loading one of the resources.
   * @throws FileNotFoundException If the specified resource file can't be found.
   * @throws IOException If the specified resource file can't be loaded.
   */
  public static void loadResources(String resourceFile) throws FileNotFoundException, IOException, ResourceNotLoadedException {
    Properties resourceProperties = new Properties();
    resourceProperties.load(loadInputStream(resourceFile));
    
    resourceCount = resourceProperties.size();
    resourcesLoaded = 0;

    Enumeration names = resourceProperties.propertyNames();
    String[] propNames = new String[resourceProperties.size()];

    int i=0;
    while (names.hasMoreElements()) {
      String name = (String) names.nextElement();
      propNames[i++] = name;
    }

    Arrays.sort(propNames, new ResourceComparator());
    
    for (int j=0; j< propNames.length; j++) {
      String propertyName = propNames[j].toLowerCase();
      String propertyValue = resourceProperties.getProperty(propertyName);

      Thread.yield();
//      System.out.println("PropertyName  : " + propertyName);
//      System.out.println("PropertyValue : " + propertyValue);
      
      if ((propertyValue != null) && (propertyValue.trim().length() > 0)) {
        if (propertyName.indexOf("image") != -1) {
          Image img = loadImage(propertyValue);
          if (img != null) {
            resources.put(propertyName, img);
          } else {
            throw new ResourceNotLoadedException(propertyName);
          }
        } else {
          resources.put(propertyName, loadInputStream(propertyValue));         
        }
      } else {
        throw new ResourceNotLoadedException(propertyName + " has no value.");
      }
      resourcesLoaded ++;
    }    
  }

  /**
   * Calculate how many resources have been loaded.
   * @return The percentage of resources that have been loaded.
   */
  public static int getloadResourcesProgress() {
    return Math.round((ONE_HUNDRED_PERCENT / Math.max(1, resourceCount)) * resourcesLoaded);    
  }

  /**
   * Retrive a resource from the cache of resources.
   * @param resourceName The name of the resource as specified inside the resource properties file.
   * @throws ResourceNotFoundException If the specified resource can't be found.
   * @throws ResourceNotLoadedException If the specified resource can't be found, but loading of resources is still taking place.
   * @return The specified resource.
   */
  public static Object getResource(String resourceName) throws ResourceNotFoundException, ResourceNotLoadedException {
    String resourceNm = new String(resourceName.toLowerCase());
    if (resources.containsKey(resourceNm)) {
      return resources.get(resourceNm);
    } else {
      if (getloadResourcesProgress() != ONE_HUNDRED_PERCENT) {
        throw new ResourceNotLoadedException(resourceNm);                
      } else {
        throw new ResourceNotFoundException(resourceNm);        
      }
    }
  }

  /**
   * Remove a resource from the cache of resources, if the resource can't be found no further action takes place.
   * @param resourceName The name of the resource to be deleted.
   */
  public static void deleteResource(String resourceName) {
    String resourceNm = new String(resourceName.toLowerCase());
    if (resources.containsKey(resourceNm)) {
      resources.remove(resourceNm);
    }
  }

  /**
   * Load an image, this method can load resources from jar files
   * as well as from the local file system.
   * @param pPath The path to the images being loaded.
   * @return The loaded image.
   */
  public static Image loadImage(String pPath) {
    ClassLoader cl = ResourceUtils.class.getClassLoader();

    ImageIcon lvIcon = null;
    URL lvURL = cl.getResource(pPath);
    if (lvURL == null) {
      lvURL = ClassLoader.getSystemResource(pPath);
    }

    if (lvURL == null) {
      lvIcon = new ImageIcon(pPath);
    } else {
      lvIcon = new ImageIcon(lvURL);
    }
    
    if (lvIcon == null) {
      System.out.println("File not found : " + pPath);
      return null;
    } else {
      return lvIcon.getImage();
    }
  }

  /**
   * Load a binary stream, this method can load resources from jar files
   * as well as from the local file system.
   * @param pPath The path to the resource being loaded.
   * @return The InputStream of the resource.
   */
  public static InputStream loadInputStream(String pPath) {
    ClassLoader cl = ResourceUtils.class.getClassLoader();

    try {
      InputStream is = cl.getResourceAsStream(pPath);
      if (is == null) {
        is = ClassLoader.getSystemResourceAsStream(pPath);
      }
        
      if (is == null) {
        is = new FileInputStream(pPath);
      }    
      
      return is;
    } catch (FileNotFoundException fnfe) {
      System.out.println("File not found : " + pPath);
      return null;
    }
  }
  
} // class

/**
 * This class is used to place in sequence the resources 
 * loaded from a properties file.
 * 
 * Fonts are loaded first, followed by the Splash screen, 
 * finally all other resources are loaded alphabetically.
 *
 */
class ResourceComparator implements Comparator {

  /**
   * Compare two objects and order them according to the rules above.
   * @param o1 The first object being compared.
   * @param o2 The second object being compared.
   * @return -1, 0 or 1 depending on tthe order of these two objects.
   */
  public int compare(Object o1, Object o2) {
    String s1 = (String) o1;
    String s2 = (String) o2;
    if (s1.indexOf("font") != -1) {
      return -1;
    } else {
      if (s2.indexOf("font") != -1) {
        return 1;
      } else {
        if (s1.indexOf("splash") != -1) {
          return -1;
        } else {
          if (s2.indexOf("splash") != -1) {
            return 1;
          } else {
            return s1.compareTo(s2);
          }
        }
      }
    }
  }

  /**
   * Test to see if a given object is equal to another object.
   * @param obj The object being compared to this object.
   * @return true if the objects are equal, false otherwise.
   */
  public boolean equals(Object obj) {
    return compare(this, obj) == 0;
  }
    
} // class

/**
 * Changelog
 * ---------
 * $log$
 * 
 */


Then you can have a splash screen class that implements Runnable that looks like this


/**
 * $Id$
 * 
 * (c) 2004 Swizel Studios.
 */
package com.swizel.manlymayhem;

import java.io.FileNotFoundException;
import java.io.IOException;

import com.swizel.RenderListener;
import com.swizel.LogicListener;
import com.swizel.exceptions.ResourceNotLoadedException;
import com.swizel.exceptions.ResourceNotFoundException;
import com.swizel.utils.FontUtils;
import com.swizel.utils.ResourceUtils;
import com.swizel.utils.TextureUtils;
import com.swizel.awt.GLComponent;

import static org.lwjgl.opengl.GL11.*;
import org.lwjgl.opengl.Display;
import org.lwjgl.util.vector.Vector2f;

/**
 * Splash screen, which will load resources while displaying a picture and progress bar.
 * 
 * @author $author$
 * @version $revision$
 */
public class Splash implements RenderListener, LogicListener, Runnable {

  private boolean finishedLoading = false;
  private boolean removeListener = false;
  private int percentagePixelWidth = Main.SCREEN_WIDTH / 100;
  
  /**
   * Default constructor.
   *
   */
  public Splash() {
  }
  
  /**
   * This method starts a new thread and waits for the splash screen images to be loaded.
   */
  public void init() {
    new Thread(this).start();

    boolean splashLoaded = false;
    while (!splashLoaded) {
      try {
        Thread.yield();
        ResourceUtils.getResource("images.splash.background");
        splashLoaded = true;
      } catch (ResourceNotFoundException rnfe) {
        // loading has finished, but splash wasn't found.
        splashLoaded = true;
      } catch (ResourceNotLoadedException rnle) {
        // do nothing here, perhaps timeout.
//        System.out.println("waiting for images.splash.background to load.");
      }
    }

    FontUtils.init();
  }

  /**
   * When the resources have been loaded the splash screen is no longer needed,
   * this method initialises the next part of the game.
   */
  public void destroy() {
    Main.addListener(new Camera());
    Main.addListener(new SkyDome());
    Main.addListener(new Terrain());
    Main.addListener(new City());
    Main.addListener(new Sun());
    Main.addListener(new HUD());
  }

  /**
   * Handle the logic for the splash screen, i.e. flagging when loading of resources is finished.
   */
  public void logic() {
    if (removeListener) {
      Main.removeListener(this);
    }
    if (finishedLoading) {
      removeListener = true;
    }
  }

  /**
   * Return the number of triangles rendered in the previous frame by this component.
   * @return The number of triangles.
   */
  public int getTriangleCount() {
    return 0;
  }

  /**
   * Render the splash screen and progress bar.
   */
  public void render() {
    int percentComplete = ResourceUtils.getloadResourcesProgress();
   
    Display.setTitle("Loading resources : " + percentComplete + "%");

    // Now display splash screen and progress bar while other resources are loaded.
    
    GLComponent.setProjectionMode();

    glEnable(GL_TEXTURE_2D);
    glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_REPLACE);

    glBindTexture(GL_TEXTURE_2D, TextureUtils.getTexture("images.splash.background"));
    
    glBegin(GL_QUADS);        

      glTexCoord2f(0, 0); glVertex2i(0, 0);
      glTexCoord2f(1, 0); glVertex2i(800, 0);
      glTexCoord2f(1, 1); glVertex2i(800, 600); 
      glTexCoord2f(0, 1); glVertex2i(0, 600);
 
    glEnd();
    
    glDisable(GL_TEXTURE_2D);
   
    // The progress bar background.
    glPushMatrix();
    glColor3f(0f, 0f, 0f);
    glRecti(0, 19,  800, 29);

    // The progress bar.
    glColor3f(0.5f, 0.5f, 0.5f);
    glRecti(0, 20,  percentComplete * percentagePixelWidth, 28);
    
    // The progress bar highlight.
    glColor3f(0.75f, 0.75f, 0.75f);
    glRecti(0, 26,  percentComplete * percentagePixelWidth, 27);
    glPopMatrix();


    GLComponent.restoreProjectionMode();
    
    Thread.yield();
  }

  /**
   * Seperate thread for loading resources using the resource manager.
   */
  public void run() {
    try {
      ResourceUtils.loadResources("data/resources.dat");
      
    } catch (ResourceNotLoadedException rnle) {
      System.out.println("1" + rnle);
    } catch (FileNotFoundException fnfe) {
      System.out.println("2" + fnfe);
    } catch (IOException ioe) {      
      System.out.println("3" + ioe);
    } finally {
      finishedLoading = true;
    }    
  }

} // class

/**
 * Changelog
 * ---------
 * $log$
 * 
 */

This all works off a resource file that looks like this


# Load fonts first.
images.fonts.text=data/fonts/font.png
images.fonts.numbers=data/fonts/numbers.png
#images.fonts.test=data/images/test.png

#Splash images, always make sure these are first in the list.
images.splash.background=data/images/splash.png

#Images for the skydome.
images.skydome.clouds=data/images/landscape/clouds.png

#Images for the terrain.
images.city.terrain1=data/images/landscape/terrain1.png
images.city.terrain2=data/images/landscape/terrain2.png
images.city.terrain3=data/images/landscape/terrain3.png

What you’ll see is that I load fonts first followed by the splash screen background, when that is loaded anything else the order doesn’t matter since I can now show a nice screen and progress bar.

Sorry for the long post, hope this helps.

Andy

ooh that is long :stuck_out_tongue: I’ll have to check it out later today.

If you read the 3 posts I made in reverse order it might make more sense.

  1. The properties files: Contains the mapping of the logical resource names to the actual resource names as stored in your jar file.

  2. The splash screen: Passes the properties file name to the resource loader which runs in a separate thread. Meanwhile the main render thread of the splash screen probes the resource loader asking it what percentage of files have been loaded. It then displays a progress bar to reflect this.

The splash screen does nothing until the background image for the splash screen has been loaded, and any fonts, so you have a very brief delay before the splash appears but at least it gives you something nice to look at for long running jobs. You could get the splash screen up immediately if you have a dynamically generated/rendered background instead.

  1. The resource loader: Sorts the properties file so that fonts and the splash screen are loaded first, then everything else is loaded alphabetically, this could of course be extended to load level 1 data before level 2 data etc.

Methods exist to get the resource from the resource cache, exceptions are thrown if the resource isn’t found or is the resources are still being loaded (since the one being asked for may be further down the list). These methods are called by my texture manager later on to bind the loaded texture to an OpenGL texture id.

Thats about it, simple really.

Andy.