Design or pattern for handling background consumer/producer tasks

OpenGL Development
#1

Background:

I have a 3D engine implemented using LWJGL aimed largely at rendering a large ‘world’ - terrain, vegetation, animated creatures, weather, day/night cycle, etc. The engine manages the resources that are required for rendering such as the various textures, meshes, models, etc.

As the player navigates this world the engine caches the various types of data discarding/releasing resources that are no longer required and loading/caching newly required data on demand. This is handled by a set of background threads that perform I/O tasks (loading images, loading a new terrain ‘chunk’, building a model mesh, etc) and a queue of tasks to be executed on the OpenGL context (allocating a new texture, uploading data to a VBO, deleting a texture, etc).

Current design:

This all works nicely with the exception of tasks that have dependencies. I’ll illustrate with an example:

Let’s say the engine needs to apply a new texture to an object in the scene, this consists of the following steps:

  • load the texture image from the file system (I/O background task)
  • allocate a texture on the GPU (OpenGL context task)
  • upload the image to the texture (OpengL)
  • attach the texture to the material for the object (OpenGL, not really but avoids rendering issues)

There are a few problems to resolve here:

  1. Ideally I want these steps (or tasks) to be atomic and re-usable, for example the engine needs to load images for several other parts of the system - the code is exactly the same and therefore can (and should) be encapsulated and re-used.

  2. Some of this can be run in parallel (e.g. load image and allocate texture), some of it must be sequential, e.g. cannot upload the image until we have loaded it and allocated the texture.

  3. Some of the steps have ‘resource dependencies’ on previous steps, e.g. in particular the upload step requires the allocated texture ID and the image.

The first two turn out to be relatively straight-forward, it’s the last one is what I am struggling with - I cannot seem to come up with a decent design that allows re-usable and relatively atomic tasks to be linked together when there are inter-task dependencies.

Some pseudo-code:

interface Task extends Runnable {
    TaskQueue getQueue();
}

// Generic load-an-image task
class LoadImageTask implements Task {
    private final String path;
    private BufferedImage image;

    public LoadImageTask( String path ) {
        this.path = path;
    }

    TaskQueue getQueue() { return TaskQueue.BACKGROUND; }

    public void run() {
        // load the image from the given location
    }
}

// Uploads an image to a given texture
class UploadTextureTask implements Task {
    private BufferedImage image;
    private Texture texture;

    ...

    TaskQueue getQueue() { return TaskQueue.RENDER_THREAD; }

    public void run() {
        texture.buffer( image );
    }
}

// Example task manager for the scenario outlined above
class Example extends TaskManager {
    ...

    // Load the texture image
    final LoadImageTask loadImage = new LoadImageTask( ... );
    add( loadImage );

    // Allocate a texture
    final AllocateTextureTask allocate = ...
    add( allocate );

    // Upload texture image
    final UploadTextureTask upload = ...
    add( upload, loadImage, allocate );
}

The add method in TaskManager registers a task on the relevant queue in the background. The manager gets notified when each task is completed. When all current tasks are finished (loadImage and allocate in this case) the manager starts the next task(s) in the sequence.

Note that the final add call for the upload task tells the manager that it has dependencies on the load and allocate tasks.

The problem:

As things stand the resources (in this case the image and the texture objects) are copied from the dependant tasks by reflection when those tasks are completed. This just seems dirty but I cannot think of a better method.

I’ve tried defining consumer and producer interfaces with generic getter/setter methods and linking dependant tasks that way but this approach breaks down if a task consumes more than one resources (as the upload task does above).

I have researched this problem here and on other sites but haven’t come across any good designs - they either seem to just consist of cut-and-paste code or some sort of shared state (usually a map) with lots of nasty casting everywhere. Maybe I am not searching for the correct terms? Or maybe my whole approach is rubbish?

Has anyone implemented or come across anything similar? Any suggestions, criticisms, pointers are welcome.

Apologies for wall of text, also posted this here: http://stackoverflow.com/questions/23542776/design-or-pattern-for-handling-background-consumer-producer-tasks

  • stride
#2

Make Task a class (as opposed to an interface), and let it hold a set of dependencies and a flag whether it is done. Let the Task hold the Result it produced. The Tasks depending on the current task will need it.

When processing the pending Tasks queue, check whether all dependencies of the Task are done, and if so, remove it from the queue and execute it, flagging it done.

You will/may encounter a Task a few times, when iterating over de pending Tasks, until their dependencies are done.

Multiple threads have their own pending Task queue. Task done-flag will have to be volatile. Dependencies can refer to Tasks on other queues. It’s easy to schedule image loading Tasks on a background thread, and have a texture updating Task depending on them, being executed on the render-thread, pulling the image-data from its dependency Result value.

Don’t worry about ‘design’ or ‘efficiency’, code is there to get the job done, it’s not art.on’t worry about ‘design’ or ‘efficiency’, code is there to get the job done, it’s not art.

#3

Java 8 introduced the CompletableFuture class. With that you can do this kind of stuff very easily.


static Image loadImage(String path) {
}
static Texture allocateTexture() {
}
static void uploadTexture(Image image, Texture texture) {
}
...
CompletableFuture<Image> image = CompletableFuture.supplyAsync(() -> loadImage(path));
CompletableFuture<Texture> texture = CompletableFuture.supplyAsync(() -> allocateTexture());
image.thenAcceptBothAsync(texture, (Image im, Texture tx) -> uploadTexture(im, tx));

Of course you could also have your loadImage method return a CompletableFuture instead of an Image to shorten the rest of the code.
And by the way - using CompletableFutures in this way will never block a thread. So it will work fine even if you have a million tasks.

#4

That’s pretty much what I have now so hopefully I’m working on the right lines.

The main issue I was trying to highlight was the one of how to deal with tasks that produce and consume resources (such as the texture image example) without having to write cut-and-paste boiler-plate code for every use-case.

[quote=“DrZoidberg,post:3,topic:48667”]
Java 8 introduced the CompletableFuture class. With that you can do this kind of stuff very easily.

[quote]

This could be a very good call. I had a good look at the various concurrent features such as Future (the background queue I mentioned is a ThreadPoolExecutor) but there wasn’t anything there that I could see that addressed the specific problem of how to deal with tasks that produce/consume. I’ve yet to look into Java 8 features in any detail so perhaps now’s the time.

Cheers for the thoughts guys.

  • stride
#5

How about the following example:


Map<String, Task> img2task = ...;

List<Task> pendingImgTasks = ...;
List<Task> pendingTexTasks = ...;

public CreateTextureTask eventuallyCreateTexture(String imgPath) {
   Task imgTask = img2task.get(imgPath);
   if(imgTask==null) {
      // never loaded this image before, or discarded it
      imgTask = new LoadImageTask(imgPath);
      img2task.put(imgPath, imgTask);
      pendingImgTasks.add(imgTask);
   }

   CreateTextureTask texTask = new CreateTextureTask();
   texTask.addDependency(imgTask);

   pendingTexTasks.add(texTask);
   return texTask;
}



class LoadImageTask extends Task<BufferedImage> {
   ...

   public BufferedImage execute() {
      return ImageIO.read(...);
   }
}

class CreateTextureTask extends Task<Integer> {
   ...

   public Integer execute() {
      int handle = glGenTextures();
      ...
      return handle;
   }
}


class Task<T> implements Runnable {
   private final List<Task> dependencies = new ArrayList<>();

   public void addDependency(Task task) {
      dependencies.add(task);
   }

   public boolean canStart() {
      for(Task dependency: dependencies)
         if(!dependency.isDone())
            return false;
      return true;
   }



   public abstract T execute();



   private volatile T result;

   public void run() {
      if(!this.canStart()) throw new IllegalStateException();
      T result = this.execute();
      if(result==null) throw new NullPointerException();
      this.result = result;
   }

   public boolean isDone() {
      return result != null;
   }

   public T getResult() {
      T result = this.result;
      if(result == null) throw new IllegalStateException();
      return result;
   }
}

I’m not sure there is much boiletplate, as your callsites will be like:
[icode]CreateTextureTask t = eventuallyCreateTexture(“res.tree.png”);[/icode]

Also consider your most valuable resource to be time. A tiny bit of code duplication is acceptable if it saves you from hours/days of crafting some super-neat design that provides exactly the same functionality. High quality code is not the goal, after all - something us developers tend to forget. :slight_smile:

#6

Some good suggestions there, cheers.

Also sound advice.

#7

In addition to suggestions @riven made I have been getting to grips with the Java 8 (which is screwing with my head as I’m also starting to ramp up using Scala for my real-life work!)

These new goodies seem to address the problem I was facing.

I haven’t quite completed the re-factoring but it’s looking pretty good so far:

  • less boiler-plate code
  • the tasks are now entirely re-usable
  • inter-task dependencies are handled by out-of-the-box Java functionality rather than me having to build from scratch

Thanks for the all input :slight_smile:

#8

libgdx’s AssetManager is a system for running tasks that can have a part on the GL thread and a part on a separate thread. It was quite tricky to get right. This enables rendering a progress bar (or other anything else) while things are loading. It also has ref counting, so assets can auto unload when no longer needed.

Another experience I have similar to what you are asking about I solved with Riven and MatthiasM’s green thread library:


This allowed me to run code on the render thread, but yield so I can still render things. Without being able to yield in my tasks, I would have had to store an enormous amount of state so I could resume the work later.