Graphics Backend Abstraction

@theagentd Now how would you combine OpenGL (which from Java uses raw values) and Vulkan (which from Java uses structs) together into one API? Would the API manage the memory or is the user responsible for that?

I just hide the structures.

Example:

  1. I create a texture, get an abstract texture object back, which either wraps a OGL int target and int textureID, or a Vulkan long pointer.
  2. I want to add it to a descriptor set, so I pass it into a descriptor set. The OGL descriptor set casts it into the OGL implementation and gets the target+textureID, while the Vulkan casts to the VK implementation and gets the pointer.
  3. OGL backend emulates descriptor sets, adding the texture info to some temp memory which can later be bound. VK sets up a stack allocated structure and updates the descriptor set.

In the case of batchable operations (descriptor set updates, command buffer submissions, swapchain swaps, etc etc etc), the interface also exposes a batch system where you get a batch object, something like this:


CommandSubmissionBatch batch = graphics.getCommandSubmissionBatch();
for(int i = 0; i < 32; i++){
    batch.add(commandBuffers[i]);
}
batch.submit();

In this case, the OGL version will just loop over the command buffers and execute them, while the VK one will actually only submit one draw call for all of them.

Some questions:

  • Who will use the API?
  • How likely is it to have anyone add a backend?
  • Why would this be desireable or even useful?
  • Do you plan to make a living from providing an API?
  • Why is Vulkan only not good enough?
  • Do you need it for your game?
  • Aren’t you just procrastinating to hide your creativity block from yourself?

The last one killed all my game dev ambitions years ago by keeping myself occupied with tech details until my money ran out, so don’t fall for that. (You might need to work for insurance companies otherwise)

Initially, only me and SkyAphid. We will be basing NNGINE, our 3D engine on it, so WSW and all our future games will use this abstraction. If the engine becomes powerful enough to attract attention, we may attempt to market and license the engine. At them moment, we have no concrete plan of commercializing it. This is mainly for our own purposes.

The abstraction is modular. The idea is that you can add multiple backends for different features. The purpose of the abstraction is essentially to make a more powerful and efficient abstraction than LibGDX that we can use to support multiple platforms and APIs, so I do not have any expectations that others will implement their own backends. Regardless, I want it to be possible to support new APIs easily.

We have specific games that are a good fit for Android and HTML5 planned. The ability to switch between APIs is a great feature IMO as it means the games made on top of the abstraction are completely API independent, meaning that porting to a different API is a no-op.

We do not plan on it, but keep it open as a potential idea. I do not think it will make me rich.

Backwards compatibility, especially on Android. Also, seamlessly supporting WebGL is nice (although the user will need to be aware of the limitations of GWT).

For our games currently in development, yes-ish. Vulkan support for We Shall Wake would help immensely as the game’s CPU scalability is currently only limited by single-threaded API calls. Even just the OpenGL backend I’m developing should give a significant boost to WSW’s performance thanks to software command buffers and a dedicated rendering thread, let alone the Vulkan backend. In addition, we have a concrete game with an almost complete design document which would only be viable with Android and GWT support. This abstraction is essentially the convergence of a number of requirements.

No, I prioritize work on RF and paid work, developing it in my spare time. I appreciate your concern and I can relate to the money troubles. I am currently living very cheaply and getting by, and I know what awaits me if I end up not being able to support myself as I’ve worked at a bank writing Java web applications.

I guess it’s time for me to speed up my work on this. I’ve been stuck on some stupid technical details that don’t have a clean solution for a little bit too long now. =P

I don’t like telling people about WIP in that sense unless it’s something concrete that others can appreciate. I don’t want praise for plans with no real substance. Hence, I’ll be back about this once I have something concrete to show.

Porting being a no-op is, I think, an illusion. Different platforms require different gameplay and different performance trade offs. Whether or not your graphics-engine can seamlessly transition from HTML5/WebGL to multiple dedicated GPUs is a mere side note. The effort that is put into building an VK-like API on top of OpenGL is impressive, but by the time you’re ready to publish your game, VK has probably gone mainstream, and WebGL would support it by then.

It all reminds me of some of the efforts Cas and I took upon us, to scale the sprite-engine from the low-budget to high-end GPUs, and seeking fast-paths through outdated drivers and/or hardware. It just got in the way of publishing the games, and that framerate bump wasn’t ultimately worth it.

If this technical tinkering takes you 18 months, in that time high-end GPU performance has doubled, but more interestingly: your low-end supported hardware doubled in performance too. To make all that raw power, and super efficient API/engine worth the effort, you need a small army of artists and designers. A one or two man team most likely cannot come close to what the big (and free) engines support, so why not pick an art-style that hides the technical inefficiencies? It’s hard enough to turn a tech demo into a polished, playable, purchable game. Very little of game programming is performance sensitive, so why focus all your energy on it?

My $0.02 :-*

I’ve abstracted the graphics backend in my engine by using a system where entities have a “view” id (generated at runtime) and the graphics backend relates a renderer to that “view”.


public interface Renderer<T> {
	public static final int NONE = -1;
	default public Renderer<T> create(T entity) {
		return this;
	}
	public void begin(T entity, GameState state, View view); // <- BEGIN RENDERING
	default public void end(T entity, GameState state, View view) {
		// nothing
	}
	default public void update(T entity) {
		// nothing
	}
	default public void destroy(T entity) {
		// nothing
	}
}


// code found in "entity" type object
	public void setRenderer(int id)
	{
		destroyRenderer();
		
		renderer = Axe.renderers.get( id );
		
		if (renderer != null)
		{
			renderer = renderer.create( this );
		}
	}
	
	protected void destroyRenderer()
	{
		if (renderer != null)
		{
			renderer.destroy( this );
			renderer = null;
		}
	}
	
	public void destroy()
	{
		destroyRenderer();
	}

	public void update(GameState state, View<V> firstView)
	{
		if (renderer != null)
		{
			renderer.update( this );	
		}
	}
	
	public void draw(GameState state, View<V> view)
	{
		if (renderer != null)
		{
			renderer.begin( this, state, view );
			
			// render children
			
			renderer.end( this, state, view );
		}
	}

Where Axe.renderers is an instance of:


public class Registry<T>
{
	private int identifier;
	private T[] items;

	public Registry( T... items )
	{
		this.items = items;
	}

	public int create()
	{
		return identifier++;
	}

	public void register( int identifier, T item )
	{
		if (identifier >= items.length)
		{
			items = Arrays.copyOf( items, identifier + 1 );
		}
		
		items[identifier] = item;
	}

	public <X extends T> X get( int identifier )
	{
		return (identifier >= items.length || identifier < 0 ? null : (X)items[identifier]);
	}

}

When you have a displayable object you define an ID


public class Sprite
{
    public static int VIEW = Axe.renderers.create();
}

And when you load a graphics library they do this:


Axe.renderers.register( Sprite.VIEW, new SpriteRenderer() );

This will vary if you aren’t using OOP and are using DOP

The renderer class has a create method which can return a new renderer (per entity instance) or they can all shared a renderer (globally or based on the entity data). You can create different views for each type (ie Sprite) that can render it differently.

The restriction here is that all possible drawable things need to be an object and have a view.

This has been working fine for me!

Of course, that different platforms require different trade offs is a given. A game designed for a keyboard will not work in app form, and a graphics engine designed for 500GB/sec bandwidth GPUs will not run well on a mobile GPU. However, there most definitely are overlaps in the technical department. For example, allowing the user of an Android tablet to plug in a controller and play the game just like on PC is definitely a plus, and a shitty Intel GPU has more in common with a mobile GPU than a dedicated 400$ GPU from AMD or Nvidia. In other words, alternate ways of controlling the game will be a nice feature to have on mobile/tablets, while a version of the engine tweaked for mobile will work very well as a “toaster mode” setting for weak desktop GPUs too, something we’ve gotten requests for for WSW.

Note: By “engine”, I mean the engine built on top of the abstraction, not the abstraction itself. In the example above, I was specifically referring to using deferred rendering for desktop to take full advantage of the huge amounts of processing power and bandwidth available on desktop to achieve great lighting, while falling back to forward rendering on mobile GPUs where bandwidth is scarce and a 24 byte/pixel G-buffer isn’t practical. Again, this is pretty much exactly the same problem that Intel GPUs have as they use system RAM, so using forward rendering on Intel GPUs could easily double or triple performance for us (but only at the absolute lowest settings with minimal lighting of course).

I appreciate your advice regarding technical stuff, but I think that this will be worth it. SkyAphid focuses 100% on developing games using the stuff I write, and I spend most of my time on helping out with that. It’s worked out so far. If anything, a solid backend will allow us to work faster when creating the actual game as there’s much less to worry about when it comes to performance.

It looks like @theagentd is developing a “middleware” for game engine developers.
In my opinion, it is probably of little use to actual “game developers”, who require a much higher level of abstraction from a game engine, to be able to say: “Here is my model file, please render that with a nice sky, a sun and beautiful reflecting water.” and possibly just modeling and scripting everything in a nice editor, like the CryEngine Sandbox editor. Those people hardly want to get down to such low-level details as “command buffers” and “swap chains”.
But for game engine developers, this could be a thing.

madalin stunt cars 2 * five nights at freddy’s * instagram video downloader * happy wheels

It matters not; I have once again enslaved @theagentd and he is in my thrall. This time he may not be released.

Cas :slight_smile:

demanding release!

<— You can’t contain this! =P

Minion! Back to work!

Cas :slight_smile: