Embedding bits of OpenGL in JavaFX scene graph

OpenGL Development
#1

What’s the prevailing wisdom on how to draw bits of the scenegraph using OpenGL?

I’m not looking for anything particularly fancy: I’m quite happy without event/mouse integration or any of that stuff; I just want to draw rectangular areas of stuff with OpenGL rather than JavaFX primitives.

What I’m currently doing is (using LWJGL2.9.3) is using a Pbuffer, drawing whatever to it every frame using a JavaFX AnimationTimer, and then calling glReadPixels to read the rendered stuff back into a ByteBuffer.

I then feed the ByteBuffer data into a PixelWriter from an Image which is inside an ImageView node:


	glReadPixels(0, 0, 800, 600, GL_BGRA, GL_UNSIGNED_BYTE, pixels);
	PixelWriter pw = writableImage.getPixelWriter();
	pw.setPixels(0, 0, 800, 600, javafx.scene.image.PixelFormat.getByteBgraInstance(), raw, 800 * 600 * 4 - 800 * 4, -800 * 4);

I’ve used some Unsafe hackery so that the “raw” array referred to there is actually being written directly into by glReadPixels, which I have to do because glReadPixels requires a direct ByteBuffer whereas setPixels only takes either a byte[] array or a heap ByteBuffer (irritating but not too hard to work around in the end). So that at least saves a fairly big copy.

Is there a better (more efficient) way than this?

Cas :slight_smile:

#2

i’d go with the shared array (pointer to native array <-> direct bytebuffer) too.

curious too if there is a “non hacky” way to do it without a large copy.

#3

[quote=“princec,post:1,topic:55096”]
See LWJGL-FX. It is a bit complicated because:

a) it does both OpenGL-in-JavaFX and JavaFX-in-OpenGL
b) it does asynchronous readbacks/uploads with double/triple-buffering for performance
c) it has multiple readback/upload implementations for optimal performance on different GPUs (AMD_pinned_memory on AMD, INTEL_map_texture on Intel for linear texture layouts, etc)

You could clean it up and keep only the stuff you need.

LWJGL-FX uses the direct ByteBuffer in setPixels and it works fine. Why wouldn’t it work? The WritableImage is indeed backed by a heap ByteBuffer and the copy goes through java.nio.Bits.copyToArray (it’s an unsafe.copyMemory loop).

[quote=“princec,post:1,topic:55096”]
What happens now is:

  • GPU to CPU readback (1st copy)
  • Off-heap ByteBuffer to on-heap WritableImage (2nd copy)
  • WritableImage to GPU upload (3rd copy, by JavaFX)

It should be possible to get rid of the 2nd copy, with unsafe/reflection hackery. Either replace the on-heap buffer with the off-heap one, or make the readback write directly to it (risky due to GC). That could be a decent win if the 3D scene is large enough, though the readback is probably the slower of the 3 copies.

#4

This seems to work (unoptimized):

ByteBuffer direct = ...
WritableImage writableImg = ...

// Get the platform image
Method getWritablePlatformImage = javafx.scene.image.Image.class.getDeclaredMethod("getWritablePlatformImage");
getWritablePlatformImage.setAccessible(true);
com.sun.prism.Image prismImg = (com.sun.prism.Image)getWritablePlatformImage.invoke(writableImg);

// Replace the buffer
Field pixelBuffer = com.sun.prism.Image.class.getDeclaredField("pixelBuffer");
pixelBuffer.setAccessible(true);
pixelBuffer.set(prismImg, direct);

// Invalidate the platform image
Field serial = com.sun.prism.Image.class.getDeclaredField("serial");
serial.setAccessible(true);
Array.setInt(serial.get(prismImg), 0, Array.getInt(serial.get(prismImg), 0) + 1);

// Invalidate the WritableImage
Method pixelsDirty = javafx.scene.image.Image.class.getDeclaredMethod("pixelsDirty");
pixelsDirty.setAccessible(true);
pixelsDirty.invoke(writableImg);

The benefit is quite small, 5-7% faster in my test.

#5

Thanks Spasi, most useful. I’ll try some experiments on Monday back at work.

I must say I’m surprised at how slow some aspects of JavaFX are that I’ve had to resort to trying OpenGL to get the performance I need. It’s also boggling but predictable that they’ve not made an OpenGL backend available in the Windows distribution, nor any way to allow for hooks into the GL or DX context to make this sort of extension easy.

I notice there’s a WGL extension to share resources between DX and GL which would have been ideal if JavaFX had provided some easier way in at that point.

Cas :slight_smile:

#6

[quote=“princec,post:5,topic:55096”]
I’ve been looking at CEF as an alternative to JavaFX for rich interfaces. It has a nice C API and supports offscreen rendering, but looks like it has the same issue; no support for zero-copy or at least a GPU-to-GPU copy.

#7

Using the little bit of code I had and that snippet I’m getting very good performance. Or at least, far better than JavaFX on its own. NV_path_rendering is the future - I really wish it were generally available in OpenGL because it would fundamentally change everything, ever.

Cas :slight_smile: