LWJGL3FX - A sneak peek

Engines, Libraries and Tools
#1

This is an effort by @KaiHH and me to integrate LWJGL 3 with JavaFX.

Some of you may remember LWJGL-FX, which was a proof of concept to simply show how nice it would be to render OpenGL graphics within JavaFX. Technically it wasn’t doing anything worthwhile, it was just trying to use the existing public JavaFX APIs in an efficient manner. Despite my efforts, the end-result was very inefficient in terms of both memory and CPU usage.

Some time before LWJGL 3.0.0 was released, I had written a Prism backend (the part of JavaFX that does the actual rendering) using LWJGL 3. It worked, but other stuff got in the way and I never had time to do anything with it. A few days ago I recruited KaiHH to help me do it right this time. The old code was ported to the latest LWJGL releases and we figured out a good approach for the integration. This is the first result.

The images below are screenshots of this JavaFX sample, modified so that the classic Gears OpenGL demo is rendered behind the main content. The drop shadow effect is enabled on the JavaFX image. The window is maximized in a 1920x1200 monitor. Minus the menu and taskbar, the OpenGL view is slightly larger than 1080p. The background color animates, that’s why it isn’t the same in all screenshots. Open the images in a new-tab for the full size.

V-sync enabled, 60 fps, 3.9% CPU

1920×1200 211 KB

The following were launched with -Djavafx.animation.fullspeed=true. Despite the name, there’s a hard cap at 1000Hz for the animation pulses. This means that there’s a fixed overhead of 1ms and we cannot go lower than that (i.e. cannot go higher than 1000fps).

V-sync disabled, 1.0 + 0.41ms per frame

1920×1200 139 KB

V-sync disabled, HUD disabled, 1.0 + 0.01ms per frame

1920×1200 103 KB

#2

Hey guys… what is the actual status of this then?

Cas :slight_smile:

#3

We have decided to prioritize the Android/ARM builds over this. Other than limited time, this project has several difficulties:

  • Needs quite a bit of work to be production ready.
  • We’re not sure if it’s worth doing lots of unpaid work for something that Oracle ought to support out of the box and if there will actually be people that want to use an unofficial backend.
  • We’re not sure about licensing issues and if Oracle is going to come after us. Which is why the code isn’t public yet.

Anyway, current status:

It’s still at a proof-of-concept stage but works very well. As you could probably tell by the measurements above, it’s a pure GPU solution with no CPU copies. In fact, there are not even GPU copies, other than the compositing JavaFX does anyway. We render directly to the texture backing the GL viewport, without disrupting JavaFX’s context state. Issues and pending work:

  • It currently works on Windows and Linux. We must add support for CGL and EGL + real OpenGL ES. The plan is to add a cross-platform API for GL context management to lwjgl3. What ContextAttribs did in lwjgl2, but probably closer to GLFW’s hint-based API and implementation.
  • We’ll need some kind of API that configures the FBO associated with the GL viewport.
  • Overriding the default Prism implementation requires the backend code to be loaded via -Djava.ext.dirs. Won’t be friendly to novice users.
  • Prism is not documented and does not have a stable API. This means that new JDK versions will likely break our implementation. We’ll need multiple versions and have maintenance overhead. The current implementation is compatible with JavaFX 8 only. Java 9 has also dropped -Djava.ext.dirs, we’ll probably need module workarounds to run there.
  • Long-term: Prism looks like it’s been hacked until it worked. It’s really quite a bad code base to work in. We’d really like to do a cleanup where possible. Being able to do everything in Java will help a lot.
  • Longer-term: It’d be fun to add a Vulkan backend for improved performance.
#4

@cas : did you go with jawt plus swing/javafx yet ?

#5

The one time I’ve used it was LWJGL3+JavaFX using the dreaded buffer blitting. Performance was barely acceptable for the thing I needed :frowning:

Has anyone heard/seen of the Marlin renderer?

Cas :slight_smile:

#6

Yeah I’ve heard of the Marlin renderer.

#7

Wonder if it’s any easier to integrate than Pisces?

Cas :slight_smile:

#8

There’s now an alternative implementation available, JFXGL. It was created by Jeff Martin. Compared to ours:

Pros

  • It uses GLFW for context management and as the windowing system.
  • You have full control of the render loop (the fixed 1ms per frame overhead mentioned above does not apply).
  • Doesn’t have the -Djava.ext.dirs issue.
  • You can use it right now.

Cons

  • It handles JavaFX graphics only, other features are not supported (pickers, popups, etc.).

We’ve shared our code with Jeff and his code is obviously also available, so both approaches will likely improve soon.

#9

Hey @Spasi, did you guys get any further with your implementation? I have some folks using Ardor3D with SWT and Eclipse RCP that are getting tired of SWT and are looking at potentially switching to JavaFX.

#10

I would recommend JFXGL, for now at least. Jeff has made good progress.

#11

Oracle has announced that JavaFX and WebStart will not be available anymore, starting from Java 11 (September 2018, the next LTS release):

Java Client Roadmap Update

Also, an OpenJFX mirror is now available on github at javafxports/openjdk-jfx.

I would still very much prefer CEF+#1006, but I think this development will be good for JavaFX. It’s also a good opportunity. No more hacking our way into the built-in JavaFX. The biggest technical hurdle is removed and we don’t even have to worry about compatibility. You could bundle any JavaFX fork you like in your application, not only with a different backend, but also a different API (if necessary).

So, I’ll probably resume work on LWJGL3FX in about 2-3 months. It will be a public fork, under the LWJGL organization on Github. If @KaiHH or anyone else would like to contribute, I’d be very happy to do this as a team effort. The initial plan:

  • Refactor the GLES backend to use lwjgl3. It should work on Windows/Linux/macOS, at least.
  • Eliminate the awkward native code paths and use Java/lwjgl3 for everything.
  • Eliminate the hard-coded 1ms animation delay.
  • Support both:
    • Render OpenGL views inside the JavaFX node hierarchy.
    • Render a JavaFX node hierarchy to texture, use texture in a GLFW+GL/Vulkan application.

If interested, please post here or contact me directly.

#12

Personally, I would rather prefer improving AWT/Swing support for LWJGL3 instead.
Given that Oracle is planning to continue investment in that area in Java 11, and given that, unlike LWJGL2, LWJGL3 still has no fully working AWT/Swing backend which many developers would probably like to use. So compared to LWJGL2, LWJGL3 is actually lacking behind in that regards.
I have a fully-working Windows-only implementation ready in the LWJGLX/lwjgl3-awt Github repo, but Linux and MacOS support is still pending.

#13

Hey Spasi - I’ve always been super-interested in having JavaFX completely backed by LWJGL as it’d basically make creating complex game UIs a total doddle. As usual I’m rather pressed for time but willing to help wherever I can.

Cas :slight_smile:

#14

[quote=“KaiHH,post:12,topic:58143”]
I suppose that some of the work could be shared. A cross-platform context management API/impl would be equally useful to both JavaFX and AWT/Swing. After that’s done, implementing an AWT back-end similar to lwjgl2’s should be simple. So, I don’t mind doing that too, it’s not a lot of work compared to what needs to be done for JavaFX.

The big difference is that AWT is set in stone, we’re limited by what the JDK supports/allows. A JavaFX port opens up opportunities that will be impossible with AWT.

#15

There is still no equivalent of Java AWT Native Interface for OpenJFX / JavaFX yet as far as I know.

#16

Newbie question: is there any difference between a full integration of FX with GLFW, and just having a FX window and an GLFW window in the same process? With the latter I didn’t encount any problems on on linux and windows. I heard on Mac even this “co-existing” windows don’t work…

#17

[quote=“floyd_1982,post:16,topic:58143”]
Exactly, both JavaFX and GLFW have their own event loops and they don’t know anything about each other. It’s a conflict that cannot be resolved on macOS.

The plan is to offer two modes:

A. JavaFX + Glass are used as the windowing system (frames/input/etc). You’re now able to do OpenGL rendering directly on a JavaFX node/canvas, without a performance hit. GLFW is not used in this case.
B. GLFW is used as the windowing system. You use the JavaFX rendering subsystem (Prism) to render a UI and/or 2D graphics to an FBO/texture. You use that texture however you like in your GLFW application. Glass is not used in this case.

The screenshots in the original post were made with a mode A solution.

#18

Do I understand this correctly that mode A. and B. are the solutions for integrating FX and OpenGL in one window like in your first post?

Let’s say I didn’t need that and I’d like to have a GLFW window AND a JavaFX stage with their own event loops. Do these sabotage each other so that things break? How do they even interact when GLFW runs on the main thread and JavaFX on the FX thread for instance?

Thanks for your explainations!

#19

Again, you cannot mix the two. On macOS, there’s a single event loop that runs on the main thread (also has to be the first thread in the process).

First option is to have two windows created and managed by JavaFX, the first is covered by a canvas with OpenGL rendering on it, the second has a standard JavaFX stage. Second option is to have two windows created and managed by GLFW, both are typical GLFW/OpenGL windows, but the second uses JavaFX for rendering the stage. The first option is simpler to implement, the second would require translating GLFW events to JavaFX events to make the UI interactive, etc.

#20

Well, interesting to see what that client roadmap means there too! There’s no mention at the moment of what happens with AWT after Java 11 (even if it will be in JDK 12), or who the “interested parties” involved in conversation around stewardship are. That’s causing some consternation in a few places, but might also open up some opportunities …