Off screen rendering, drawing images onto themselves and other shenanigans

Java 2D JavaFX
#1

I’ve been enticed by the additional blend modes available in JavaFX to have a little dabble with the API.
However doing low level stuff seems incredibly clumsy?

Am I correct in thinking that drawing to an image in vram (equivalent of Java2D VolatileImage), and then drawing said image onto another image in vram, is now only possible through composition of canvas elements in the scene graph?

If this is the case, how would the current api facilitate cyclic drawing dependency? e.g. drawing a vram image onto itself? (Or, more efficiently, onto an intermediary surface, then onto itself)

A scene graph is all well and good, but shouldn’t the API also expose enough of the underlying rendering pipeline so that you can implement a comparable scene graph in user code if JavaFX’s offering is inadequate?

I came to JavaFX expecting a Swing 2.0, but without Canvas exposing its underlying vram surface, the API seems needlessly clunky and restrictive for anything more complex than basic image composition.

Am I missing something?

#2

What you’re missing is LWJGL.

Cas :slight_smile:

#3

Quite, but within the scope of the question? :-*

:edit:

Another limitation that’s fairly prominent:
if I want to procedurally generate an image (GraphicsContext onto a Canvas seems the only way?), then reuse said generated image multiple places in the scene graph, then I must snapshot it into a WriteableImage?

It makes sense that there’s no way of having multiple Canvases in the scene graph share an underlying surface, as that’d break the scene compositing process.

However snapshotting requires the creation of multiple extraneous objects, and significant memory copying; it’s slow.
Does the JavaFX API really have no performant way of accomplishing this very basic functionality?!?!

Java2D might have been a terribly API, but at least it was reasonably performant & logical once you got around its over-engineered abstraction, and knew what was actually happening under the hood.

It’s maddening that all that seems to be requires is:

WriteableImage.getGraphicsContext()

You’d then be able to leverage JavaFX’s more feature-rich graphics pipeline, without being burdened by the limitations of the scene graph.
I’ve not been so thoroughly confounded by such a fundamentally flawed API design since I did MIDP development a decade ago.

#4

Yes, JavaFX appears to have basically been designed in a bubble without any input from people who actually needed a modern UI API.

For starters, the creators of JavaFX have entirely ignored existing OpenGL APIs available - open source! - for Java, f.ex LWJGL and JOGL, and instead, pointlessly, totally rolled their own implementations.

Secondly they have inexplicably decided to entirely hide the underlying rendering APIs from the front-facing APIs. This means that nobody else can easily write a plug-in renderer that, for example, used LWJGL instead of the supplied renderers.

That would have gone a long way towards dealing with a whole lot of issues with JavaFX. Specifically there is no efficient, straightforward way to render JavaFX directly into an OpenGL context, so that means integrating JavaFX UIs with existing games engines is hopelessly inefficient, requiring a raft of hacks to get it to work (some one has managed it but it ain’t pretty). And that’s just the graphics side of things, never mind user input.

Thirdly they’ve completely hidden most of the skinning / L&F APIs, and where they’ve not hidden it they’ve not documented any of it. There are moves to improve this but don’t hold your breath. The upshot is that it is very hard to make your own custom controls completely from scratch, and it’s very hard to properly customise the L&F of existing controls if not impossible.

All of this is annoying because it could have been avoided.

Cas :slight_smile:

#5

Call me cynical, but I never even bothered with JavaFX after I saw their first demo: a 2D ‘clockwork’ comprised of half a dozen (!) rotating opaque arcs. It was stuttering for over 10sec before it became somewhat smooth at 15fps. This was their tech to showoff JavaFX to the world… with the promise that it would become less stuttery, blaming JIT compilers and cold caches. Meanwhile LWJGL was spitting out tens of thousands sprites per frame at 60fps. I lost faith in the competence of the developers right there.

After… applets, poor graphics APIs, poor audio APIs, poor media APIs, why even consider looking at Suns/Oracles clientside tech, where even if they’d succeed, by making it part of the JRE they tied themselves to the backward-compatibility ball-and-chain from day one. We’d have to live forever with every single design-flaw: a revamp would be out of the question, or it would have a clunky API.

Rant. Rant. Rant.

Any project, clientside, serverside or middleware, will have a few flawed iterations. Why ship it in an eco-system that does not allow you to shed your mistakes?

#6

Reckon I could do a better job of it now :slight_smile: Only taken me 20 years to figure out how, mind…

Cas :slight_smile:

#7

[quote]It’s maddening that all that seems to be requires is:

WriteableImage.getGraphicsContext()
[/quote]
Is this something that can be requested from these folks?
http://mail.openjdk.java.net/mailman/listinfo/openjfx-dev

Where does one go to request something like this?

#8

You’d go there, yes, but they are strangely reluctant to crack open the JavaFX APIs in a manner that you might call “useful” to anyone doing this sort of thing. I think JavaFX’s design is probably too flawed since conception - it is what it is. Maybe it’d be better to design something completely different and pinch the good bits from JavaFX. In the meantime it’s a reasonably good replacement for Swing but still nowhere near as flexible yet.

Cas :slight_smile:

#9

Yes, IMO JavaFX is a dead end. It’s still not possible to do some things you can do with Swing (don’t see either Swing-based IDE upgrading!), while there’s less access to low-level stuff. The Swing-JavaFX interop is also “funky”, which means evolving projects is hard - rather important to increasing use. I think it would have been better to have an up-to-date replacement for AWT, and let the actual UI development happen outside of the JDK (not that JavaFX is really in it).

I agree with @Riven about many of the poor clientside API choices. Inside most JavaFX builds is GStreamer, but with access to about 1% of its functionality. As the maintainer of the Java bindings for GStreamer I find that disappointing (if fruitful!), but that you then can’t hook into the JavaFX rendering in any useful way, or even easily overlay rendering in a JavaFX UI, is a real pain.

Disagree on the JavaSound API - that’s the one I think is well designed (if the backends a bit crusty). People dislike it because they don’t read the docs properly! :wink: An API that doesn’t try and abstract differences across platforms, but provides the ability to query abilities, is IMO far more useful.

#10

There’s way too much NIH with Java. Why JavaSound when there was OpenAL? Why a “secret hidden implementation” of OpenGL/GLES in JavaFX when there was LWJGL? Why completely hide GStreamer away? etc.

Cas :slight_smile:

#11

In general I sympathise with that, although JavaSound is more low level than OpenAL - PortAudio would be a better comparison - I actually believe something of the complexity of OpenAL belongs outside. But where I think JavaSound is better than the other media examples (similar to File vs nio2) : there are two ways to approach the cross-platform thing - one is to target the lowest common denominator, the other is to provide the ability to query, access and plugin to the features of the underlying platform. The new java.awt.desktop stuff in Java 9 is another example.

But in general, the NIH stuff is more about not exposing it than not using it, in which case I wish there would be more of a move to have the implementations outside the JDK or JavaFX eg. using service providers / modularisation. That way the “default” implementations or the alternatives are working on a level playing field, and there’s no hidden access.

#12

This seems very related to a (non-gaming) problem that I am experiencing (that is also why I registered):

I am currently developing a viewer for 3d data that shows orthogonal slices of the data in three separate separate ImageView instances inside a GridPane. The fourth panel shows a 3d rendered scene of selected objects and the orthogonal slices as textured meshes that get updated whenever the image property of the ImageView changes. When the window size gets large, the UI gets unresponsive whenever the image property is changed because instead of sharing textures, the textures need to be created from the image property, both for the ImageView and the mesh, on the application thread!

I am here mostly to rant, but if you have any suggestion to either

  • share textures between an ImageView and a PhongMaterial, or
  • create textures from an Image off the application thread,

I’d be glad for any input.