Java2D/JOGL Interoperability Demo

OpenGL Development JOGL Development
#1

The Java2D implementation in Mustang (JDK 6) build 51 and JOGL now have a powerful new interoperability capability.

In the JOGL library, when 100% correct Swing integration is required, the GLJPanel widget must be used. This is a pure lightweight widget which overlaps correctly with other Swing widgets. Previously, the best-case code path for the GLJPanel performed OpenGL rendering into a pbuffer, read back the frame buffer into an AWT BufferedImage, and drew the resulting pixels via Graphics.drawImage(). While the results were repeatable across platforms, the frame buffer readback and BufferedImage draw made this solution significantly slower than the heavyweight widget case and therefore unusable for many applications.

With Mustang build 51 and the current JOGL sources, interoperability between Java2D’s new Single-Threaded Rendering OpenGL pipeline and JOGL’s GLJPanel has been introduced. (See Chris Campbell’s blog entry for some more details.) JOGL can now render directly into the OpenGL drawable used by the Java2D OpenGL pipeline, providing full hardware acceleration for the GLJPanel and eliminating the readback of the frame buffer. The result is that the GLJPanel is almost as fast as the heavyweight GLCanvas and provides complete integration with Java2D. With this capability, you can:

[] Overlay Swing components (lightweight menus, tooltips, and other widgets) on top of OpenGL rendering.
[] Use 3D graphics anywhere you would ordinarily use a Swing widget. (Inside a JTable, JTree, …)
[] Draw 3D OpenGL graphics on top of Java2D rendering. The JOGL JGears demo (also in the JRefract demo menu as “Gears”) provides an example of this where the background is drawn with a Java2D GradientPaint and a translucent GLJPanel with a zero-alpha background draws the gears on top.
[] Draw Java2D graphics on top of 3D OpenGL rendering. The JGears demo again provides an example of this: the icons and frames-per-second counter are drawn on top of the gears using Java2D.

The best part is that no changes to the JOGL application are necessary due to JOGL’s GLEventListener rendering model. Simply add your GLEventListener to a GLJPanel instead of a GLCanvas.


https://jogl-demos.dev.java.net/jrefract-sol-sm.jpg

Try it out! A demo is available on line via Java Web Start. There are still some remaining issues (it looks like the InfiniteShadowVolumes demo crashes the JVM on X11, for one) but most demos are working perfectly. To run the demo:

[] Download Mustang build 51 or later from java.net.
[] Install either the JRE or JDK bundle.
[] Check to make sure the Mustang version of Java Web Start is registered as the helper application for MIME type “application/x-java-jnlp-file”.
[] Click the launch links below!

JRefract demo WITHOUT Java2D/JOGL Interoperability
JRefract demo WITH Java2D/JOGL Interoperability

Notice in particular the huge speed increase in the Gears demo. These demos are fully accelerated on Windows and X11 OSs and have been tested with NVidia and ATI hardware. We plan to work with Apple Computer to be able to achieve these speedups on Mac OS X as well.

There are some known issues that should be mentioned:

[] There is significant lag of delivery of mouse events on some Windows machines with some of the more computationally intensive demos. We will try to diagnose and fix this soon.
[] With NVidia graphics cards on Windows, there is a significant slowdown when dragging the JInternalFrames around the JDesktopPane. This is due to a bug in NVidia’s drivers which we have heard will be fixed in their next driver update. (Note that resizing operations are very fast.)

The new code has been checked in to the JOGL source tree under the JSR-231 branch. If you would like to experiment with it, please check out and build the source tree:


cvs co -r JSR-231 -P jogl jogl-demos

Adding -Dsun.java2d.opengl=true (specify “True” instead to get debugging information from Java2D) to your command line will enable the Java2D OpenGL pipeline; JOGL will automatically take advantage of it in its GLJPanel implementation. You can get debugging information from the JOGL side by specifying -Djogl.debug.Java2D -Djogl.debug.GLJPanel. The JOGL/Java2D integration can be disabled with -Djogl.gljpanel.noogl.

We are excited about the new visual effects this integration will enable. Please post here with any comments, questions or suggestions. The new JOGL APIs this support is built on top of are planned to be in the public API of JSR-231, which should be out for public review soon.

#2

As noted in the relevant bug in suns bug db, could the relevant api’s please be documented? I’ve noted two of the 3 method names, but so far I have not located them in the code base of b51. What the 3rd name is ofcourse anyones guess - I have therefore yet to locate the three infamous methods, that miraculously allows opengl interoperability…
It would be nice if Sun tried to help everybody in providing proper access instead of doing it all in the background, thus limiting others in implementing an OpenGL binding…

If lwjgl had ever gone ahead and tried to implement proper panel support, would it ever have been possible? Personally I would not think so, because sun wouldn't have had any incentive to help out. It's stuff like this that makes me understand why people want a open standardized version, rather that the closed stuff we have now. I have previously always been a proponent of keeping java closed - but this made me think otherwise. *if* lwjgl had wanted to do this, we would have been told to write a JSR request, and wait some 2-3 years for it to complete, but when something is needed by some half-internal project, sun can whip it up in 10 mins - thats just so unfair (not to sound too whiny).

It would be nice to compete on somewhat same grounds, but it’s just not possible when we can’t control the actual distribution.

* Matzon rambles on about sun cannibalizing Log4J and other projects.

#3

just to point something out - this is not about planed lwjgl support for panels, since thats not our primary concern - but we should have been able to, if we wanted to.

#4

LWJGL could have implemented exactly the same JPanel support that JOGL had until recently: render to pbuffer, read back pixels, draw to screen using Graphics / BufferedImage. It’s fast enough for small windows but too slow for larger ones.

The new APIs are currently experimental and aren’t in a state where they can be set in stone. The new capability that has been exposed by Java2D under 6309763 was worked on as a side project by Chris Campbell from the Java2D team and myself and is only a proof of concept. There is another open RFE, 5037133, which is intended to be a more general bridge. I do think that the set of APIs we came up with is pretty minimal and avoids exposing too much of the Java2D internals so maybe they will not require too much more work.

If the LWJGL team had wanted to do a project like this, they could have contacted the Java2D team about it on the forums they frequent both on this site as well as javadesktop.org. Aside from turnaround times of getting test builds and getting time from the appropriate engineers (which isn’t always easy even when working within Sun) the JOGL project had no inherent advantage here aside from a longstanding desire to see this functionality become available.

#5

exactly, which is why it’s basically unusable.

As I said, I have read those reposrts, and commented on one of them - need more documentation! - 2 method names (of 3!) in some unknown package and unknown file is kinda hard to locate!

I applaud the effort, it’s just the way that it has been done, that sucks. No documentation, apart from some semi documented bug report.
What really annoys me about all this (and this is just my opinion) is that I am close too 100% sure that if we had commented on wanting to have access to J2D internals prior to any jogl project we would have most definately gotten a: no, sorry cant do - submit a JSR.

I am not sure how all this feature adding works - coz on the one hand Sun tells people to submit JSR’s since java is an open process yada yada - and on the other hand you now say we could just ask the devs directly…

I am not trying to flame you, or the effort - I just want this whole process to be more open, including the jsr-231.

#6

The reason it isn’t heavily documented is that it isn’t ready to be specified yet. If you want to see how it works you can always check out the JOGL sources, see which methods it’s accessing and how, and go read the Mustang sources (which are pretty well commented around the new APIs). You’re hypothesizing when you talk about what “might have happened” – the only changes that have to go through a JSR are specified public APIs. Whether you would have had success convincing the appropriate developers is anybody’s guess.

I personally had your project lead Cas added to the JSR-231 expert group. Unfortunately I can’t say that his contributions to the expert group were noteworthy – he didn’t attend any of the conference calls and sent fewer than five emails to the mailing list between June 2004 and February 2005, which is when most of the expert group’s decisions on API design and organization were made.

JSR-231 is noteworthy in its openness as the implementation has been going on under the BSD license in a public workspace and mentioned on these forums continually.

#7

but isn’t that the whole purpose of a JSR - to get info from people, and come up with alternate ideas for an implementation ?
By doing it all behind closed doors, makes it kinda hard to find any potential blockers for other users of said API. Potentially, you could design it in a way that Product XYZ cannot use it for some unknown reason - purely speculating ofcourse.

Yeah, I intend to :slight_smile:

Having read the jsr-231 code, I finally managed to find some of the methods:
invokeWithOGLContextCurrent - called invokeDisplayAction in bugreport ?
isQueueFlusherThread - called isQFTThread in bug report?
and the last unnamed method (actually several?):
getOGLViewport, getOGLScissorBox, getOGLSurfaceIdentifier

It’s not exactly fun all of this detective work…

I can’t and wont comment on Cas’ involvement, but suffice to say that he thought that windowed mode was a “debug” mode in lwjgl :wink:

Err - just because the codebase is in a branch of jogl, doens’t exactly make it easy to determine what constitutes a JSR-231 implementation. I can see some classes under javax.media.opengl, but I have no idea whats part of jogl and whats part of jsr-231…

Looking through the code, I am somewhat unclear as to how tightly jsr-231 is tied to awt/swing? In my point of view, I would have thought it abstracted out somehow, so that an SWT implementation would be possible? (ignoring politics).

#8

Good news… I hope it all works out well for the final APIs and Mustang release.

As far as LWJGL not being used… it seems it was quite clear in the beginning that LWJGL was a gaming only API that did at least three things OpenGL, OpenAL, and Input. If you look at the whole history of the gaming APIs that the Sun folks open sourced… where they split what LWJGL had into three separate APIs… and then how the direction that the JOGL API took made it generally useful for many applications including, but not limited to, games… well, it seem that this all worked out well. It just took a bit longer than the gaming community could stand, which made LWJGL successful.

The primary conflict was way back when the Sun group was originally working on their gaming APIs… and they were developed using a JSR process that was much less open, at a time when being open would have really helped the people here. But all of that is history. Sun did finally open the gaming APIs (when they appeared to not be getting anywhere what the JSR) and the resulting integration with Swing that we have now has come from it. Be happy.

#9

This will be discussed more once the specification goes to public review, but the key API is GLDrawableFactory.getGLDrawable(Object, GLCapabilities, GLCapabilitiesChooser) which creates an OpenGL drawable for the passed GUI object. Sun’s implementation accepts only a Canvas as argument. Other implementations are free to accept an SWT widget or similar.

#10

Thank you Ken and Chris. This is tremendously good news for us.
At this turning point, I’d like to be more involved than just being
a user and spewing out unreasonable requests in these forums.
I’d like to work with you guys at the design level and provide
feedback on how JOGL/Java2D is used in the project I’m leading.
I feel this can have real benefits to the way this integration shapes up.
Please let me know how to proceed or paths I can take.

BTW, can you elaborate on what you mean by ‘The result is that the
GLJPanel is almost as fast as the heavyweight GLCanvas’?
What is the worst case scenario and what kinds of things make
it slower than GLCanvas?

.rex

#11

I know the history too - but it’s hardly an excuse to be doing stuff behind closed curtains…

Did it? - Jogl works well, despite some key issues. Joal is IMO largely untested, has had no releases for a loong time and project lead is AFAIK not working on it any more, JInput is semi broken on linux last time I checked and again, project lead has stopped working on it AFAIK, only Endolf is working on fixing it. I really dont think this constitutes a gaming api that worked out well…
fwiw, I have always thought it odd that they duplicated the exact same features of LWJGL - if anything they should have said, hey we want interoperability with awt/swing, can we do this with the current LWJGL code base?, Would you be interested in working in that direction? From my point of view, the whole “gaming” API is a huge flop, and reaks of NIH syndrome.

edit - clarification

#12

Thanks!, could you list the public classes so I can look at what work is needed to support jsr-231? - private if need be?

#13

Great news !

Does it means with mustang we will only have to ship the jogl.jar and no dlls (as we will be sharing the java2d internal ogl renderer) ?

(that would be great !).

Lilian

#14

I am largely ignored and generally not of much use in the JSRs on account of not having a mulimillion dollar interest backing my opinions. It’s also pretty clear that in JSR231 that movement is glacial; that is, staggeringly slow to accomplish anything; and once set on a particular path there’s no stopping it. Not that I care too much! LWJGL remains the only sanely designed OpenGL binding IMHO :slight_smile:

Cas :slight_smile:

#15

The latest APIs haven’t even been posted to the expert group yet so it would be premature to post javadoc publicly. However, you can get the latest source code with


cvs -d :pserver:guest@cvs.dev.java.net:/cvs co -r JSR-231 -P jogl jogl-demos

You’ll need to build the sources and javadoc in order to have a complete API set, i.e.,


cd jogl/make
ant win32.vc6 javadoc

For what it’s worth, I have always thought the LWJGL full-screen support would have fit in nicely to JOGL’s GLDrawableFactory pattern; we could have added a createFullScreenDrawable() using LWJGL’s full-screen code path (and I proposed this a couple of years ago to the LWJGL team, but it didn’t go anywhere). At this point it should be possible to implement this by modifying the JSR-231 implementation to take null, a GraphicsDevice, or similar construct to signal to the JSR-231 implementation to go down the alternate code path. However, work has been done in Mustang to improve the JDK’s built-in full-screen support so hopefully additional native code to handle this case shouldn’t be necessary (for one thing, it works on Linux now).

#16

We monitor these forums continually and you can reach me via email at my java.net mailing address. Please either post or email and let us know how your project is progressing and if any issues come up with the new support.

Swing specifies a persistent back buffer for its components; in other words, it isn’t (easily) possible to implement Swing with a buffer-swapping paradigm. For this reason Swing’s “buffer swap” is implemented internally by Java2D by a (hardware-accelerated) pixel copy operation, at least to the best of my understanding. (This may be equivalent to rendering with a certain texture under some circumstances). This is not quite as fast as the case of swapping the buffers of a heavyweight. There are also some issues with, for example, mouse lag on some Windows systems that we’re still working on tracking down.

#17

I’m afraid not. There is still native code required on the JOGL side in order to share the OpenGL drawable with the Java2D implementation.

#18

Hi. I’m glad that the transparency (opaque being false) works, despite it’s slowness still, for the GLJPanel without falling back to the much dreaded microsoft’s lame gl version. Regarding full-screen, has the displayChanged() started trial implemenation? What is the current status or progress on the NURBS in jsr231? - AK77

#19

One immediate question I have is how much
texture memory Java2D will use. It is well-known
on MacOSX that the whole UI is accelerated
and the irony is that it takes resources away
from apps (e.g. Motion) that rely heavily on the
GPU. Fill rate and # of resident textures are all
affected and it is hard to tell exactly how much GPU
memory is available. MacOSX users are simply
advised to use as small a screen resolution
as possible when running apps like Motion.
With this in mind, I think a mechanism to control
how aggressive Java2D uses acceleration
will be very helpful in tuning and determining
the actual performance of the app.

So are you guys working towards something that’s
as fast as the heavyweight or staying with H/W
pixel copying whenever Java2D and JOGL is used
together? This will affect how I design the GUI and
workflow for my app.

.rex

#20

Cool.

This is the key fetaure that we have been missing for a long time now.

Keep up the good work
// Tomas :slight_smile: