Reshape-forcing and multismapling with Java2D-OpenGL-Pipeline

Saxer · September 17, 2007, 8:38pm

Hi!

Our diploma hits the final stage. So I just wanted to thank all of you and especially Ken for the given support.

Just for completeness I have two issues open in my agenda.

We once talked about a “forced reshape” which was suggested to be initialiced via panel.setSize(panel.getSize()). Unfortunately that doesn’t work for scrolled GLJPanels which are not visible due to the scrolled viewport. The used hack passed a BufferedImage.getGraphics-Objekt to the paint-method but that was hackish and some kind of redundant and slow. So how could we solve this in general. In my case a very “soon” reshape is desired
The other issue concerns the choise of the glcapabilities. Maybe that’s an issue specific to the tested graphics board (ati). Without the pipeline enabled I get multisampling working like a charme (but slow). With the pipeline the multisample support is just not there, which makes the rendering fast, but ugly :-(.

I don’t know if those issues are of general nature but maybe we could solve it in general (if anyone sees the problems I mentioned)

Thanks so far
Klemens

kbr · September 17, 2007, 10:41pm

I still don’t fully understand what you need in this area nor why the existing functionality doesn’t work for you. If the issue is that you need to change the “real” size of the GLJPanel, then you should be able to just use setSize(). If you need to update your own notion of how big it is, you can query its size in your display() callback each time.

In order to support this we will need to add a new rendering path to the GLJPanel which renders to its own FBO using GL_EXT_framebuffer_multisample and then draws that to the screen. A similar modification is needed in order to properly support setOpaque(false) against the GLJPanel when the Java2D/OpenGL pipeline is enabled. There isn’t an RFE filed about this yet so feel free to file one. However it will probably be a while before we (read: I) get to this, unless a member of the community would like to step up…(hint, hint)

broumbroum · September 17, 2007, 11:34pm

This issue might be common to all JComponent’s . The size of one JComponent keeps locked by the preferredSize it is assigned. Since highweight components do need this value to lay the child components out to the display screen, try to setPreferredSize() your GLPanel every time you “force reshape” and then revalidate() the frame ! I’m sure you’ll see the solution to appear.

Saxer · September 18, 2007, 7:57am

The problem is not that I can’t force a reshape in general. The setSize(getSize()) approach works for the common case. But it seems, that because of some Swing optimations the GLJPanel doesn’t get “paintetd” and therefore not reshaped when it is’s scrolled outside the visible area.

What I learned from the code is, that the reshape comes out of the paintComponent-Method. When this method is not invoked… no reshapes will follow. That’s why I did the hack which forces the reshape reliably.

Concerning the multisampling I’m wondering why the “default-fbo” doesn’t at least support it. It could be disabled programmatically (gl.glDisable(GL.GL_MULTISAMPLE)) for all of the Java2D-OpenGL-operations. But we could use it in our Jogl-apps by enabling it. I don’t know if this idea could work in general, but it sound promising and easy.

kbr · September 18, 2007, 3:48pm

I don’t understand why you need the GLJPanel to be painted if it isn’t visible.

However, you should be able to call GLJPanel.display() to force a redraw, even if the results won’t actually appear on-screen.

I’ve discussed this in the past with the Java 2D team. The main issue is that they want to optimize the use of VRAM. For example, they used to implement some clipping operations using the stencil buffer, but switched to using the depth buffer so they wouldn’t need the extra n bits per pixel. You can feel free to file an RFE to perhaps get some control over this via environment variables, but it will be faster to do this in JOGL.

Saxer · September 18, 2007, 5:41pm

I don’t need it painted, but to paint it is the only way to force the reshape… as you know the reshape is invoked from within the paintComponent method

I tried the display()-call but I think I don’t have a current-context for that. I tried it like that:

Java2D.invokeWithOGLContextCurrent(renderPanel.getGraphics(), new Runnable() {

			public void run() {
				
				renderPanel.display();
			}
		});

Yes I read about the VRAM thing. I’ll think about the RFE.

kbr · September 19, 2007, 7:22am

Much too complicated…just call renderPanel.display() from any thread. You should not need to know about the Java2D class. It’s an implementation detail.

Saxer · September 19, 2007, 8:19pm

That causes an exception about a none current context

broumbroum · September 19, 2007, 9:39pm

Well, is your application intended to get 3D animations on screen or is it an ordinary Java2D you want to get accelerated on OGL thing ?
For some unknown reasons , OGL doesn’t support direct rendering with Java2D, it’s already integrated as a Java2D Pipeline. Activate it through its commandline option -Dsun.java2d.opengl=True and that’s the way it gets on. Now that you want to force the render and multi-sampling the std repaint-update-paint process won’t run properly for accurate real-time rendering.
Use a Canvas and its (BufferStrategy).show() method to force rendering ops that include painting, scaling, transforms, sampling. etc. ! All the necessary code can be found at Canvas 1.6 Javadoc, as recent updates have been made since 1.5.
Then we can look forward to accelerate reshaping and sampling with JAI/JIIO extensions . That’s all about Java2D Pipelines. Regarding OpenGL it is not always available on most low-cost systems, even the latest Pentium and 64bits machines.

kbr · September 21, 2007, 7:49am

Could you please post a stack trace? The demos.jgears.JGears demo does exactly this, using an Animator from its own thread to repeatedly call display() against a GLJPanel. Note that if you call display() from within one of your GLEventListener’s callbacks you might run into trouble, but you should be able to do the display() call asynchronously on another thread to work around this.

Saxer · September 21, 2007, 4:14pm


Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
	at java.nio.HeapIntBuffer.<init>(Unknown Source)
	at java.nio.IntBuffer.allocate(Unknown Source)
	at javax.media.opengl.GLJPanel$Updater.display(GLJPanel.java:1111)
	at com.sun.opengl.impl.GLDrawableHelper.display(GLDrawableHelper.java:78)
	at com.sun.opengl.impl.GLPbufferImpl$DisplayAction.run(GLPbufferImpl.java:222)
	at com.sun.opengl.impl.GLDrawableHelper.invokeGL(GLDrawableHelper.java:194)
	at com.sun.opengl.impl.GLPbufferImpl.maybeDoSingleThreadedWorkaround(GLPbufferImpl.java:208)
	at com.sun.opengl.impl.GLPbufferImpl.display(GLPbufferImpl.java:88)
	at javax.media.opengl.GLJPanel.paintComponent(GLJPanel.java:659)
	at de.fusion.core.swing.FusionGLJPanel.paintComponent(FusionGLJPanel.java:383)
	at javax.swing.JComponent.paint(Unknown Source)
	at javax.swing.JComponent.paintChildren(Unknown Source)
	at javax.swing.JComponent.paint(Unknown Source)
	at javax.swing.JLayeredPane.paint(Unknown Source)
	at javax.swing.JComponent.paintChildren(Unknown Source)
	at javax.swing.JComponent.paintToOffscreen(Unknown Source)
	at javax.swing.RepaintManager$PaintManager.paintDoubleBuffered(Unknown Source)
	at javax.swing.RepaintManager$PaintManager.paint(Unknown Source)
	at javax.swing.RepaintManager.paint(Unknown Source)
	at javax.swing.JComponent.paint(Unknown Source)
	at java.awt.GraphicsCallback$PaintCallback.run(Unknown Source)
	at sun.awt.SunGraphicsCallback.runOneComponent(Unknown Source)
	at sun.awt.SunGraphicsCallback.runComponents(Unknown Source)
	at java.awt.Container.paint(Unknown Source)
	at javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
	at javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
	at javax.swing.RepaintManager.seqPaintDirtyRegions(Unknown Source)
	at javax.swing.SystemEventQueueUtilities$ComponentWorkRequest.run(Unknown Source)
	at java.awt.event.InvocationEvent.dispatch(Unknown Source)
	at java.awt.EventQueue.dispatchEvent(Unknown Source)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
Exception in thread "AWT-EventQueue-0" javax.media.opengl.GLException: Error making context current: 0
	at com.sun.opengl.impl.windows.WindowsGLContext.makeCurrentImpl(WindowsGLContext.java:169)
	at com.sun.opengl.impl.windows.WindowsPbufferGLContext.makeCurrentImpl(WindowsPbufferGLContext.java:102)
	at com.sun.opengl.impl.GLContextImpl.makeCurrent(GLContextImpl.java:134)
	at com.sun.opengl.impl.GLDrawableHelper.invokeGL(GLDrawableHelper.java:182)
	at com.sun.opengl.impl.GLPbufferImpl.maybeDoSingleThreadedWorkaround(GLPbufferImpl.java:208)
	at com.sun.opengl.impl.GLPbufferImpl.display(GLPbufferImpl.java:88)
	at javax.media.opengl.GLJPanel.paintComponent(GLJPanel.java:659)
	at de.fusion.core.swing.FusionGLJPanel.paintComponent(FusionGLJPanel.java:383)
	at javax.swing.JComponent.paint(Unknown Source)
	at javax.swing.JComponent.paintChildren(Unknown Source)
	at javax.swing.JComponent.paint(Unknown Source)
	at javax.swing.JLayeredPane.paint(Unknown Source)
	at javax.swing.JComponent.paintChildren(Unknown Source)
	at javax.swing.JComponent.paint(Unknown Source)
	at javax.swing.JComponent.paintToOffscreen(Unknown Source)
	at javax.swing.RepaintManager$PaintManager.paintDoubleBuffered(Unknown Source)
	at javax.swing.RepaintManager$PaintManager.paint(Unknown Source)
	at javax.swing.RepaintManager.paint(Unknown Source)
	at javax.swing.JComponent._paintImmediately(Unknown Source)
	at javax.swing.JComponent.paintImmediately(Unknown Source)
	at javax.media.opengl.GLJPanel.display(GLJPanel.java:252)
	at de.fusion.core.scenegraph.FusionSG$1.run(FusionSG.java:754)
	at java.awt.event.InvocationEvent.dispatch(Unknown Source)
	at java.awt.EventQueue.dispatchEvent(Unknown Source)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.run(Unknown Source)
Exception in thread "AWT-EventQueue-0" javax.media.opengl.GLException: Error making context current: 0
	at com.sun.opengl.impl.windows.WindowsGLContext.makeCurrentImpl(WindowsGLContext.java:169)
	at com.sun.opengl.impl.windows.WindowsPbufferGLContext.makeCurrentImpl(WindowsPbufferGLContext.java:102)
	at com.sun.opengl.impl.GLContextImpl.makeCurrent(GLContextImpl.java:134)
	at com.sun.opengl.impl.GLDrawableHelper.invokeGL(GLDrawableHelper.java:182)
	at com.sun.opengl.impl.GLPbufferImpl.maybeDoSingleThreadedWorkaround(GLPbufferImpl.java:208)
	at com.sun.opengl.impl.GLPbufferImpl.display(GLPbufferImpl.java:88)
	at javax.media.opengl.GLJPanel.paintComponent(GLJPanel.java:659)
	at de.fusion.core.swing.FusionGLJPanel.paintComponent(FusionGLJPanel.java:383)
	at javax.swing.JComponent.paint(Unknown Source)
	at javax.swing.JComponent.paintChildren(Unknown Source)
	at javax.swing.JComponent.paint(Unknown Source)
	at javax.swing.JLayeredPane.paint(Unknown Source)
	at javax.swing.JComponent.paintChildren(Unknown Source)
	at javax.swing.JComponent.paint(Unknown Source)
	at javax.swing.JComponent.paintToOffscreen(Unknown Source)
	at javax.swing.RepaintManager$PaintManager.paintDoubleBuffered(Unknown Source)
	at javax.swing.RepaintManager$PaintManager.paint(Unknown Source)
	at javax.swing.RepaintManager.paint(Unknown Source)
	at javax.swing.JComponent._paintImmediately(Unknown Source)
	at javax.swing.JComponent.paintImmediately(Unknown Source)
	at javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
	at javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
	at javax.swing.RepaintManager.seqPaintDirtyRegions(Unknown Source)
	at javax.swing.SystemEventQueueUtilities$ComponentWorkRequest.run(Unknown Source)
	at java.awt.event.InvocationEvent.dispatch(Unknown Source)
	at java.awt.EventQueue.dispatchEvent(Unknown Source)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.run(Unknown Source)

Saxer · September 21, 2007, 4:14pm


Exception in thread "AWT-EventQueue-0" javax.media.opengl.GLException: Error making context current: 0
	at com.sun.opengl.impl.windows.WindowsGLContext.makeCurrentImpl(WindowsGLContext.java:169)
	at com.sun.opengl.impl.windows.WindowsPbufferGLContext.makeCurrentImpl(WindowsPbufferGLContext.java:102)
	at com.sun.opengl.impl.GLContextImpl.makeCurrent(GLContextImpl.java:134)
	at com.sun.opengl.impl.GLDrawableHelper.invokeGL(GLDrawableHelper.java:182)
	at com.sun.opengl.impl.GLPbufferImpl.maybeDoSingleThreadedWorkaround(GLPbufferImpl.java:208)
	at com.sun.opengl.impl.GLPbufferImpl.display(GLPbufferImpl.java:88)
	at javax.media.opengl.GLJPanel.paintComponent(GLJPanel.java:659)
	at de.fusion.core.swing.FusionGLJPanel.paintComponent(FusionGLJPanel.java:383)
	at javax.swing.JComponent.paint(Unknown Source)
	at javax.swing.JComponent.paintChildren(Unknown Source)
	at javax.swing.JComponent.paint(Unknown Source)
	at javax.swing.JLayeredPane.paint(Unknown Source)
	at javax.swing.JComponent.paintChildren(Unknown Source)
	at javax.swing.JComponent.paint(Unknown Source)
	at javax.swing.JComponent.paintToOffscreen(Unknown Source)
	at javax.swing.RepaintManager$PaintManager.paintDoubleBuffered(Unknown Source)
	at javax.swing.RepaintManager$PaintManager.paint(Unknown Source)
	at javax.swing.RepaintManager.paint(Unknown Source)
	at javax.swing.JComponent._paintImmediately(Unknown Source)
	at javax.swing.JComponent.paintImmediately(Unknown Source)
	at javax.media.opengl.GLJPanel.display(GLJPanel.java:252)
	at de.fusion.core.scenegraph.FusionSG$1.run(FusionSG.java:754)
	at java.awt.event.InvocationEvent.dispatch(Unknown Source)
	at java.awt.EventQueue.dispatchEvent(Unknown Source)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.run(Unknown Source)
Exception in thread "AWT-EventQueue-0" javax.media.opengl.GLException: Error making context current: 0
	at com.sun.opengl.impl.windows.WindowsGLContext.makeCurrentImpl(WindowsGLContext.java:169)
	at com.sun.opengl.impl.windows.WindowsPbufferGLContext.makeCurrentImpl(WindowsPbufferGLContext.java:102)
	at com.sun.opengl.impl.GLContextImpl.makeCurrent(GLContextImpl.java:134)
	at com.sun.opengl.impl.GLDrawableHelper.invokeGL(GLDrawableHelper.java:182)
	at com.sun.opengl.impl.GLPbufferImpl.maybeDoSingleThreadedWorkaround(GLPbufferImpl.java:208)
	at com.sun.opengl.impl.GLPbufferImpl.display(GLPbufferImpl.java:88)
	at javax.media.opengl.GLJPanel.paintComponent(GLJPanel.java:659)
	at de.fusion.core.swing.FusionGLJPanel.paintComponent(FusionGLJPanel.java:383)
	at javax.swing.JComponent.paint(Unknown Source)
	at javax.swing.JComponent.paintChildren(Unknown Source)
	at javax.swing.JComponent.paint(Unknown Source)
	at javax.swing.JLayeredPane.paint(Unknown Source)
	at javax.swing.JComponent.paintChildren(Unknown Source)
	at javax.swing.JComponent.paint(Unknown Source)
	at javax.swing.JComponent.paintToOffscreen(Unknown Source)
	at javax.swing.RepaintManager$PaintManager.paintDoubleBuffered(Unknown Source)
	at javax.swing.RepaintManager$PaintManager.paint(Unknown Source)
	at javax.swing.RepaintManager.paint(Unknown Source)
	at javax.swing.JComponent._paintImmediately(Unknown Source)
	at javax.swing.JComponent.paintImmediately(Unknown Source)
	at javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
	at javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
	at javax.swing.RepaintManager.seqPaintDirtyRegions(Unknown Source)
	at javax.swing.SystemEventQueueUtilities$ComponentWorkRequest.run(Unknown Source)
	at java.awt.event.InvocationEvent.dispatch(Unknown Source)
	at java.awt.EventQueue.dispatchEvent(Unknown Source)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.run(Unknown Source)
Exception in thread "AWT-EventQueue-0" javax.media.opengl.GLException: Error making context current: 0
	at com.sun.opengl.impl.windows.WindowsGLContext.makeCurrentImpl(WindowsGLContext.java:169)
	at com.sun.opengl.impl.windows.WindowsPbufferGLContext.makeCurrentImpl(WindowsPbufferGLContext.java:102)
	at com.sun.opengl.impl.GLContextImpl.makeCurrent(GLContextImpl.java:134)
	at com.sun.opengl.impl.GLDrawableHelper.invokeGL(GLDrawableHelper.java:182)
	at com.sun.opengl.impl.GLPbufferImpl.maybeDoSingleThreadedWorkaround(GLPbufferImpl.java:208)
	at com.sun.opengl.impl.GLPbufferImpl.display(GLPbufferImpl.java:88)
	at javax.media.opengl.GLJPanel.paintComponent(GLJPanel.java:659)
	at de.fusion.core.swing.FusionGLJPanel.paintComponent(FusionGLJPanel.java:383)
	at javax.swing.JComponent.paint(Unknown Source)
	at javax.swing.JComponent.paintChildren(Unknown Source)
	at javax.swing.JComponent.paint(Unknown Source)
	at javax.swing.JLayeredPane.paint(Unknown Source)
	at javax.swing.JComponent.paintChildren(Unknown Source)
	at javax.swing.JComponent.paint(Unknown Source)
	at javax.swing.JComponent.paintToOffscreen(Unknown Source)
	at javax.swing.RepaintManager$PaintManager.paintDoubleBuffered(Unknown Source)
	at javax.swing.RepaintManager$PaintManager.paint(Unknown Source)
	at javax.swing.RepaintManager.paint(Unknown Source)
	at javax.swing.JComponent._paintImmediately(Unknown Source)
	at javax.swing.JComponent.paintImmediately(Unknown Source)
	at javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
	at javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
	at javax.swing.RepaintManager.seqPaintDirtyRegions(Unknown Source)
	at javax.swing.SystemEventQueueUtilities$ComponentWorkRequest.run(Unknown Source)
	at java.awt.event.InvocationEvent.dispatch(Unknown Source)
	at java.awt.EventQueue.dispatchEvent(Unknown Source)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.run(Unknown Source)

Saxer · September 21, 2007, 4:16pm

I think

de.fusion.core.scenegraph.FusionSG$1.run(FusionSG.java:754)

is the initial error

kbr · September 21, 2007, 8:41pm

I see an OutOfMemoryError at the top of your stack traces which looks to me like a likely candidate for the failure. Have you tried running with a larger heap size and does that help?

Saxer · September 22, 2007, 9:55am

I’m sorry… the heap-problems came from one mistake I made within the code-optimizations.

But I found out, that the GLJPanel uses this code (which produced the final out of memory-error):
glFormat = GL.GL_BGRA;
glType = (hardwareAccelerationDisabled
? offscreenContext.getOffscreenContextPixelDataType()
: hwGLFormat);
readBackInts = IntBuffer.allocate(readBackWidthInPixels * readBackHeightInPixels);

Couldn’t BufferUtils.newIntBuffer() give better heap-behaviour by using an native / direct buffer? As I can see the buffer is rebuild every (let’s say) frame. Maybe it could speed up the rendering by holding the buffer within the heap, because I don’t think, that a big number of GLJPanels are used at one time.

Finally I have to say, that the simple GLJPanel.display() call didn’t solve the missing reshape problem. The BufferedImage-hack is the only solution I have found yet.

kbr · September 22, 2007, 3:08pm

IntBuffer.allocate() takes the data out of the Java heap. Have you tried compiling JOGL from the source yet? Could you try switching over to using BufferUtil.newIntBuffer() and see whether that changes the behavior? Or do you have a small test case? If so please file a bug report and attach it.

Saxer · September 22, 2007, 8:50pm

I’ll test the recompilation next week. I think those changes could give a bit more speed…

Fortunately my heap-problems are gone (I had 10 million ints in the heap ;D). So there were no real problems with the IntBuffer in the GLJPanel.

Which behaviour do you mean with: “Could you try switching over to using BufferUtil.newIntBuffer() and see whether that changes the behavior?”

kbr · September 23, 2007, 3:33am

I meant that if you change the GLJPanel to use BufferUtil, does it get rid of the OutOfMemoryError and subsequent GLExceptions you were seeing?

Saxer · September 23, 2007, 7:29am

As I said the out of memory error is gone as I changed a bad bug within my own code. But I’ll make a testcase which demonstrates the reshape problem.