Machine dependent problems in Virtual Globe

rune.aasgaard · January 12, 2005, 5:28am

Dear all!

I am currently porting a Virtual Globe application (http://globe.sintef.no/) from Java3d to JOGL, and have run into some problems that shows up on only a few machines. I wonder if you could take a look and help me looking for fixes and workarounds. I do also hope that so many as possible could try the application and give me a hint of problems and sucesses. mail to: rune.aasgaard@sintef.no

Web start URL for the JOGL globe:
http://globe.sintef.no/webstart/virtualglobe.jnlp

The GL stuff in the application is pretty straightforward, using vertex, texture, normal arrays and glDrawElements. There is most probably not trivial errors, as it works on most computers, with a few exceptions. All of them also have pretty recent drivers and they all run the Java3d version of the Globe without problems.

I can’t give you the complete sourcecode, but I will show some extracts where useful. I don’t think it will be possible to create a shaved down minimum example.

The problems I have experienced are:

The strangest of them all: The application starts, loads its data, shows the java GUI, JOGL is initialized and the render loop runs. Both the init and display calls are obviously called and even the “frame pr. second” counter shows resonable values, but nothing appears in the GL window! Not even the clearColor! Only a Java-gray background. The most embarrasing thing is that this happens for a journalist that may give the project a little PR…

Platform: XP pro, Intel 865G integrated graphics, newest drivers, runs the JGears demo without problems. Java 1.4.2

The whole JavaVM freezes up after a few seconds! Even the Java Web Start Controle panel freezes! The computer has no problems with the java3d version, running OpenGl java3d. When the VM freezes it really stops using CPU, and the memory usage shows reasonable values.

Platform: Dual PIII 1GHz, Linux Suse 9.0, GF2 GTX, about 1/2 year old drivers, Java 5: latest official release.

The last one really looks like a driver or graphics card problem: The application freezes when I do a “dump GL window to clipboard”, using glReadPixels. It uses very little CPU, and reasonable application memory, but the total Windows memory count increases steadily, indicating that something is rotten in the graphicas drivers.

Plattform: PIII Celeron 1.4GHz, XP pro, ATI radeon 9200 plain ol’ PCI, new drivers.

Anybody got an idea?

kbr · January 12, 2005, 5:35pm

First, your port looks pretty cool; sorry you’re running into problems. Unfortunately, everything looks fine on my machine (including capturing the current view to the clipboard) so I can’t reproduce the problems you’re seeing. I see you’ve specified -Dsun.java2d.noddraw=true in your JNLP file which is good, so DirectDraw-related incompatibilities shouldn’t be causing problems.

Issue (1) looks like problems others have had with Intel graphics chips. Try specifying -DATI_WORKAROUND=true on the command line and/or in your JNLP file. That should transfer all of the OpenGL work onto the AWT event queue thread and may address multithreading issues in the drivers.

Issue (2) looks like a possible deadlock. Can you hit Ctrl-\ (control-backslash) on the console when that occurs and post a stack trace?

Issue (3) looks like you’re calling glReadPixels from the wrong thread, i.e., when the context is not current. However, this should fail on most if not all graphics cards, including mine, but seems to work. Could you try hitting Ctrl-Break on the command line and posting the thread dump from this scenario if one is available?

girvine · January 13, 2005, 5:01am

Interestingly this works quite fine on my Inspiron 9200 (Centrino 2.1GHz + Mobile Radeon 9700) with standard Dell drivers…

My only comment is it’s relatively slow to update with the new higher resolution graphics as you zoom in. I guess it must be loading it all across the network though so perhaps that’s it but my link is far from flooded. Maybe it’s the international connection from Aus’. shrug

FYI: frame rate is general 26fps up to 30fps.

rune.aasgaard · January 13, 2005, 7:10am

To Ken:

The ATI_WORKAROUND worked for problem 2 (!), but not for problem 1. It seems a little unfair to ATI to name this workaround after them when it is also used for other graphic processors…

I shall take a look at problem 3 this evning (it’s my home computer). I believe everything is done properly. The glReadPixels call is done in the render thread. The gui (awt) thread raises a “request_dump” flag in the render object, the render thread checks this flag when all drawing is completed, and if it is raised it calls glFinish and glReadPixels. The resulting byte buffer is copied into a BufferedImage and is handled over to the awt thread that saves it to file or to the clipboard.

To girvine:

The slowness is most probably resulting from the network connection across the world. You should go into the file menu and turn on the local cache. Then the data for areas you already have visited are stored on your local disk, and are not re-read the next time you go to the same area.

robnugent · January 13, 2005, 1:17pm

Rune,

This is a really nice app. Worked well for me (GeForce FX5200/Win2K/Athlon 1800+).

I had one problem which was a solid hang when I tried to copy the current image to the clipboard - I ended up having to kill the ‘javaws’ process.

It’s really nice to see how the model resolution changes when you tweak the details settings when in wire frame mode - an impressive amount of detail when you zoom in.

Out of interest, is this the SRTM elevation data that you are using, or some other data set ?

Rob

rune.aasgaard · January 13, 2005, 6:59pm

Ken:

Now I’ve had another look at error no 3, (hang in glReadPixels), and it is really not possible to get any stack or thread dump from it because the console window also hangs. In the Windows application error log I find an event about “Application hang” in javaw, at address 0x00000000.

I am able to reproduce the error on our other home PC, a Dell laptop with GF5200 Go (but none of the PCs at work), and a collegue has reproduced it on his home PC (the same type dell).

I’ll post an extract of the source code tomorrow.

Rob:

It seems like you also have run into error no. 3!

Yes it is SRTM for Europe and North America (GTOPO30 for the rest of the world). Only the preliminari version so far. I expect to upgrade to the complete coverage finished SRTM dataset in a month or two.

pdjensen · January 13, 2005, 7:00pm

I find this application exhibits two of the resize problems I’ve seen in my own application, at least on a Mac 1.33Ghz 17" Powerbook.

I have a suspicion the problem lies in the JOGL arithmetic in determining the upper left screen coordinates, where they are allowed to extend outside of the canvas.

It looks like the repaint after a resize is delayed behind the window event - as if the previously reported size as the window continuously repaints during the drag of a corner is what is displayed when the drag completes. So if the window is expanding, there is white space or garbage at the right and bottom edges. If it is decreasing, the view may be cropped to something larger.
If I resize, at least on the Mac, I can move the bottom of the window up into the menu area, causing the following:

[Quote]Java Web Start 1.4.2_05 Console, started Thu Jan 13 14:43:55 MST 2005
Java 2 Runtime Environment: Version 1.4.2_05 by Apple Computer, Inc.
Logging to file: /Users/pdj/Desktop/JWSlog
Opened connection to elevation server: http://globe.sintef.no/globe.fcgi
OpenGL vendor: ATI Technologies Inc.
OpenGL renderer: ATI Radeon 9600 OpenGL Engine
OpenGL version: 1.5 ATI-1.3.36
OpenGL extensionsGL_ARB_transpose_matrix GL_ARB_vertex_program GL_ARB_vertex_blend GL_ARB_window_pos GL_ARB_shader_objects GL_ARB_vertex_shader GL_ARB_fragment_shader GL_EXT_multi_draw_arrays GL_EXT_clip_volume_hint GL_EXT_rescale_normal GL_EXT_draw_range_elements GL_EXT_fog_coord GL_APPLE_client_storage GL_APPLE_specular_vector GL_APPLE_transform_hint GL_APPLE_packed_pixels GL_APPLE_fence GL_APPLE_vertex_array_object GL_APPLE_vertex_program_evaluators GL_APPLE_element_array GL_APPLE_flush_render GL_NV_texgen_reflection GL_NV_light_max_exponent GL_IBM_rasterpos_clip GL_SGIS_generate_mipmap GL_ARB_imaging GL_ARB_point_parameters GL_ARB_texture_env_crossbar GL_ARB_texture_border_clamp GL_ARB_multitexture GL_ARB_texture_env_add GL_ARB_texture_cube_map GL_ARB_texture_env_dot3 GL_ARB_multisample GL_ARB_texture_env_combine GL_ARB_texture_compression GL_ARB_texture_mirrored_repeat GL_ARB_shadow GL_ARB_depth_texture GL_ARB_shadow_ambient GL_ARB_fragment_program GL_ARB_occlusion_query GL_ARB_point_sprite GL_ARB_vertex_buffer_object GL_EXT_compiled_vertex_array GL_EXT_texture_rectangle GL_EXT_texture_env_add GL_EXT_blend_color GL_EXT_blend_minmax GL_EXT_blend_subtract GL_EXT_texture_lod_bias GL_EXT_abgr GL_EXT_bgra GL_EXT_stencil_wrap GL_EXT_texture_filter_anisotropic GL_EXT_separate_specular_color GL_EXT_secondary_color GL_EXT_blend_func_separate GL_EXT_shadow_funcs GL_EXT_texture_compression_s3tc GL_EXT_blend_equation_separate GL_EXT_texture_mirror_clamp GL_APPLE_ycbcr_422 GL_APPLE_vertex_array_range GL_APPLE_texture_range GL_APPLE_float_pixels GL_APPLE_pixel_buffer GL_NV_blend_square GL_ATI_texture_mirror_once GL_ATI_text_fragment_shader GL_ATI_blend_equation_separate GL_ATI_blend_weighted_minmax GL_ATI_texture_env_combine3 GL_ATI_separate_stencil GL_SGIS_texture_edge_clamp GL_SGIS_texture_lod GL_SGI_color_matrix
Fetching images from: http://globe.sintef.no/WMSproxy.fcgi?wms-host=onearth.jpl.nasa.gov&wms-path=wms.cgi&layers=modis,global_mosaic&pos-code=
Fetching images from: http://globe.sintef.no/WMSproxy.fcgi?wms-host=onearth.jpl.nasa.gov&wms-path=wms.cgi&layers=modis,global_mosaic&pos-code=
net.java.games.jogl.GLException: java.lang.reflect.InvocationTargetException
at net.java.games.jogl.GLCanvas.displayImpl(GLCanvas.java:205)
at net.java.games.jogl.GLCanvas.display(GLCanvas.java:75)
at net.java.games.jogl.Animator$1.run(Animator.java:107)
at java.lang.Thread.run(Thread.java:552)
Caused by: java.lang.reflect.InvocationTargetException
at java.awt.EventQueue.invokeAndWait(EventQueue.java:834)
at net.java.games.jogl.GLCanvas.displayImpl(GLCanvas.java:203)
… 3 more
Caused by: net.java.games.jogl.GLException: glGetError() returned the following error codes after a call to glViewport(): GL_INVALID_VALUE
at net.java.games.jogl.DebugGL.checkGLGetError(DebugGL.java:13783)
at net.java.games.jogl.DebugGL.glViewport(DebugGL.java:12638)
at net.java.games.jogl.GLCanvas$1.run(GLCanvas.java:114)
at net.java.games.jogl.impl.macosx.MacOSXOnscreenGLContext$1.run(MacOSXOnscreenGLContext.java:76)
at net.java.games.jogl.impl.GLContext.invokeGL(GLContext.java:287)
at net.java.games.jogl.impl.macosx.MacOSXOnscreenGLContext.invokeGL(MacOSXOnscreenGLContext.java:81)
at net.java.games.jogl.GLCanvas$DisplayOnEventDispatchThreadAction.run(GLCanvas.java:238)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:171)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:454)
at java.awt.EventDispatchThread.pumpOneEventForHierarchy(EventDispatchThread.java:234)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:184)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:178)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:170)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:100)
[/quote]

kbr · January 13, 2005, 9:31pm

[quote]Now I’ve had another look at error no 3, (hang in glReadPixels), and it is really not possible to get any stack or thread dump from it because the console window also hangs. In the Windows application error log I find an event about “Application hang” in javaw, at address 0x00000000.

I am able to reproduce the error on our other home PC, a Dell laptop with GF5200 Go (but none of the PCs at work), and a collegue has reproduced it on his home PC (the same type dell).
[/quote]
Can you try to use the ProfileGL composable pipeline to see where in your application the hang occurs? I am guessing it is during some OpenGL call, possibly even the glReadPixels routine, and the question at that point would be why it is happening.

rune.aasgaard · January 14, 2005, 7:07am

Dear Ken and everybody else!

I have solved issue no 3, the freeze in widow capture. Eventually it wasn’t a direct JOGL problem after all, the hang really happened in a buffered image copy after the glReadPixels call.

The relevant code is attached, with comments describing the problems (for others to learn!)


public class JOGLCamera 
    implements GLEventListener, WindowListener {
 
  // Lots of irrelevant stuff deleted

  private class CaptureRequester {
    BufferedImage image = null;
    boolean request = false;
  };
  private CaptureRequester captureRequester = new CaptureRequester();

  static private ColorModel glColorModel;
  static private ColorModel glAlphaColorModel;
  static {
    glColorModel = new ComponentColorModel(ColorSpace.getInstance(ColorSpace.CS_sRGB),
        new int[] {8,8,8,0},
        false,
        false,
        ComponentColorModel.OPAQUE,
        DataBuffer.TYPE_BYTE);
    glAlphaColorModel = new ComponentColorModel(ColorSpace.getInstance(ColorSpace.CS_sRGB),
        new int[] {8,8,8,8},
        true,
        false,
        ComponentColorModel.TRANSLUCENT,
        DataBuffer.TYPE_BYTE);
  }

  // Lots of irrelevant stuff deleted
  
  public void display(GLDrawable drawable) {
    GL gl = drawable.getGL();
    GLU glu = drawable.getGLU();

    // Set up all of the per frame stuff and do rendering
    // Lots of stuff deleted   

    // Here we capture the image
    synchronized (captureRequester) {
      if (captureRequester.request) {
        ColorModel cm = glAlphaColorModel;
        int num_comp = 4;
        int format = GL.GL_RGBA;

        gl.glFinish();

        // Get data into a buffer
        ByteBuffer imageBuffer = ByteBuffer.allocateDirect(width*height*num_comp);
        imageBuffer.order(ByteOrder.nativeOrder());
        gl.glReadPixels(0, 0, width, height,
                        format, GL.GL_UNSIGNED_BYTE, imageBuffer);
        imageBuffer.rewind();

        // Copy to buffered image
        WritableRaster raster = Raster.createInterleavedRaster(DataBuffer.TYPE_BYTE, width, height, num_comp, null);
        captureRequester.image = new BufferedImage(cm, raster, false, new Hashtable());

        byte[] data = ((DataBufferByte) raster.getDataBuffer()).getData();
        for (int i = height-1; i >= 0; --i)
          imageBuffer.get(data, i*width*num_comp, width*num_comp);
        captureRequester.request = false;
        captureRequester.notifyAll();
      }
    }
  }

  // Request a window capture, wait for the image to be created
  // This call should not be performed in the awt thread (in actionListeners...), 
  // as it will block Waiting for the opengl thread, which again may block waiting for 
  // the awt thread!
  // Instead, create a separate Thread.
  public BufferedImage getCapture() {
    BufferedImage captured = null;
    synchronized (captureRequester) {
      captureRequester.request = true;
      while (captureRequester.request) {
        try { captureRequester.wait(); }
        catch (InterruptedException ex) {}
      }
      captured = captureRequester.image;
      captureRequester.image = null;
    }
    if (captured == null) return null;

    // Convert to the device's preferred image format
    // NOTE!! The original bug was that this call was inside the 
    // synchronized block. It seems like Graphic2D.drawImage calls
    // requires opengl, and blocks if the opengl thread is blocked!
    BufferedImage image =
        GraphicsEnvironment.
        getLocalGraphicsEnvironment().
        getDefaultScreenDevice().
        getDefaultConfiguration().
        createCompatibleImage(captured.getWidth(), captured.getHeight());
    image.createGraphics().drawRenderedImage(captured, new AffineTransform());
    return image;
  }
}

There is a minor remaining problem, when getCapture is called in a separate thread the writing of the final image may take some time, and the user may attempt to read the image into another application before it is completely written. I may fix this by using a non-blocking getCapture that registers an listener/observer to be called when the capture is complete…

So, back to the unsolved issues…

Anybody reproduced my error no 1? Could it be related to the problems pdjensen is experiencing? Please?

kbr · January 14, 2005, 4:16pm

Glad you fixed the clipboard capture problem. Java2D doesn’t use OpenGL internally by default (unless you request the OpenGL pipeline which is new in 1.5) but there definitely could be handshaking between the thread calling drawRenderedImage and the AWT event queue thread. However in this case I would expect that you could hit Ctrl-Break to get a thread dump, and it shouldn’t cause a crash. Nevertheless I’m glad you’ve found a workaround.

Regarding Issue #1, I wonder whether the DefaultGLCapabilitiesChooser may be choosing a pixel format that the chipset can’t handle well. I would have pointed you at Issue 81 on the JOGL Issue Tracker where the team at Mojang Specifications contributed an alternate implementation. However that source code wasn’t attached to the bug and is no longer on their web server. I’ve emailed them about it and if I hear back then I’ll put the code in the bug database as an attachment. You could also try writing your own GLCapabilitiesChooser and see if that makes a difference.

rune.aasgaard · January 19, 2005, 5:25am

Well, the troublesome computer runs the JGears test correctly, and my window initialization shouldn’t be much different from that. Could it be of any help to create a debug GLCapabilitiesChooser wrapper that prints some more info of possible choises?

Just to see if I have (not) done anything stupid in the initialization I’ll post extracts of my initialization code:


public class JOGLCamera
    implements GLEventListener, WindowListener {

  private JPanel graphics_view;

  private GLCanvas glcanvas;
  private Animator animator;

  public JOGLCamera() {
    glcanvas = GLDrawableFactory.getFactory().createGLCanvas(new GLCapabilities());
    // glcanvas.setGL(new TraceGL(glcanvas.getGL(), System.err));
    // glcanvas.setGL(new DebugGL(glcanvas.getGL()));
    glcanvas.addGLEventListener(this);
    graphics_view = new JPanel();
    graphics_view.setLayout(new BorderLayout());
    graphics_view.add(glcanvas,  BorderLayout.CENTER);
  }

 public Component getGraphicsView() {
    return graphics_view;
  }


  public void windowOpened(WindowEvent evt) {
    animator = new Animator(glcanvas);
    animator.start();
  }

  public void windowClosing(WindowEvent e) {
    animator.stop();
    animator = null;
  }

  public void init(GLDrawable drawable) {
    // this.gldrawable = drawable;
    GL gl = drawable.getGL();
    GLU glu = drawable.getGLU();
    gl.glColor3f(1.0f, 1.0f, 1.0f);
    gl.glClearColor(0.2f, 0.2f, 0.5f, 1.0f);


    gl.glEnable(GL.GL_CULL_FACE);
    gl.glEnable(GL.GL_DEPTH_TEST);
  }
}

And the JOGLCamera is included in the main frame:


class ApplicationFrame extends JFrame {
  JSplitPane  rootSplitPane = new JSplitPane();
  JPanel      graphicView = new JPanel(new BorderLayout());

  public ApplicationFrame(String[] args) throws HeadlessException {
    setTitle("Virtual Globe");

    createGUI(args);
    universe.parseArgs(args);

    setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

    pack();
    setVisible(true);
  }

  private void createGUI() {
    rootSplitPane.setOrientation(JSplitPane.HORIZONTAL_SPLIT);
    ....
    graphicView.setMinimumSize(new Dimension(300,300));
    graphicView.setPreferredSize(new Dimension(600,600));
    
    ....
  
    rootSplitPane.setRightComponent(graphicView);
    getContentPane().add(rootSplitPane);


    camera = universe.getGraphics().createCamera();
    if (camera instanceof WindowListener)
      addWindowListener((WindowListener)camera);
    graphicView.add(camera.getGraphicsView(), BorderLayout.CENTER);
  }
}

kbr · January 20, 2005, 9:36pm

There is some debugging code in the DefaultGLCapabilitiesChooser which can be enabled by uncommenting the DEBUG flag in it and recompiling. I would like to make this debugging information dependent on system properties (there are several places in the code base where doing so would be helpful) and if you’d like to file a bug report about that please feel free. You could also extract the debugging code from the DefaultGLCapabilitiesChooser and make a wrapper Chooser as you suggested.