Can a Swing GUI be used in a direct-rendering game?

I said I’d tell how my experiences with Swing panned out, so…

I’ve read a lot about how GUIs work, including the SWT (IBM’s alternative to AWT/Swing) and as far as I can see (not very), Swing is fundamentally flawed for use in direct-rendering games since it is so closely tied to the AWT Event Dispatch Thread (EDT). As Ask_Hjorth_Larsen said it looks like the only thread-safe way Swing can be used is to have the whole game running in the EDT which won’t work.

The SWT (Standard Widget Toolkit) would work in direct-rendering games since there is no EDT, all communication with the Operating System for events and painting is done through the main thread. But I’ve decided not to use the SWT because its not completely flexible & portable due to the way SWT components are all backed by their native/heavyweight OS equivalents.

I’ve decided to go with my own crappy GUI system because at least it works and I know what it’s doing.

I’d love to know if anybody has used Swing successfully without doing EventQueue.invokeAndWait() etc, but unless I’ve stuffed up something obvious, the above example shows how Swing can’t be used in a direct-rendering game.

Keith

I think the problem is that the AWT event thread is accessing your swing components at the same time that you are drawing them from another thread. I have had the same problem and the following solution worked for me.

Add the following class…


import java.awt.AWTEvent;
import java.awt.EventQueue;
import java.awt.Toolkit;

//import jfreerails.controller.ReportBugTextGenerator;


/**
 * This event queue is synchronized on the MUTEX. This lets one control when
 * events can be dispatched.
 * 
 * Note, changed to be a singleton to get it working on pre 1.4.2 VMs.
 * 
 * @author Luke
 * 
 */
final public class SynchronizedEventQueue extends EventQueue {
	public static final Object MUTEX = new Object();

	private static final SynchronizedEventQueue instance = new SynchronizedEventQueue();

	private static boolean alreadyInUse = false;

	/** Enforce singleton property. */
	private SynchronizedEventQueue() {
	}

	public static synchronized void use() {
		if (!alreadyInUse) {
			/* set up the synchronized event queue */
			EventQueue eventQueue = Toolkit.getDefaultToolkit()
					.getSystemEventQueue();
			eventQueue.push(instance);
			alreadyInUse = true;
		}
	}

	protected void dispatchEvent(AWTEvent aEvent) {
		synchronized (MUTEX) {
			try {
				super.dispatchEvent(aEvent);
			} catch (Exception e) {
				/*
				 * If something goes wrong, lets kill the game straight away to
				 * avoid hard-to-track-down bugs.
				 */
				//ReportBugTextGenerator.unexpectedException(e);
			}
		}
	}
}


then change your game loop to…


public static void main(String[] arrghImAPirate) {
		SwingDirectRendering frame = new SwingDirectRendering();
		SynchronizedEventQueue.use();
		while (true) {
			synchronized (SynchronizedEventQueue.MUTEX) {

				Graphics g = frame.getBufferStrategy().getDrawGraphics();

				//Graphics g = frame.content.getGraphics()
				// to show that its not a BufferStrategy
				// problem you can uncomment the above and leave
				// out the BufferStrategy code

				frame.content.paintComponents(g);
				frame.getBufferStrategy().show();
			}
		}
	}



On my machine it seems to solve the deadlock problem.

Hope it helps

Luke

100% Sensational! :smiley: Luke you are a legend, that peice of code is a silver bullet!

Not only is it kicking ass in the test case but I just tried it in my game and it works perfect!

I thought that the synchronized block would sap performance but no, the frame rate doesn’t change whether or not I use the SynchronizedEventQueue. That’s a very elegant solution.
Being able to use Swing in my menus will make life a lot easier 8).

By the way, what is the cost (if any) of sync’ing these events? Speed? Mouse & keyboard responsiveness? In the test case how much of the game loop needs to be synchronized? To reduce cost of sync’ing, could you safely change it to this since much time is taken blitting from the bufferStrategy to the screen compared with painting to the bufferStrategy:

while (true) {

Graphics g = frame.getBufferStrategy().getDrawGraphics();
synchronized (SynchronizedEventQueue.MUTEX) {
frame.content.paintComponents(g);
}
frame.getBufferStrategy().show();
}
}

Many thanks,
Keith

Wow, I never thought of trying anything like that before. When I get a chance, I’ll try doing something similar in my code to see what happens. Thanks for the idea.

Whoa, whoa, wait a minute.

It’s possible to use active rendering within the EDT unless I have completely misunderstood the concept. Rendering is not magically active or passive as determined by which thread it runs from. The repaint() system, which queues and collapses calls and so on is passive rendering. If you perform the rendering immediately when desired, as opposed to queueing and collapsing, then it is active rendering. If this is correct (which is not trivial because the invokeAndWait method which I’m going to suggest may involve some EDT queueing along with peripheral input and so on, though no repaint management takes place), then the invokeAndWait method can easily be used to render actively from the EDT. I assume the invokeAndWait is sufficiently efficient (i.e. that the passive rendering overhead which we hate is incurred mostly by the paint management and thus not the queueing itself).

Jeff said:

[quote]Right. hats one thread.

Where is the other one?

Its that which im having a hard time imgining.
[/quote]
It’s the main Thread of the game, which is used for the main loop. That’s not the EDT, so it is another Thread.

So basically the user fiddles with the mouse and the corresponding stuff happens in some actionPerformed or mouseMoved method from the EDT. Perhaps the user action causes the window to scroll. Now, it’s a pain in the lower back to queue this event and execute it from the main thread instead of the EDT. So if it’s executed from the EDT then the window scrolls while the main thread is likely busy painting, possibly resulting in tearing because - well - perspective changes while the screen is updated.

I hope this was clear.

the difference between passive and active is that passive is wenn it’s told to/wenn it’s needed vs active which is all the time.

the whole deathlock issue has to do with swing not beeing thread save. and has inheritely nothing todo with the whole active of passive rendering. It’s the way we achive active rendering that causes the issues.

I tried Luke’s code. Since I was using active rendering before (yes, from within the EDT), that wasn’t really the issue. I was hoping for a performance gain. It turns out that frame rate is, on average, the same.

It may end my problem with dialogs being displayed oddly, but I’m not really concerned with that as I wrote my own almost-a-dialog class almost a year ago.

I also tried removing the NullRepaintManager code (which came from Developing Games in Java). There was no performance gain, but I was getting periodic NullExceptions for some reason. I was already randomly getting NullExceptions at start up once I started using Java 6 beta. But the NullExceptions were caught and had no effect except to add text to the error log. Since I then suspected that the problem was somehow solved by NullRepaintManager, I moved the NullRepaintManager to occur earlier in the code. Now there are no more NullExceptions at all.

I think the issue is that the regular Java RepaintManager was trying to draw Components that weren’t completely initialized after I went into full-screen exclusive mode but before I installed the NullRepaintManager. I could probably do away with the NullRepaintManager altogether if I made sure that Components weren’t made visible before their time, but it’s easier just to keep the NullRepaintManager.

Hi,

I was reading your postings since I wanted to use Swing as data input framework for my game so that I don’t need to develop one of my own.

Here I send you a little test program which open a dialog for text input and it seems to work OK. Don’t know if it represent the kind of problems you have with your rendering code.

I followed the active rendering examples from the book Killer Game Programming it seems to be deadlock free (can’t assure it though).

The problem I found with this is that it seems to me that the code is somewhat slow and the cube doesn’t get rendered solid but I shows some flickering. So I post the code to see if anyone has a better idea of why those things are happening.

Regards,
Gabriel


import java.awt.*;
import java.awt.event.*;
import javax.swing.*;

import com.sun.j3d.utils.timer.*;

public class ActiveSwingTest implements Runnable {

  static final int VEL = 1;

  // Number of frames with a delay of 0 ms before the animation thread yields
  // to other running threads.
  private static final int NO_DELAYS_PER_YIELD = 16;

  // no. of frames that can be skipped in any one animation loop
  // i.e the games state is updated but not rendered
  private static int MAX_FRAME_SKIPS = 5; // was 2;

  JFrame f;
  JPanel panel;
  Image backBuffer;

  JDialog dialog;

  private long gameStartTime;
  private long prevStatsTime;
  private boolean running;
  private Graphics2D graphics;
  private long period;
  private long framesSkipped = 0;

  int x = 0;
  int y = 0;
  int vx = VEL;
  int vy = VEL;

  public ActiveSwingTest() {
    initGraphics();
  }

  public void initGraphics() {
    panel = new JPanel();
    panel.setPreferredSize(new Dimension(800, 600));
    panel.setFocusable(true);
    panel.requestFocus();
    panel.setIgnoreRepaint(true);
    readyForTermination();

    f = new JFrame();
    f.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    f.getContentPane().add(panel);
    f.setIgnoreRepaint(true);

    f.setResizable(false);
    f.pack();

    backBuffer = createBackBuffer();
    if(backBuffer == null) {
      return;
    }

    f.setVisible(true);
  }

  public void run() {
    long timeDiff = 0;
    long overSleepTime = 0L;
    int noDelays = 0;
    long excess = 0L;

    gameStartTime = J3DTimer.getValue();
    prevStatsTime = gameStartTime;
    long beforeTime = gameStartTime;

    running = true;

    graphics = (Graphics2D) backBuffer.getGraphics();

    while(running) {
      update(timeDiff);
      render(graphics);

      paintScreen();

      long afterTime = J3DTimer.getValue();
      timeDiff = afterTime - beforeTime;
      long sleepTime = (period - timeDiff) - overSleepTime;

      if(sleepTime > 0) { // some time left in this cycle
        try {
          Thread.sleep(sleepTime / 1000000L); // nano -> ms
        }
        catch(InterruptedException ex) {}
        overSleepTime = (J3DTimer.getValue() - afterTime) - sleepTime;
      }
      else { // sleepTime <= 0; the frame took longer than the period
        excess -= sleepTime; // store excess time value
        overSleepTime = 0L;

        if(++noDelays >= NO_DELAYS_PER_YIELD) {
          Thread.yield(); // give another thread a chance to run
          noDelays = 0;
        }
      }

      beforeTime = J3DTimer.getValue();

      /* If frame animation is taking too long, update the game state
         without rendering it, to get the updates/sec nearer to
         the required FPS. */
      int skips = 0;
      while((excess > period) && (skips < MAX_FRAME_SKIPS)) {
        excess -= period;
        update(timeDiff); // update state but don't render
        skips++;
      }
      framesSkipped += skips;
    }

    System.exit(0); // so window disappears
  }

  private void showDialogo() {
    if ( dialog == null ) {
      dialog = new JDialog(f, "Example dialog", true);
      final JTextField t = new JTextField("hello");
      JButton bok = new JButton("OK");
      bok.addActionListener(new ActionListener() {
            public void actionPerformed(ActionEvent e) {
              System.out.println("text="+t.getText());
              dialog.setVisible(false);
              dialog.dispose();
              dialog = null;
            }
          });
      JButton bcancel = new JButton("Cancel");
      bcancel.addActionListener(new ActionListener() {
                public void actionPerformed(ActionEvent e) {
                  dialog.setVisible(false);
                }
              });

      final Container c = dialog.getContentPane();
      c.setLayout(new BorderLayout());
      c.add(t, BorderLayout.CENTER);

      final JPanel buttonPanel = new JPanel();

      buttonPanel.add(bok);
      buttonPanel.add(bcancel);

      c.add(buttonPanel, BorderLayout.PAGE_END);

      dialog.pack();
      dialog.setLocationRelativeTo(f);

      dialog.setVisible(true);
    }
    else {
      dialog.setVisible(true);
    }
  }

  private void paintScreen() {
    // use active rendering to put the buffered image on-screen
    try {
      final Graphics g = panel.getGraphics();
      if(g != null) {
        g.drawImage(backBuffer, 0, 0, null);
      }
      g.dispose();
    }
    catch(Exception e) {
      System.out.println("Graphics context error: " + e);
    }
  }

  private Image createBackBuffer() {
    final Image dbImage = panel.createImage(800, 600);
    if(dbImage == null) {
      System.out.println("could not create the backbuffer image!");
    }
    return dbImage;
  }

  private void readyForTermination() {
    panel.addKeyListener(new KeyAdapter() {
      // listen for esc, q, end, ctrl-c on the canvas to
      // allow a convenient exit from the full screen configuration
      public void keyPressed(KeyEvent e) {
        int keyCode = e.getKeyCode();
        if((keyCode == KeyEvent.VK_ESCAPE) || (keyCode == KeyEvent.VK_Q) ||
           (keyCode == KeyEvent.VK_END) ||
           ((keyCode == KeyEvent.VK_C) && e.isControlDown())) {
          running = false;
        }
        else if ( keyCode == KeyEvent.VK_D ) {
          showDialogo();
        }
      }
    });
  }

  private void update(long dt) {
    x += vx;
    y += vy;

    if ( x < 0 ) {
      x = 0;
      vx = VEL;
    }
    else if ( x > 700 ) {
      x = 700;
      vx = -VEL;
    }

    if ( y < 0 ) {
      y = 0;
      vy = VEL;
    }
    else if ( y > 500 ) {
      y = 500;
      vy = -VEL;
    }
  }

  private void render(Graphics2D g) {
    g.setColor(Color.RED);
    g.fillRect(0, 0, 800, 600);
    g.setColor(Color.WHITE);
    g.fillRect(x, y, 100, 100);
  }

  public static void main(String[] args) {
    ActiveSwingTest test = new ActiveSwingTest();
    new Thread(test).start();
  }

}

Try using the BufferStrategy class instead of drawing to your own BufferedImage and then drawing that to the screen. BufferStrategy can use page flipping if your graphics hardware supports it. And, if if it doesn’t support page flipping, I believe it can wait for the vertical retrace period (or whatever it’s called) to avoid drawing to the video memory while it’s being displayed to the monitor.

If you don’t want to do use BufferStrategy for some reason, at least try accelerating the BufferedImage using Image.setAccelerationPriority. That’s a bad idea though because Image.setAccelerationPriority just suggests that image acceleration occurs. So, you should really use BufferStrategy.

Just wanted to add to this to the thread:

http://weblogs.java.net/blog/enicholas/archive/2006/04/leaking_evil.html

Its a memory leak bug which would pop up as an OutOfMemoryError when the GraphicsConfiguration is changed. It popped up for me when I tried changing to full screen mode and back in the middle of the game. Its fixed in Mustang (Java 1.6).

Sun’s Scott Violet of the Swing team just gave a concrete response here:

http://forums.java.net/jive/thread.jspa?messageID=121654&#121654

"Swing isn’t thread safe, and is not meant to be used off the EDT. Trying
to do so otherwise will cause all sorts of problems.

-Scott"

java-gaming.org is pushing the boundaries as usual! 8)

I liked your final comment in the thread…

“If Swing should not be painted outside the EDT, then maybe The Java Tutorial http://java.sun.com/docs/books/tutorial/extra/fullscreen/rendering.html should be changed so that it didn’t contain…”

I’m actually glad that comment was in the tutorial or otherwise I wouldn’t have even tried to get Swing working :slight_smile: It gave me the impetus to investigate, and thanks to Luke for posting the SynchronizedEventQueue solution as well as everyone else’s help, it works!

For those looking for an example of all this, see:

http://www.java-gaming.org/forums/index.php?topic=15140.msg120376#msg120376

Keith

Invaluable thread #1 on this topic, thanks guys for putting this effort into it :slight_smile: