Can a Swing GUI be used in a direct-rendering game?

Newbie & Debugging Questions
#1

Hi,

I just gave up on the roll-my-own GUI and converted to Swing - its what I know and its beautiful. I use direct rendering in my game loop thread and while I know that painting Swing/AWT components directly from such a thread is not safe, I read in a book somewhere that you can stop the AWT Event dispatching thread from painting by installing a ‘NullRepaintManager’, a class you make as a sub-class of RepaintManager that over-rides:

public void addInvalidComponent(JComponent c)
public void markComponentDirty(JComponent c)
public void addDirtyRegion(JComponent c)
public void paintDirtyRegions()

with empty method bodies. You install it as follows::

RepaintManager rm = new NullRepaintManager();
rm.setDoubleBufferingEnabled(false);
RepaintManager.setCurrentManager(rm);

Apparently that stops ‘OS-level’ paint requests but I’m not sure if it stops internal painting on the AWT Event thread. I did this to stop the AWT Event dispactching thread from painting so I can call aSwingComponent.paint(myGraphics) from my game thread without threading problems. It seemed to work fine at first, but sometimes my game freezes and I fear that its because there is a thread conflict. I assume its because things like JTextFields paint themselves (like the flashing caret) on some thread (the Event Dispatching thread?) which conflicts with the painting of that JTextField in my game loop thread.

How can I stop painting in the Event Dispatching thread for good or am I barking up the wrong tree?

Thanks,
Keith

#2

“Use the setIgnoreRepaint method on your application window and components to turn off all paint events dispatched from the operating system completely, since these may be called during inappropriate times, or worse, end up calling paint, which can lead to race conditions between the AWT event thread and your rendering loop.”

http://java.sun.com/docs/books/tutorial/extra/fullscreen/rendering.html

This is pretty m,uch vrigin territory though so expect some gotcha’s you’ll have to figure out yourself.

#3

Thanks Jeff, I’ll give it a go.

That article in the java tutorial is a relief because all other Sun documentation says that Swing components can’t be painted from any thread except the Event Dispatch thread…

Hopefully someone has used Swing in a direct-rendering game and can verify that it can be made to work.

If I do get it working I’ll post my experiences here.

Keith

#4

You can also use the invokeAndWait (or invokeLater) method of javax.swing.SwingUtilities, that will allow you execute code in the Event Dispatch Thread. If you execute all the game code there eventually then it’s much easier to have normal swing components as game controls, because you won’t have to queue and synchronize everything.

#5

If you let Swing/AWT render your game then
(a) You lose performance.
(b) You lose predictability (when a paint actually happens is some-random-measure-of-time-after-the-repaint-is-called)

In general hthis works okay for non-action games, but for action games whre frame rate and close timing are critical, Id recommend against such a technique.

#6

All the other articles youve read assume you are using Swing/AWT. If you were then yes, yo uwoudl have yo paint from the EVent thread because Swing is not thread safe.

In this case you are totally by passing those mechanisms and turnign off Swing/AWT rendering entirely so its safe to paint from antoher thread. You still shoudl only paint from one thread though. (in active rendered apps this is generally your own game thread.)

#7

I’m really confused about this… there’s conflicting info out there.

Although the AWT event thread can’t paint the Swing components because I turned that off, it can still modify the state of the components at the same time I paint them from my game thread.

For example, what if I paint a JButton from my game thread while its state is being changed in the AWT Event thread when the button is rolled over whith the mouse (or the JTextField is typed into, etc). So how can I possibly paint Swing components from my own thread??? I fear this is why my game freezes when I play around with the Swing menus I added.

Thanks,
Keith

#8

Painting does not modify model state so its perfectly safe as long as you dont re-enter the paint code.

I’ve never seen an issue with modifying the model from outside of the AWT threads. If you want to be sure though, ask the guys in the JavaDesktop forums.

#9

Great, I wasn’t aware of that. I wonder how that works because if I type delete (runs in AWT Event Thread) in a JTextField and remove a character mid-way through my game thread’s paint, you’d think that there’d be an ArrayOutOfBoundsException or the like thrown when the game thread tries to paint the character that no longer exists.

This hasn’t happened though, when I hold down delete in the JTextField after a short time my game thread just freezes and stops painting anything… Anyhow, I’ll make sure its not because I’m re-entering the paint code accidentally.

Cheers,
Keith

#10

Yep, Swing on JOGL works fine for me. Ignore repaints and repaint the swing parts afresh each frame.

#11

Thanks Dave, that’s good to know. Unfortunately my experiences are of deadlocking. And I just found these articles that its unsafe to use Swing at all unless its from the Event dispatch thread (EDT).

From a Sun developers blog site - http://weblogs.java.net/blog/alexfromsun/archive/2006/02/debugging_swing.html

“Initially there was a rule that it is safe to create and use Swing components until they are realized
but this rule is not valid any more, and now it is recommended to interact with Swing from EDT only”

And this on the Java Tutorial site - http://java.sun.com/docs/books/tutorial/uiswing/misc/threads.html

"To avoid the possibility of deadlock, you must take extreme care that Swing components and models are created, modified, and queried only from the event-dispatching thread. "

The above tell me that Swing cannot be directly-rendered (ie painted directly in a separate game thread). But this conflicts with the full-screen mode tutorial - http://java.sun.com/docs/books/tutorial/extra/fullscreen/rendering.html:

“Feel free to call Swing methods such as paintComponents, paintComponent, paintBorder, and paintChildren directly from your rendering loop”

For my Swing menus I have only had deadlocks a couple of times, but the JTextFields I use reliably cause AWT Event thread deadlocks when I type or delete characters. What kind of Swing components are you using in your game? Maybe it only works because you use simple components which rarely cause a dead-lock with your game thread.

I’d love to hear other developer’s experiences, surely not everyone has created their own self-rolled GUI. I assumed Swing was the default for game developers…

Cheers,
Keith

#12

I think I’m using virtually every Swing component known somewhere or other! Not everything works quite as expected (JOptionPane is a big no-no) but mostly it does. All my painting etc is done ultimately under the JOGL display () method which I suspect is in fact on the EDT.

#13

Given that MIke Martak, the author, was part of the AWT team at the time he wrote that article. Id personally trust him unless you find later stuff thats epcificly says this as broken.

Keep in mind that full-screen exclusive mode is an esoteric thing that very few people whow rite abotu Swing even know exists.

How are you using them? Id like to see the code as Im having trouble even iomaging a multi-trheaded access case to the model…

#14

If pressing a button or doing anything with the mouse - which all happens from the EDT UNLESS lots of work is done to requeue all the stuff to the game thread in every mouse listener and… jeez I’m dizzy now - affects the game state, and if on the other hand the game state might affect the model of a swing component (say, a thing dies and is removed from a list) then we have two threads accessing the model of a swing component.

I am surprised that this isn’t a common problem, which is why I made a thread (org.javagaming.Thread, not java.lang.Thread :slight_smile: ) about it a while ago, and it basically seemed that no one uses swing components for their games. I for one don’t want to write my own version of JTree so I use the approach mentioned by me above (SwingUtils.invokeAndWait to ensure logic update and active rendering from the EDT). Really, it’s the only solution I can imagine!

#15

I use Swing Components in direct-rendering games as well. In fact, I set up one Component for the whole screen that everything gets drawn to. I use regular Components as well. Dialogs are the only things that don’t seem to work.

I also have a NullRepaintManager and used setIgnoreRepaint to true. Here’s my code that actually draws the Components to the screen:

/**	shows the GameFrame*/
public void showFrame() {
	//initialization
	BufferStrategy bufferStrategy = mainFrame.getBufferStrategy();
	Graphics g = bufferStrategy.getDrawGraphics();

	if(!(g instanceof Graphics2D)) {
		ErrorLog.output(
			ErrorList.applicationGameFrameGraphics2DUnavailableInShowScreen);
		System.exit(0);
	} //end if we don't have a Graphics2D

	final Graphics2D g2 = (Graphics2D)g;
	g2.setRenderingHints(renderingHints);
	Insets insets = mainFrame.getInsets();
	g2.setTransform(AffineTransform.getTranslateInstance(insets.left, insets.top));

	//draw Swing components
	//wrap the drawing in an EventQueue if and only if this method isn't already in an
	//the AWT event thread.  Swing Components must be drawn in the AWT event thread, but
	//invokeAndWait can't be called from within the AWT event thread.
	final JLayeredPane layeredPane = mainFrame.getLayeredPane();
	try {
		//if this is in the AWT event thread, just paint the components
		if(EventQueue.isDispatchThread())
			layeredPane.paintComponents(g2);
		else { //else this isn't in the AWT event thread, so invokeAndWait
			EventQueue.invokeAndWait(new Runnable() {
				public void run() {layeredPane.paintComponents(g2);}
			});
		} //end else this isn't in the AWT event thread
	} catch(Exception exception) {
		ErrorLog.output(exception,
			ErrorList.applicationGameFrameErrorPaintingComponents);
	}

	//on Linux, this prevents tearing
	Toolkit.getDefaultToolkit().sync();

	//stop drawing and flip the buffer
	g.dispose();
	bufferStrategy.show();
} //end showFrame

I doubt the code will help you because you’re already doing the same thing. But I posted it anyways. It was originally inspired by similar code from a book called “Developing Games in Java”. My code’s “better” than the code in the book, but it’s the same general idea.

#16

Thanks everyone for the help.

Me too.

I tried to make a test case that produced the same problem but it’s working fine >> not deadlocking which I couldn’t understand. But now I realize that in my game the JTextField is in a movable JInternalFrame whilst in the test case it isn’t. Perhaps I’m having the same trouble as fletchergames since JInternalFrames are probably similar to Dialogs- “Dialogs are the only things that don’t seem to work”.

I’ll post the test case when I get home and have tried it with the JInternalFrame. Basically I create a frame and put compnents in it and just continuously paint them in the main thread. I manipulate the components with the mouse & keyboard (and this happens in the EDT which must sometimes conflict with the main thread, but doesn’t for my current test case…)

Yes! By David Brackeen, that’s the book where I got the code from. Thanks for posting your code.

The interesting thing about what you do is that you paint your Swing components in the EDT (through invokeAndWait) onto the BufferStrategy and then you show the BufferStrategy in your own thread. So you’re kind of using a weird combination of passive and active rendering. This should have the same draw back pointed out by Jeff earlier that (part of) the painting is done in the EDT - so your screen paints may come late while you wait for the EDT to do the job - not really direct/active rendering.

Since all of you - Dave, fletchergames and Ask_Hjorth_Larsen seem to only be having success with painting from the EDT then perhaps Swing cannot be used for non-JOGL direct-rendering games.

But I’ll look deeper into my test case (which is working, strangely enough) and I’ll post the code ASAP.

Keith

#17

From - http://www.clientjava.com/blog/2004/08/24/1093363375000.html

“The single-thread rule (from the Java Tutorial)
Once a Swing component has been realized, all code that might affect or depend on the state of that component should be executed in the event-dispatching thread.
…
Realized means that the component’s paint() method has been or might be called. A Swing component that’s a top-level window is realized by having one of these methods invoked on it: setVisible(true), show(), or (this might surprise you) pack().”

Since painting depends on the state of the component then painting from outside the Event Dispatch Thread (EDT) must not be thread-safe. Is this reasoning correct because it makes Swing useless for active-rendering and also contradicts what Jeff said in an earlier post.

Thanks,
Keith

PS:

I only mentioned the full screen mode tutorial because it prescribes calling setIgnoreRepaint() on components. I am not actuallly using full screen mode when doing direct-rendering with Swing.

#18

Right. hats one thread.

Where is the other one?

Its that which im having a hard time imgining.

#19

No.

In that context it means it has been ro might be called BY THE AWT PAINT tHREAD.

The problem with all these things you keep quoting is that they are ALL written with the asusmption that you are doing normal AWT/Swing repainting, not active rendering.

I’m not saying your wrong, because I don’t know those internals, but I think its more likely you are then you arent. I DO knwo that all these articles are orthogonal to the issue.

This is because the ONLY source on this option and what it really does under the hood is Mike Martak, who is no longer with us and thus cannot comment. Given that, I take his writings, which DO address manual painting, as definitive.

Question: have you gotten rid of all the WEIRD sh-t you were doing?

If not thats far more likely to be the cause of your problems.

Thats all I have to give on this subject and Im repeatign myself so I’ll stop here.

#20

Test case that deadlocks:

import java.awt.;
import javax.swing.;
import java.awt.image.;
import java.awt.geom.;

public class SwingDirectRendering extends JFrame{

public JPanel content;
public JTextField textField;

public SwingDirectRendering(){
    super("SwingDirectRendering Test");

	setIgnoreRepaint(true);			// disable OS-triggered repaint requests
	RepaintManager.currentManager(this).setDoubleBufferingEnabled(false);
	// disable double-buffering since we do this in the BufferStrategy

	content = new JPanel(new BorderLayout());
    setContentPane(content);

    textField = new JTextField("Type & delete stuff here", 50);

    JPanel panel = new JPanel(new BorderLayout());
    panel.add(textField);
    content.add(panel);
    //content.add(textField);	// comment out the above 3 lines and
    							// uncomment this and it will work -
    							// there will be no deadlock

	setUndecorated(true);
    setDefaultCloseOperation(EXIT_ON_CLOSE);
    setExtendedState(MAXIMIZED_BOTH);

	setVisible(true);

	createBufferStrategy(2);
}

public static void main(String[] arrghImAPirate){
	SwingDirectRendering frame = new SwingDirectRendering();

	while (true){
		Graphics g = frame.getBufferStrategy().getDrawGraphics();

		//Graphics g = frame.content.getGraphics()
		// to show that its not a BufferStrategy
		// problem you can uncomment the above and leave
		// out the BufferStrategy code

		frame.content.paintComponents(g);
		frame.getBufferStrategy().show();
	}
}

}

I’ve found that component.paintComponent(Graphics) works when its children have no child components themselves. When its child components have their own child components then deadlock results. You can see this by commenting out the intermediate JPanel that sits between the content pane and the JTextField.

Help appreciated :),
Keith