Active Rendering with Swing (swing rendering off the EDT)

Hi,

I can’t reply to the old thread http://www.java-gaming.org/forums/index.php?topic=13211.30 for some reason so I’ve started this one.

SInce Kova has mentioned that he’s interested in how all this works, here’s an example. It has 3 classes & shows how to (1) get Swing menus in your game 8) & (2) switch between windowed & full screen exclusive mode while in the game :D.

The reason why the code is quite long is that this is doing what is meant to be impossible - rendering Swing off the EDT. This is code that I put together myself, but which I owe all credit to Luke and David Brackeen who are responsible for the SynchronizedEventQueue & the NullRepaintManager. Copy the 3 classes into your IDE and compile & run Main.java. The code also works fine with the java2D OGL pipeline (except for a bug in java6 with OGL, but Chris Campbell is working on it ;)) & D3D pipeline.

Let me know if it works :P,
Keith

The example files are attached to this message just below this writing, just download them & rename the file so it ends in .java, note that you have to be logged in as a member to see the hyper-link to the files below…

grrrrrrrrr… this attachment of files is causing all kinds of problems
OK, the files show up just below this :wink:

Thank you Keith.
Hats off to Luke and David Brackeen, and a big no-no to swing team.
If anybody is interested in much simpler alternative, but requires subclassing every swing component you need (and overriding paint method), check out
http://www.java-gaming.org/forums/index.php?topic=15096.0

When just runing and closing app sometimes I get NullPointerException:


Exception in thread "Thread-2" java.lang.NullPointerException
	at java.awt.Component$BltBufferStrategy.show(Unknown Source)
	at Main$CustomPanel.render(Main.java:205)
	at Main$2.run(Main.java:80)
	at java.lang.Thread.run(Unknown Source)

and sometimes when switching to full screen mode I get:


Exception in thread "Thread-2" java.lang.IllegalStateException: Component must have a valid peer
	at java.awt.Component$FlipBufferStrategy.flip(Unknown Source)
	at java.awt.Component$FlipBufferStrategy.show(Unknown Source)
	at Main$CustomPanel.render(Main.java:205)
	at Main$2.run(Main.java:80)
	at java.lang.Thread.run(Unknown Source)

plus JFrame’s title bar is really screwed up, but I guess in active rendering you rearly use that

See the comment I put in Main’s close method. If you wait for rendering to finish before disposing the window then this shouldn’t happen.
and sometimes when switching to full screen mode I get:


Exception in thread "Thread-2" java.lang.IllegalStateException: Component must have a valid peer
	at java.awt.Component$FlipBufferStrategy.flip(Unknown Source)
	at java.awt.Component$FlipBufferStrategy.show(Unknown Source)
	at Main$CustomPanel.render(Main.java:205)
	at Main$2.run(Main.java:80)
	at java.lang.Thread.run(Unknown Source)

This is a bugger, I’m not sure how to get rid of it, perhaps the full screen mode code needs to be in a sync block sync’ed on SynchronizedEventQueue.MUTEX, but I think I tried that and still got the Exception. If you can figure it out, let me know :slight_smile:

Yeah, like I said in the comment in Main’s constructor when setUndecorated(false) is called, the title bar shouldn’t be used. I only included it to show that resizing the window works fine with this code.

I’m glad you like it, you should try setting the Look and Feel, that was the big plus for using Swing to me. Kirill’s Substance Look and Feel is the best one I’ve found. Also, since we paint the glass pane of the JFrame, you can add tooltips to buttons and they’ll paint properly. Cool, eh? :smiley:

Here’s the example as a Web Start link, mind you its not very exciting :

http://www.freewebs.com/commanderkeith/ActiveRenderingSwing.jnlp

It requires ‘all permissions’ to go into full screen mode.

Keith

ok I’m pretty much frustrated now… I’ve been trying to get it working all day now. I’ve also made a simple example and swing components draw fine but they don’t get any action listeners. JTextFields are not focusable, JButtons aren’t clickable and so on… they are just drawn. So please someone take a look at the code, it’s very similar to sample Keith gave, and tell me what’s wrong.

2 classes, each in it’s file, undecorated JFrame as fullscreen app, turn of with alt+f4

Viktorije.java:


/*
 * Created on 2006.10.21 by Kova
 */
package viktorije;

import java.awt.Dimension;
import java.awt.EventQueue;
import java.awt.Toolkit;

import javax.swing.JFrame;

public class Viktorije extends JFrame {

	public final static int DEFAULT_FPS = 60;
	
	public static int res_height;
	public static int res_width;
	
///////////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////////	
	
	// MAIN
	public static void main(String[] args) {
		int fps = DEFAULT_FPS;
	    long period = (long) 1000.0/fps;
	    System.out.println("fps: " + fps + "; period: " + period + " ms");
	    new Viktorije("Viktorije", period*1000000L);     // ms --> nanosecs
	} // end: main()
	
	// CONSTRUCTOR
	public Viktorije(String title, long period) {
		super(title);
		setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
		try{
			EventQueue.invokeAndWait(new Runnable(){
				public void run() {
					NullRepaintManager.install();	// this installs the NullRepaintManager on the AWT's Event Dispatch Thread
				}
			});
		} catch (InterruptedException e) {
			e.printStackTrace();
		} catch (java.lang.reflect.InvocationTargetException e) {
			e.printStackTrace();
		}
		SynchronizedEventQueue.use();		// this stops the AWT's Event Dispatch Thread from modifying the Swing menus at the same time we paint them.
		setIgnoreRepaint(true);
		setUndecorated(true);
		Dimension screen_dim = Toolkit.getDefaultToolkit().getScreenSize();
		res_width = (int)screen_dim.getWidth();
		res_height = (int)screen_dim.getHeight();
		setBounds(0,0,res_width,res_height);
		setResizable(false);

		setVisible(true);	// must be called before createBufferStrategy() or exception is thrown, don't know why
		synchronized(SynchronizedEventQueue.MUTEX){
			createBufferStrategy(2);
		}
		
		ViktorijePanel viktorije_panel = new ViktorijePanel(this, period);
		setContentPane(viktorije_panel);
		
		viktorije_panel.startAnim();		// Kova: starts game loop thread
	}

}

ViktorijePanel.java:


package viktorije;

import java.awt.*;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.image.BufferStrategy;

import javax.swing.JButton;
import javax.swing.JPanel;
import javax.swing.JTextField;

public class ViktorijePanel extends JPanel implements Runnable {
    
	private Thread animator;	
	
	private volatile boolean running = false;
	private boolean game_over = false;
	
	private GraphicsDevice device;
	private DisplayMode display_mode;
	private BufferStrategy buffer_strategy;
	private Viktorije viktorije;
	
///////////////////////////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////////////////////////

	// CONSTRUCTOR
	public ViktorijePanel(Viktorije viktorije, long period) {
		this.viktorije = viktorije;
		this.buffer_strategy = viktorije.getBufferStrategy();
		setIgnoreRepaint(true);
		setOpaque(true);
	    
	    setLayout(null);
	    JTextField tf = new JTextField("TEST ... use ALT+F4 to shutdown");
	    JButton btn = new JButton("press this to exit");
	    btn.setBounds(200,300,150,25);
//	    btn.addActionListener(new ActionListener() {		// dosen't work, but even if it works this shouldn't be neccessarry
//			public void actionPerformed(ActionEvent e) {
//				System.exit(0);
//			}
//	    });
	    tf.setBounds(200,200,200,20);
	    add(tf);
	    add(btn);
//	    viktorije.requestFocus();	// dosen't work
//	    this.requestFocus();		// also dosen'w work
	}  // end: constructor
	
	public void paint(Graphics g) {
		System.out.println("ViktorijePanel.paint(): this should rarely (best never) be called");
	}
	
	// Kova: start main game thread ("Viktorije Animator")
	public void startAnim() {
		if (animator == null || !running) {
	        animator = new Thread(this, "Viktorije Animator");
	        animator.start();
		}
	} // kraj: startAnim()

	// Kova: start main game thread
	public void run() {
	    running = true;
	    GraphicsEnvironment environment = GraphicsEnvironment.getLocalGraphicsEnvironment();
	    device = environment.getDefaultScreenDevice();
	    display_mode = new DisplayMode(1024,768,32,DisplayMode.REFRESH_RATE_UNKNOWN);
	    System.out.println("device.isFullScreenSupported() == " + device.isFullScreenSupported());

	    while(running) {
	    	gameUpdate();	// Kova: update game state
	        gameRender();   // Kova: render the game to a buffer
	        paintScreen();  // Kova: draw the buffer on-screen	        	
	        sleep();
	    }
	    
	} // end: run()

	private void sleep() {
		try {
			Thread.sleep(20);	// Kova: 50 fps
        } catch (InterruptedException e) {
        	e.printStackTrace();
        }
	}

	// Kova: update game state for 1 tick
	private void gameUpdate() {}

	private void gameRender() {
		Graphics2D g2d = null;
		synchronized (SynchronizedEventQueue.MUTEX){
			g2d = (Graphics2D)buffer_strategy.getDrawGraphics();	// CommanderKeith: must sync this or else there can be Exceptions thrown when resizing window (especially in OGL mode).
		}
		
		g2d.setColor(Color.GREEN);
	    g2d.fillRect(0, 0, Viktorije.res_width, Viktorije.res_height);		// Kova: clear the background

	    g2d.setColor(Color.white);
	    g2d.fillRect(0, 45, Viktorije.res_width, 6);	
   
	    g2d.setColor(Color.BLACK);
        g2d.drawString("say: ", 25, 725);

		synchronized (SynchronizedEventQueue.MUTEX){
			this.paintComponents(g2d);
			viktorije.getGlassPane().paint(g2d); // paint glass pane to paint tool tips	
		}
	}  // end of gameRender()

	private void paintScreen() {
	    try {
	    	buffer_strategy.show();
	        Toolkit.getDefaultToolkit().sync();			        // Sync the display on some systems (on Linux, this fixes event queue problems)
	    }
	    catch (Exception e) { // quite commonly seen at applet destruction
	    	System.out.println("Graphics error: " + e);
	    	e.printStackTrace();
	    }
	} // end of paintScreen()

	public void setFullScreen(boolean fullScreen) throws IllegalArgumentException {
		if (isFullScreen() == fullScreen) {
			return;				// CommanderKeith: we're already in fullscreen, so return.
		}
		if (fullScreen) {
			viktorije.setResizable(false);
			device.setFullScreenWindow(viktorije);
			if (display_mode != null && device.isDisplayChangeSupported()) {
				try {
					device.setDisplayMode(display_mode);
					System.out.println("display mode set");
				} catch (IllegalArgumentException e) {
					System.out.println("display mode not supported");
					device.setFullScreenWindow(null);
					viktorije.setResizable(true);
					throw e;
				}
			} else {
				device.setFullScreenWindow(null);
				viktorije.setResizable(true);
				throw new IllegalArgumentException("device.isDisplayChangeSupported() == false");
			}
		} else {
			device.setFullScreenWindow(null);
			viktorije.setResizable(true);
		}
		viktorije.createBufferStrategy(2);
		this.buffer_strategy = viktorije.getBufferStrategy();
	}
	
	public boolean isFullScreen() {
		if (device.getFullScreenWindow() == null){
			return false;
		}
		return true;
	}

}  // end: ViktorijePanel

Your inner panel ‘ViktorijePanel’ has zero width & height so it’s not reacting to anything.

Insert this in its constructor & it will work:
setSize(400,400);

Its funny that its child components still paint. I bet that the same problem would have happened without the active rendering code, its just that you would’ve seen nothing painted so the zero size problem would have been more apparent. Maybe try to get the GUI working with normal swing first and then transform it using the Active Rendering technique. Swing is tricky enough without hacks like what we’re using here :-.

Thanks for trying it out. Best wishes with your project,

Keith

Thanks a lot Keith, don’t believe I forgot that… components being painted throw me off since I’m not used to this new concept of manually painting swing. :slight_smile: Actually it’s not strange at all that they were painted since I was the one that called paintComponents() wich usually wouldn’t be called until they are visible I guess. It’s working fine, swing paints when I do it :slight_smile: … this also shows that paint methodes in swing do just that, paint, and deceiding wheter to paint or not isn’t included there but in some pre logic, which is logical :slight_smile: On the other hand this is one more thing I have to worry about, is a component visible so I can paint it. It will be pretty much simple in my game since it’s all absolute, but this will give headaches to someone using not so strict interface.

So you think paint methodes are the one that deceide to paint or not… hmm, maybe. I can’t really test GUI as normal first, becouse those are 2 different things of doing it, not comparable, so I don’t know what you mean that there would be same problem with non active rendering. It’s clear that if panel has no dimension that it wouldn’t be painted.

Thank you, to you also

One more thing Keith (and whoever used this), did you notice fps drop after you applied this technicque? On my faster computer I didn’t noticed since it’s always 60, which is max, but on my slower one fps dropped from 55 to 44, and that’s not so good, looks pretty choppy. I tested if it was changing from manually double buffering to buffer strategy, but it isn’t it, my test shows that buffer strategy should be faster.
I’m under impression that this is due to synchronizing before drawing.

Yeah, FPS did drop, but its not a big deal so long as you keep the Swing painting (the bit that’s synchronized) to a minimum. Also, cut down on how many layers of background Swing paints. I set the background of my JInternalFrame menu see-through, that way there’s no double-filling.

For my game I just have a little tiny ‘menu’ JButton which is painted in the corner of the screen on top of the rest of the game’s graphics. For big game menus like options dailogs, a slower FPS shouldn’t matter.

Also, the synchronized blocks will only significantly drag on performance when the lock is ‘in contention’ - ie when the sync’ed code has to actually wait for the other sync’ed code.

One work-around to having to sync at all would be too paint an image of the menuButton, and detect its presses using a normal MouseListener. Then when that’s pressed, paint the big sync’ed Swing menu on top of the game graphics.

Keith.

well that is the problem… I would understand if I had a big menu that I’m rendering in the game… but for now I don’t have a single swing component in animation :slight_smile: Just for main menu wich isn’t displayed during game animation of course.

This is how I do it: Viktorije is JFrame, ViktorijePanel is a JPanel. I put ViktorijePanel as content pane of Viktorije and set card layout to it (only one component visible at a time). Then I create few JPanels with swing components, each representing a menu or a sub menu of main menu, like host game submenu, connect to submenu, settings sub menu and so on. I add them all to ViktorijePanel and I add a animation JPanel also. When I want to display some menu, I use layout.show() to display adequate JPanel with swing components for that menu. That’s all I do. When game is running animation JPanel is displayed and all others are not visible. In animation panel I don’t have a single swing component, it’s basicly just an empty JPanel for now, so why the drop in fps?? I’ve put setOpaque(false) on both ViktorijePanel and animation JPanel. What am I missing here?

p.s. unrelated to the problem, but now when I render swing I can turn off double buffering for it? Does it hurt / have effect if I leave it on?

So do you still call:

synchronized (SynchronizedEventQueue.MUTEX){
this.paintComponents(g2d);
viktorije.getGlassPane().paint(g2d); // paint glass pane to paint tool tips
}

because you shouldn’t need to. When you don;t call this then I can’t see why the sync’ed Swing rendering could cause a slow down since there shouldn’t be any contention on the lock SynchronizedEventQueue.MUTEX since it will only be called by the EDT when you move the mouse & press keys.

PS: double-buffering swing as well as using you BufferStrategy is pointless- its just triple-buffering which can only make things slower.

yes I still call the synchronized paint() … but I expect that paint() returns immediatly since components aren’t visible… or am I wrong? When I commented out that part it went back to 56 fps :slight_smile: Thank you Keith once again, but what is swing good for if I can’t paint it? :slight_smile: This is killing my fps to bad to accept it, a 20% slower game just for few gui components is too too too much. And only if it was just slower, fps drops but there’s also a stall in rendering which is very visible so game jumps. And when I add incorectly (or not at all) rendered swing components (drop down menu with custom cell color renderer) on top of that, I think I will go back to passive rendering of swing and activly rendering of game animation, but with swing components rendered as I first did it, subclassed with overriden paint to paint to buffered image, and then draw that image when activly rendering.

…who would have guessed that synchronizing on a paint that probably returns immediatly is so time consuming :frowning:
btw. that computer that I was using for testing is a AMD Athlon 2600+ with Radeon 9000pro, so It’s not slow at all, It can run CoD2 with ~40 fps.

I think that what might be happening is that your CardLayout is being circumvented by the direct call to paintComponents(…) so maybe all layers are being painted, instead of just the one that’s meant to be visible, so check that. I’d avoid CardLayout, in my game I just add and remove components when the time comes.

Also, I have not been able to get Swing to revert from active to passive rendering yet, though I didn’t try very hard. Please let me know if you figure out a way to do it!

Thanks,
Keith

Funny thing… today as I started working on project as I commented synchronized part animation stayed at 44 fps. I clearly remember yesterday when reading your post that I just commented that sync part and it came back to 55 fps, and now it dosen’t work. I’ve been trying to get it to 55 again all day :slight_smile: This is all too much to debug so I’ll revert to revision before conversion and start all over again with new experience I got.

If that was the case I would see all of menues on top of each other, and fps would be much lower then, becouse if I draw even one submenu while rendering game animation fps drops to 30. Submenus are quite rich of options. I think paintComponents() is always called, no matter of layout, and then it’s layout deceides what to draw in those inner paint methodes. Note to all that this is controary to what I’ve said before here in reply #7, when I got components painted but they were not reacting :slight_smile: … ah is there someone who will understand swing ever? :slight_smile:
CardLayout is cool and I had no problems with it, one of few things in swing that works as I expected. I think it’s much more logical and simpler than adding/removing components every time you change gui.

well I’m not actually switching between active/passive, first I have swing main menu which is passive, and then when user starts the game, game uses active rendering, where I activly render swing components that I subclassed. When you want to go back to main menu (without quiting game loop thread) you pause your game and show (or add to visible container as you do it :wink: ) main menu which is still passively rendered.

Ah, great! So what is your actual code to un-active render Swing? I presume you call set setIgnoreRepaint(false); and uninstall the RepaintManager & SynchronizedEventQueue?

Thanks,
Keith

heh, you didn’t got it :wink: , let’s try this way: it’s all passive, all doing paints at os/app requests. The only difference is that I have my components where I overriden paint() to paint on my BufferedImage instead on screen, so when time comes for active rendering I render that BufferedImage. Here is an example of subclassed JTextField:


    static class BufferedJTextField extends JTextField {
    	BufferedImage image;	// Kova: every app triggered repaint store here and use this when doing active rendering
        int bi_width;
        int bi_height;

        public void paint(Graphics g) {
            synchronized (this) {
                int width = getWidth();
                int height = getHeight();
                if (width <= 0 || height <= 0) return;
                if (image == null || width != bi_width || height != bi_height) {                    
                	image = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);
                    bi_width = width;
                    bi_height = height;
                }
                Graphics g2 = image.createGraphics();
                try{
                    super.paint(g2);
                } finally {
                	g2.dispose();
                }
                g.drawImage(image, 0, 0, null);
            } // end: synchronized
        } // end: paint()

    } // end: BufferedJTextField class
....
....
chat_textfield = new BufferedJTextField();
...setbounds, addactionlistener, container.add...
....
....

in redering method:


        synchronized(chat_tf) { // Kova: drawing swing componets in active rendering...
            Image image = chat_tf.image;
            if(image != null) {
                g2d.drawImage(image, chat_tf.getX(), chat_tf.getY(), null);
            }
        }

the code is not mine actually, I just adjusted it. The idea came from leouser from JavaDesktop->Swing forums. I asked on newless clubbies about it and posted this solution, read here. I know someone posted on “pearls of wisdom” not to synchronize on swing components becouse something else also does it, don’t remember what exactly, but I’ll trust in leouser’s code for now since I’m under impression he mastered swing :slight_smile: Later I’ll check why is it sychronized, when I have some time for long, low gain researches.

to put this to a close: activly rendering swing didn’t slow my game down, but I still wont use it since it has problems rendering custom stuff. Don’t know why my game slow down, on 2nd (slower) computer. On my primary computer the new version with buffer strategy is ~2fps faster, and on slower computer it is ~10 fps slower, strange huh? :slight_smile:

I’ve been trying to adapt my game so that it can be an Applet but I think it’s imossible.

The problem is that Applets don’t have a BufferStrategy, unlike a Window. Usually you get around this by putting a Canvas in the Applet, but Canvas is not a Container so it can’t hold Swing components.

I’ve been bending over backwards to get Swing components working inside the Canvas (by adding both to a JComponent) and while I can get my menus to paint and take keyboard input, I can’t get them to take mouse input. The Canvas steels it and I can’t figure out how to get it to work, even if the Canvas is unfocusable. After searching this topic, apparently the heavyweight Canvas just doesn’t play with Swing.

So does anyone know a solution? Basically I need to get MouseInput from the Canvas and give it to Swing components.

Or is it the case that if you want to use a BufferStrategy with direct-rendering Swing in an Applet, forget it.

Thanks,
Keith

I was working on an applet not too long ago (it’s an application now). I didn’t do anything with BufferStrategy.

Instead, I just set up a normal Swing gui. Instead of having a Canvas for the main panel, I just overrode the paintComponent method of the JPanel for the main panel and drew all the game objects there. If you want some of them to appear on top of the Swing components, you’ll have to override the paint method. It’ll have to look something like this:

public void paint(final Graphics g) {
//draw layer under Swing components
[your code here]

//paint the components
super.paint(g);

//paint the top layer above the Swing components
[your code here]
}

I had a method like that in another game (which wasn’t an applet).

You can call setIgnoreRepaint(false) on your main frame and then just call the repaint method during your main loop to do active rendering. (The repaint method still works when the repaint is ignored - it just doesn’t seem to get called unless you call it directly.) I needed the windowActivated and windowDeactivated methods in the application version, but I don’t think I needed them in the applet version.