How do I avoid accelerating a BufferedImage?

Java 2D
#1

I’m working on a game that needs the image-drawing to be accellerated to run in a reasonable amount of time. Most of the time, it works fine. But when I take a screenshot (for storing in a file to display in the load game screen), it slows way down. Presumably, what happens is that it accelerates the Images used to create the screenshot instead of the regular Images.

Is there some way to make it so that a BufferedImage will not be accellerated? I tried the setAccelerationPriority method, but it didn’t work (a priority of 0 should prevent acceleration). I also tried flushing the image acceleration data after I’m done changing it, but that didn’t work either.

Here’s my code for taking the screenshot:

public BufferedImage takeScreenshot(final int screenshotWidth,
final int screenshotHeight, final boolean bIncludeStatusBar)
{
//take a screenshot
BufferedImage fullScreenImage = new BufferedImage(
GuiUtil.getMainFrameWidth(), GuiUtil.getMainFrameHeight(),
BufferedImage.TYPE_INT_RGB);
fullScreenImage.setAccelerationPriority(0);
paint(fullScreenImage.createGraphics());

//figure out how much of the screen to use
int heightOfStartingImageToUse = bIncludeStatusBar ? GuiUtil.getMainFrameHeight() :
	yStatusBar;

//create the resized image
BufferedImage screenshotImage = new BufferedImage(screenshotWidth,
	screenshotHeight, BufferedImage.TYPE_INT_RGB);
screenshotImage.setAccelerationPriority(0);
Graphics2D g2 = screenshotImage.createGraphics();
g2.drawImage(fullScreenImage, 0, 0, screenshotWidth, screenshotHeight,
	0, 0, GuiUtil.getMainFrameWidth(), yStatusBar, null);
g2.dispose();

//flush all the screenshot image optimization data to avoid a slowdown when drawing
//the screen
fullScreenImage.flush();
screenshotImage.flush();

return screenshotImage;

} //end takeScreenshot

It occurs to me that just making the BufferedImages type ARGB might do the trick because translucent Images aren’t accelerated. However, I would like some better solution.

#2

Getting a raster will slow a BufferedImage dowb because now you have write access which means it cant use a VRAM buffered copy.

In general updating a managed image will take it out of VRAM until the system knows it isnt being updated at which point it will try to cache it agaon.

Reading abck from the image will also force it otu of VRAM because reading from VRAM is slow. Therefoir it wil lassume you want it in main memory.

If you want the exact rules I suggest you scan the Java2D forum. The guys who write all that cod ehave laid it out a number of times.

#3

Actually, setting acceleration priority to 0 should have done the trick.

You could do screenShotImage.getRaster().getDataBuffer() to make sure, but
I think there’s something else that’s causing the slowdown.

Note that BufferedImages are ‘accelerated’ in the sense that a cached copy
is created in vram only when you copy from this image (several times). Any rendering
you do to this image goes to the ‘primary’ surface which is still in the system memory.

Thanks,
Dmitri
Java2D Team

#4

I tried “screenShotImage.getRaster().getDataBuffer()” and some other things, but I’ve found no solution yet.

However, I have noticed that the following lines don’t cause the slowdown:

public BufferedImage takeScreenshot(final int screenshotWidth,
final int screenshotHeight, final boolean bIncludeStatusBar)
{
//take a screenshot
BufferedImage fullScreenImage = new BufferedImage(
GuiUtil.getMainFrameWidth(), GuiUtil.getMainFrameHeight(),
BufferedImage.TYPE_INT_RGB);

//create the resized image
BufferedImage screenshotImage = new BufferedImage(screenshotWidth,
screenshotHeight, BufferedImage.TYPE_INT_RGB);

return screenshotImage;
} //end takeScreenshot

But the following lines do:

public BufferedImage takeScreenshot(final int screenshotWidth,
final int screenshotHeight, final boolean bIncludeStatusBar)
{
//take a screenshot
BufferedImage fullScreenImage = new BufferedImage(
GuiUtil.getMainFrameWidth(), GuiUtil.getMainFrameHeight(),
BufferedImage.TYPE_INT_RGB);
paint(fullScreenImage.createGraphics());

//create the resized image
BufferedImage screenshotImage = new BufferedImage(screenshotWidth,
screenshotHeight, BufferedImage.TYPE_INT_RGB);

return screenshotImage;
} //end takeScreenshot

The paint method draws a number of BufferedImages (which must be accelerated) to the program’s main Component. I just sent the method the Graphics for the BufferedImage instead.

So, now I know not only that takeScreenshot causes the problem but also that it’s only caused when painting to a Graphics object other than the main component.

The BufferedImage isn’t accelerated, but the main Component’s buffer is. If an accelerated BufferedImage is drawn to a non-accelerated BufferedImage, does this cause the accelerated BufferedImage to become non-accelerated? I would thing it would have to because the accelerated BufferedImage is (presumably) stored in video memory instead or regular memory. That would explain everything.

So I tried changing “fullScreenImage.setAccelerationPriority(0);” to “fullScreenImage.setAccelerationPriority((float)1.0);”. It didn’t work, but that might not mean anything because I probably don’t have enough video RAM for an extra 800x600x16 image. And I can’t really be sure that that’s how the images are becoming non-accelerated.

I am reasonably sure that the images are becoming non-accelerated because the effect is pretty much exactly the same as an issue I had with non-accelerated images before. It also “magically” fixes itself when I switch to a different display within the game, which is the same thing that used to happen with the previous issue.

I still don’t know how to fix this though. It’s kind of frustrating because I wouldn’t even need BufferedImages in the takeScreenshot method except that I need something to paint to. I only need to end up with an array of ints representing the screen so that I can save it to file.

#5

Is there some way to reset the acceleration status for all images simultaneously? Maybe I could do something like that at the end of takeScreenshot to make the correct images get accelerated again. I wouldn’t mind a second or so of processor time if it meant that the screen would keep getting redrawn at the normal rate.

#6

I believe if there is no raster, and you draw the image, it gets cached down to VRAM again,. Ofcourse if you then take a raster or update the image you invalidate the raster.

Im not sure what happens with images you read from.

As I say if you read the 2D fourms there has been extension discussion of exactly what happens when.

#7

hmm, interesting one.

You don’t seem to be doing anything particularly crazy in the screenshot method, so i’d wager it is a combination of 2 factors :-

  1. Images being shunted out of vram when being written to an unaccelerated surface - this in itself doesn’t make any sense to me. It is my understanding that a managed image simultaneously exists in both system ram, and video ram. (it has to, video ram is volatile - on windows atleast, there is no guarantee of its persistence - consequently there has to be a copy kept in system memory to refresh the vram copy when its contents is lost)

Under normal circumstances, this bug would not likely be noticed - as the next time they were drawn to an accelerated surface, they would be immediately shunted back to video ram, and the problem would disappear.
However…

  1. As you have indicated you think you are already using most of your video ram. (do you know how much you are using, and how much is left? try querying GraphicsDevice.getAvailableAcceleratedMemory())
    It seems likely to me that the former bug maybe compounded with a shortage of video ram.
    As I assume video ram isn’t garbage collected in the same bullet-proof fashion as system memory. (nothing in the java2d api appears to be as bullet-proof as it should be)
    I’m wagering the attempt to reallocate some, or all of the images that were shunted out of vram, back into vram - is failing.

I’m mearly guessing though - I suggest you try and write a simple test case that demonstrates this bug, so you have some concrete proof.

p.s. it’d be very useful to know what JVM version you are using.

#8

If you guys take this to the Java 2D forum topic you are likely to get some answers from the folsk who actually wrote that code.

#9

If I switch to the main menu after the slowdown starts and then switch back to the regular part of the game, the problem disappears. If I stay in the regular part of the game, the problem continues indefinetly.

I printed out some stats about the available accelerated memory:

 [java] Accelerated memory (startup):153690080
 [java] Accelerated memory (after full screen frame is created):153161440
 [java] Accelerated memory (on title screen at start up):153173728
 [java] Accelerated memory (after the game is started):153147616
 [java] Accelerated memory (before screenshot):151595104
 [java] Accelerated memory (after screenshot):151595104
 [java] Accelerated memory (after resizing):151595104
 [java] Accelerated memory (after takeScreenshot method):151595104

I was somewhat baffled because I thought the amount would change drastically during the takeScreenshot method. Other runs of the program had small (about 2000 bytes) changes during the takeScreenshot method, but it’s clear that the 800x600 BufferedImage isn’t causing the 2000 byte change. It looks like I’m not using anywhere near as much accelerated memory as I have available.

I looked in Microsoft System Information and found that I have a Mobility Radeon 9000 with 32 MB of RAM. I’m fairly certain that that must be my video card. I thought I only had about 2 MB of video ram, but it seems that I was wrong. Apparently, running out of video RAM isn’t the problem.

The version of Java I’m using is 1.5.0_01. I’ll download and try version 1.5.0_06 to see what happens.

[Edit: Made the response more concise and more detailed.]

#10

The statistics I posted make it seem like nothing significant is being accelerated. That changes the whole situation. If nothing much is being accelerated, I have no idea what’s going on. I am using some Swing Components and a variety of strange custom stuff, so that might be causing the problem. Everything’s 2D though, so I’m surprised that there’s any problem at all.

As Anon666 said, I need to create a simple test case that causes the problem. Unfortunately, that’s difficult. The takeScreenshot method isn’t doing anything odd, but the drawing starts to take 5-6 times as long after it’s called (unless I switch to the main menu and back).

I tried the following hacks/tests:

  • using ARGB instead of RGB images in the takeScreenshot method in the hopes that they would then be ignored by image acceleration (since I have rendering set to use bitmask mode for alpha channel transparency this probably isn’t actually the case). There was no change.
  • removing acceleration from some images on the saved games screen. I was previously fairly certain that wasn’t the problem, and now I’m sure.
  • modifying my loadImage method so that all images would have an acceleration priority of 1.0 to force acceleration. There was no change.
  • modifying my loadImage method so that all images would have an acceleration priority of 0.0 to see if this broke the game even sooner. There was no change. This reinforced the idea that there may be some problem other than hardware accleration.
  • removing some dialogs on the saved games screen. I had some previous problems with slow downs after dialogs were displayed, but I had fixed that earlier. The dialog code is particularly suspicious because it’s my own custom code that doesn’t use the standard Swing JDialog class at all. Removing the dialogs had no effect.

I also checked the garbage collection (which I did previously). There are a couple of garbage collections right as the game is being saved (and the screen shot made), but it immediately goes back to the little mini-garbage collections that always happen.

As noted previously, the problem only occurs if I call the paint method to paint the screen to the BufferedImage. For this reason, I’m posting the code for the paint method being called:

/**	Paints this ViewPane.
	@param g the Graphics to paint to*/
public void paint(final Graphics g) {
	long screenDrawStartTime = System.nanoTime();

	if(!(g instanceof Graphics2D)) {
		Global.warning(ErrorList.viewPaneCanOnlyPaintToGraphics2D);
		System.exit(0);
	} //end if there's no Graphics2D

	//convert to Graphics2D
	Graphics2D g2 = (Graphics2D)g;

	//figure out the background color
	Color fillColor = getBackground();

	//if there's a background color, draw it
	if(fillColor != null) {
		g2.setColor(fillColor);
		g2.fillRect(0, 0, getWidth(), getHeight());
	} //end if there's a background color

	//paint noncomponents and Components in proper order
	paintNoncomponents(g2);
	paintComponents(g2);
	paintNoncomponentsOnTopLayer(g2);

	//display the ms required to update the screen
	if(!isDisplayingADialog(false)) {
		g2.setColor(GuiUtil.getForegroundColor());
		g2.setFont(GuiUtil.smallFont);
		g2.drawString("Screen Draw Time: " +
			((System.nanoTime() - screenDrawStartTime) / 1000) + " ms",
			xTileDisplayArea, yTileDisplayArea + GuiUtil.smallFontSize);
	} //end if this ViewPane is displaying a dialog
} //end paint

I don’t think posting that method will be particularly useful since it doesn’t do much besides call other methods. However, I figured I’d post it and see if anything jumps out at anyone. Although it’s not evident from this code, I’m using active rendering - the Swing Components are drawn when I tell them to get drawn, not when Swing feels like repainting. I imagine that sort of thing is pretty much expected on the games forum, but I mentioned it so that passive rendering won’t be suggested as the possible problem.

I think I’m going to fix an issue with having some tiles overlapping on the screen tomorrow. It’s not really a bug because I planned it that way, but it is something that I want to get rid of. That should speed up the screen drawing as a whole (or, even if it doesn’t, the screen will look a little bit better), and I’ll see how that affects the bug.

#11

Aha! I tried something I saw in another thread, and the game worked normally. (It didn’t help the guy in the other thread, but it did help me.)

I added the following argument to the JVM: -Dsun.java2d.d3d=false. The problem was completely gone, and the screen actually gets drawn slightly faster the rest of the time too.

Unfortunately, I have no idea what this means. And I assume that this is a fix specific to my computer and the JVM I’m using. It might be that setting the flag the same way on another computer would actually make things slower.

Anyone know what’s going on here? I have no idea what that flag does except that “d3d” probably stands for Direct 3D.

#12

hmm, an odd amount of graphics memory, do you have an onboard graphics card that is sharing system memory?

If so, i’d wager that has something to do with the performance anomalies you are experiencing.

#13

I have an onboard graphics card, but I wasn’t away that it could share system memory. I suppose that must be what’s going on because it wouldn’t make sense to have a video card with that much RAM. I’m afraid I have to admit that I’m a little bit incompetent when it comes to hardware.

Also, I downloaded Java version 1.5.0_06, and it made no difference.

If the graphics card is the problem, I suppose the game will actually work fine for most people. Still, I want to find some way around it if I can. If I can’t, I could just stop take screenshots for the saved games - it’s a standard feature but far from essential.

Adding the argument “-Dsun.java2d.d3d=false” to the JVM fixes the problem, but that’s a non-standard argument that will have different effects on different computers. Therefore, it’s not really an option.