ImageIO Heap 'Splosions

[Demonpants]

Hey all.

I’ve got an app that needs to do this:

1. Load a 2048x2048 image into memory.
2. Copy that image onto a total of 16 512x512 tiles of that image (which are also images).
3. Garbage collect #1.
4. So numbers 1 and 2 a total of 4 times, each time deleting 1 but keeping all the results of 2.

This means that, at the most, there will be BufferedImage’s totaling:
1 2048x2048 TYPE_INT_ARGB
64 512x512 TYPE_INT_ARGB

That’s a lot. I’ve jacked up the heap size to 1024 and it still can’t handle anything past the second pass (it craps out at 1 2048x2048 and 21 512x512).

Any ideas on how I can actually make this happen? If I use a different BufferedImage type will that be cheaper? These are background images, so I won’t really be needing TYPE_INT_ARGB.

Thanks.

[zammbi]

I use to load up a 256x15556 image and break that down into 32x32 tiles. Ran under 128 megs of ram just fine.
Maybe if you post your code we can improve on it.

[Mr_Gol]

@Demonpants

3. Garbage collect #1.

How do you know it’s really garbage collected? I’ve had some issues in the past with BufferedImages that basically refused to be garbage collected (possibly because they were still being hardware accelerated and as such still referenced somewhere internally in Java 2D, but that’s just guessing).

[Riven]

I work with tens of thousands of 3000x2500 px images, inside a single process.

It is very easy to make them cleanup after themselves:

bufimg.flush();

Failing to call that, and you’ll run out of heap pretty soon.

[Riven]

Also… bufimg.getSubimage(); returns actually a view on the big image. The big image will never be freed, as long as the views are around.

(you can draw in the sub image, and it will be visible in the big image)

Adding to that:

2048x2048xRGBA = 16M
16x 512x512xRGBA = 16MB (obviously)

even if you repeat this 4 times, without freeing any data, after 4 iterations, you’d only use 128MB, so it makes no sense that your heap of 1024MB can’t handle it…?

[Demonpants]

Here’s the code. I wasn’t at all sure the garbage collector was taking the image away - it probably wasn’t. Just for reference, bFiles has length of 4 and just points to 4 different PNG images that are 2048x2048.

I’m guessing that using getGraphics() and drawImage() within that is not just taking the part within bounds, and is somehow storing the stuff outside of it, so I end up with like 25 2048x2048? I know there are lots of better ways to do this, but I’ve got a deadline with this thing so I’ve written it all quick n’ dirty. Also, this is one part of a big UI, in which there are probably like 50 icon-sized (32x32 or 64x64) images loaded and 20 larger images (256x256) loaded.

BufferedImage b = ImageIO.read(bFiles[0]);
			
width = b.getWidth();
height = b.getHeight();
float cols = width / BACKGROUND_TILE_SIZE;
float rows = height / BACKGROUND_TILE_SIZE;
	
background = new BufferedImage[bFiles.length][(int)Math.ceil(cols)][(int)Math.ceil(rows)];
setPreferredSize(new Dimension((int)width,(int)height));
			
for (int i = 0; i < bFiles.length; i++)
{
	if (b == null)
	{
		b = ImageIO.read(bFiles[i]);
	}
	
	for (int x = 0; x < cols; x++)
	{
		for (int y = 0; y < rows; y++)
		{
			background[i][x][y] = new BufferedImage(BACKGROUND_TILE_SIZE, BACKGROUND_TILE_SIZE, BufferedImage.TYPE_INT_ARGB);

			for (int bx = 0; bx < BACKGROUND_TILE_SIZE; bx++)
			{
				for (int by = 0; by < BACKGROUND_TILE_SIZE; by++)
				{
					background[i][x][y].setRGB(bx, by, b.getRGB(x*BACKGROUND_TILE_SIZE+bx, y*BACKGROUND_TILE_SIZE+by));
				}
			}
		}
	}
	
	b.flush();
	b = null;
}

Also, calling flush() doesn’t appear to change anything (it still breaks at the same point).

[Demonpants]

Changed my code to set the RGB values directly and not only was it significantly faster but it also made it further before it crapped out. Nevertheless, it broke after almost 3 iterations (so somewhere around 45 512x512 had been loaded, plus 1 2048x2048).

Also, I’m using Java 5 if that makes a difference. And I edited my code above to match the pixel setting method instead of using Graphics.

And using TYPE_INT_RGB doesn’t help any.

[lhkbob]

I took your code and ran it in a runnable test case. When I used Java 6 on my Mac, it ran to completion and the heap size of the jvm was 80.5 Mb (reported memory usage was 79.33 M). However when I tested it with the default jvm, it crashed after using about 62 M. When I increased the heap size to 128m, Java 5 ran it to completion as well, although its reported heap usage was 80.6 out of 127.06 M.

I didn’t modify your code, so I’m not sure why it’s failing on your computer, unless integration with the UI is consuming a lot more resources??

[Demonpants]

Well, ImageIcon’s might be really expensive, I don’t know. But all those icons and things previously mentioned exist as ImageIcon’s.

[Demonpants]

Now this is really ticking me off. I commented out almost all the UI elements getting initialized and I get just a few more tiles loaded before it gets a heap error. This is seriously irritating. I can’t believe how inefficient BufferedImage’s are… I’ve almost always had problems with them.

Also, this is probably a noob question, but how are you seeing how much heap is being used?

The problem you encountered is typically a JAVA problem.
The mechanism that processes images doesn’t release the used memory
what ever you do.

Some month ago I submitted almost the same question and the solution
I found myself was to spawn an extra program to do the ‘dirty’ work
and exit then (to release the memory).

I don’t want to do that. Please tell me I don’t have to goddam do that. grumble what a waste of my already short time…

@zammbi

I use to load up a 256x15556 image and break that down into 32x32 tiles. Ran under 128 megs of ram just fine.
Maybe if you post your code we can improve on it.

Also, 256x15556 < 2048x2048.

[Riven]

using getRGB and setRGB in images is about as slow as it gets.

why don’t you simply draw your sub regions with

bufimg.getGraphics().drawImage(img, a-heck-of-a-lot-parameters, bgcolor);

That is more efficient than drawImage with a negative translate and clipping by image bounds.

[Demonpants]

@Riven

using getRGB and setRGB in images is about as slow as it gets.

why don’t you simply draw your sub regions with

bufimg.getGraphics().drawImage(img, a-heck-of-a-lot-parameters, bgcolor);

That is more efficient than drawImage with a negative translate and clipping by image bounds.

Because I’m doing it quick and dirty. I just want to this bastard heap crap to work at this point. It looks like I either try giving myself 2gb of heap, I use something like JMagick to load the images, or I run an external process. None of which are at all optimal, all of which make me angry that I need to do that. In my opinion, this should be a pretty simple task…

[EDIT]
As I don’t need all the images on screen at once (the reason there are 4 is because they correspond to 90º rotations) and because this is a level editor and therefore doesn’t need to be fast, I turned the background array into a 2D array and I only store the current rotation, the rest are split up as per above when you load them, but those splits are saved as temporary files ( file.deleteOnExit() ), then when you press the rotate button it reloads those images back into memory after unloading the old ones.

Quick n’ dirty, quick n’ dirty.

[EDIT2]
Riven, I tried your way versus setRGB(), here are time comparisons for loading, splitting, and writing 4 2048x2048 images into 64 512x512 images. The total is how long it took start to finish to do everything, including redundant stuff that has nothing to do with the different methods of copying pixels. The average is the average time it took to copy one 512x512 chunk of pixels into the new BufferedImage (over all 64 iterations).

Your way:
Total: 11024 ms
Average: 42599984 ns

setRBG:
Total: 11177 ms
Average: 70706406 ns

getRaster().setPixel():
Total: 11352
Average: 55170890 ns

[Demonpants]

I was curious, so now I’m doing the whole thing 100 times (over 10 minutes), minus all the excess stuff. This is just pure pixel copying. A total of 1.67 billion pixels are copied. The average is how long it took, on average, to copy 260,000 pixels.

Using drawImage:
Total: 277386942000 ns
Avg: 39149537 ns

Using setRGB:
Total: 428779703000 ns
Avg: 66407857 ns

Using getRaster().setPixel() with a temporary int[] array:
Total: 332476529000
Avg: 51352468

Using getRaster().setPixel() without a temporary int[] array:
Total: 347614522000
Avg: 53646431

[lhkbob]

@Demonpants

Also, this is probably a noob question, but how are you seeing how much heap is being used?

I used:

Runtime run = Runtime.getRuntime();
long totalBytes = run.totalMemory();
long usedBytes = totalBytes - run.freeMemory();
System.out.printf("%.2f M / %.2f M\n", usedBytes / (1024f * 1024f), totalBytes / (1024f * 1024f));

[Demonpants]

@lhkbob

I used:

Runtime run = Runtime.getRuntime();
long totalBytes = run.totalMemory();
long usedBytes = totalBytes - run.freeMemory();
System.out.printf("%.2f M / %.2f M\n", usedBytes / (1024f * 1024f), totalBytes / (1024f * 1024f));

Thanks.

[Riven]

If you’re going to keep all 16 versions in RAM, simply do img.getSubImage… like said, then you’ll share the pixel data, no datacopy at all.

It will cut your memory usage in half, and boost runtime performance.

[Demonpants]

@Riven

If you’re going to keep all 16 versions in RAM, simply do img.getSubImage… like said, then you’ll share the pixel data, no datacopy at all.

It will cut your memory usage in half, and boost runtime performance.

They’re different images, though. I’ve got no redundant duplicates of pixels except for the brief moment when the big image is in memory so that it can be split into tiles.