Dealing With Loading of Large Images

Java 2D
#1

Hey guys. This isn’t actually a question from a game, more a utility I made for work, but I figure you guys can answer it better than anywhere else.

I made a program to Load in a large series of images from a scanner or digital camera, name and classify them, save copies of the image in a specified size, and store the classifications into an excel document. Some of these images can get huge (up to 20mb), although I’ve been having trouble on images as small as 1 mb.

Basically, there is a JPanel on the left and the various fields you can fill in on the right. The program has the next image in the series loaded into an ImageIcon as a buffer so that you don’t need to wait for loading, then when you press the next button it displays the buffer to the current image and then advances the buffer.

Here’s my problem. I get OutOfMemoryError’s and other Java errors, because apparently it just can’t handle large images sometimes. I know that I can use -Xss to increase heap size, but I don’t want to do that. Instead I was wondering if I should perhaps use BufferedImage instead of ImageIcon, or resize a BufferedImage to be smaller while it is in the buffer, or even convert the whole thing to OpenGL because it is more efficient…

What’s a good solution for making things use less RAM? 64mb (the current heap size) should be more than enough, but apparently not. Any ideas?

#2

That large images won’t ever be accerlated in OpenGL so you won’t get any advantage.

The only thing I can think of would be scaling at loading-time, but I don’t know a java library which is capable of doing so.

lg Clemens

#3

A quick google search finds this:
http://forum.java.sun.com/thread.jspa?threadID=457266&messageID=2095583

#4

I looked at that link but it’s crazy stuff that confuses me. An 800k image would not load today, to me that’s a real issue with Java’s effectiveness. Having a few Swing components and about 2mb of images loaded should definitely not make the whole program fail, and there is absolutely no way for me to something like a try/catch to deal with an out of memory error.

And on Mac OS X there is no way to manually increase Java’s stack size, which is equally as stupid.

#5

Loading an image means uncompressing it into RAM - if you have 2mb highly compressed images then you may end up with a huge memory useage. Its no problem of java - doing the same with C++ would end up with the same problem.
Maybe you shoud learn howto create solutions instead of blaming others for not doing the things you would need in your programs.

Well, thats not Java’s nore Sun’s fault. Its apple’s implementation and as far as I know it is possible, at least it was with MRJ-2.?? back in the OS9.1 days.

Clemens

#6

I can’t understand, though, why the stack would be affected by this. Is some recursion going on somewere?

#7

Well the error I get is an OutOfMemory error and after looking around about it everyone said the solution was to increase the heap size. OutOfMemory errors comes from too many objects being instantiated on the heap. And no, I don’t believe I have any recursive calls.

Yes, I shouldn’t be blaming problems on anyone else, but the main reason I am annoyed is the fact that one of the great things about Mac OS X is that it’s very good at dynamically giving programs memory as they need them. This way I don’t have to worry about increasing program memory like I needed to in OS 9. This could be the reason I can’t manually increase Java’s stack size, but then again I shouldn’t have any problems if it was.

I am simply annoyed that I see so many programs with many many images displayed at once (including some of mine), and here I am putting only one on the screen, but Java can’t take it. So I’ll try to figure out parsing it all into bytes or something like that (what they did in that link).

Does anyone know if it would be possibly work better to set a JLabel with the image as its icon? Or is there any way at all to catch an OutOfMemory error?

#8

Wrong - OutOfMemoryError-s come from too little available space on the heap, so you need to enlarge the maximum heap size from 64mb to something you would like.

1.) Again, this has nothing to do with stack size.
2.) Are you joking or do you mean that serious what you are telling with that “giving programs memory as they need them”? Have you understood the way how java handles memory? Java allocates one big chunk of memory and does it memory handling itself instead of relying on OS features (thats why its so fast with small objects). Java limits the size of this chunk itself.
Furthermore I tried it on OSX-10.2 in PearPC and the Apple-JVM has such a switch.

Well, since the autors of the other programs did know howto create programs and it seems you don’t! Images in JLabels wondering, whats that?? Did you think about that the images in JLabels are not ment to be 2mb big? (typically icons)
If I would create a Carbon program and just loading huge images mis-using comfort-features of UI elements for displaying that images I would get exactly the same result.

The only thing you could do withought writing your own image-loaders is to use a scaled instance of your image.
Its not the question itself, everybody has questions from time to time - its the way you asked. You blamed others software for your faults, showed us that you do not really know what you’re talking about (that OpenGL or stack thing for example) and even after tries to explain howto handle it you furthermore said everything is java’s fault.
First learn howto code and after learning you can spread your FUD. First behave polite and listen!

Clemens

#9

Look, I’m sorry for blaming Java, but now you are insulting me, and I find that very rude. I will definitely admit I got pretty hotheaded, but I did not direct it at any of you, I was ranting about Java itself. Then for you to come back at me with personal insults I think is very uncalled for. Singal out what I was doing and I will stop it, but there is no reason for you to push the conflict. I did not even realize I was doing what I was doing, as many people do. I’m sorry, but don’t throw out any more insults, please. I have generally liked this community a lot up until now, but I am sure we can put this aside…

In any case –

  1. I meant heap size when I said stack size, sorry for the confusion. I’ve never had to worry about either before so I’m not an expert on the terminology.
  2. I just started using OpenGL very recently, and no, I don’t have much knowledge of it. That’s why I was asking if it would work.
  3. I was not using JLabels at all, I was asking if this might work better. I was using ImageIcons.
  4. I know very well how to use Java2D, enough to take a bunch of images and save new ones of different formats and size… I’m not incompetent.
  5. I have self-taught myself a lot of what I know, including BufferedImage’s, so anything I do wrong is because I was never taught it right. Maybe you can teach me instead of trying to put me down.
  6. I hadn’t encountered -Xss, etc. until now, so you’re right when you say I don’t understand it. However, I did read somewhere that -Xss isn’t supported on OS X, and indeed when I tried to use it saw no apparent changes at all. The function is there, but as far as I can see it does nothing.
  7. I was guessing when I said that Mac OS X can allocate memory. I wasn’t saying that’s how it is.

Phew. Hopefully I didn’t sound like a dick again. Again, I am very sorry about that.

I switched everything over to BufferedImage’s, then I resize the images as they are loaded into my image buffer. This takes some images as big as 3000x3000 and resizes them to about 500x500. I haven’t tested this method en masse yet, but I haven’t had any OutOfMemory errors so far.

My question now is whether or not this is worth doing. In order to resize the images, first I am loading in the whole image, creating scales of its width and height, creating a new buffered image with that scale, drawing the old image into the new one, then setting this new resized image as the new one. This makes it so that the images drawn to screen are much smaller, so no huge images are drawn, and I save memory between the currentImage and the buffer. However, I end up instantiating more memory for a short period of time.

Here’s an example. Say with the old method I have two 1mb images. There will theoretically be only 2mb of images in memory at a time. New method:
currentImage = 600k, buffer = 600k
That’s about half size most of the time, but while loading in another image:
currentImage = 600k, buffer = 1mb (temporary full size image loaded in) + 600k (resized image)
Is actually more memory at once. With lower numbers the new way would work better, with higher the old way. I’m not sure exactly how much memory I save when they are resized, but probably a good amount.

So, will this make a difference? Is it worth it? Should I change back? Is there possibly a better way?

#10

Peace :wink:

Yes, its definivly worth the work, the extra memory footprint which will enlarge for the short time you hold the original image and create a new one shouldn’t be critical at all. Have you had a look at the Image.getScaledInstace - you can archive quite pretty results when specifying the right image resizing properties.

Furthermore too large images can’t be cached in VRAM by Java2d so you may even get a performance boost when using smaller images.

Btw. don’t forget that images in memory tend to be a lot larger than stored compressed on disk, 3000x3000 BufferedImage/INT_ARGB (which is the default format as far as I know) weights heavy 36mb in RAM, so this would explain why you get close to the 64mb default heap size limit.

good luck, lg Clemens

#11

Wow, I definitely wasn’t aware of how much more memory images take when they are in the RAM, but it certainly makes sense. I’ll take a look at getScaledInstance – thanks a lot for the help.