Garbage collection of method return values

Performance Tuning
#1

I had a slow memory leak in an app, that I tracked down to Vector3D methods that were instantiating and returning new Vector3D objects (code below). I thought this was legal, as the new object exists on the function stack and will not be referenced once a calculation is complete. The profiler showed that these methods were producing stray Vector3D objects that weren’t being garbage collected:

	public Vector3D vectorProduct(Vector3D a, Vector3D b)
	{
		return new Vector3D(a.y*b.z-b.y*a.z, a.z*b.x-b.z*a.x, a.x*b.y-b.x*a.y);
	}

	public Vector3D getNormal(Vector3D a, Vector3D b)
	{
		return vectorProduct(a,b).normalize();
	}

The vectorProduct() method returns a new Vector3D, so the math operations can be chained as in getNormal().

I fixed this easily enough by instantiating Vector3D objects to hold answers and then reusing them instead of creating new ones for each calculation. Still I was surprised that this approach leaked memory. Is this bad form, or am I missing something else?

#2

From my experience, your code should not cause memory leak.
I have very similar code which is not causing any problem.

Did you performed a gc before comparing your reference count in your profiler ?

I mean someting like :

System.gc();
<-- First reference count here
test();
System.gc();
<-- Next reference count here

Which VM are you using ? With which profiler ?

#3

Be cleaerer. There is no memory leak possible here. Thats what GC is all about

Do you mean you have an Object leak? (That is where you retain a reference to an Object youa re un aware of and thus repvent it from being collected) A profiler can tell you what references to a given object stil lexist.

Do you just mean that you are creating garbage? That si not a leak at all, thats just garbae creation and, when the VM needs to, it will reclaim the memory.

Are you expecting that memory is collecetd the minute all refernces are gone? This is not the case. An object becomes a candidate for colletion when al lreferences are gone but when if ever, it gets collected is up to the VM. The only gaurantee is that the VM will make a “best effort” to collect enough memory to perform a requested allocation before erroring out with an Out Of Memory error.

#4

Thanks for the replies. I’m looking into any stray references to these objects lurking somewhere. Nothing so far…

And it could be that garbage collection just hasn’t been called yet, so the garbage is just lying around to be collected later. I’m not forcing a gc(). However if I leave the app running for about 24 hours it will eventually crash with a windows error message about running out of virtual memory, which implies that this garbage can’t be collected. for some reason

To profile I just ran java -Xrunhprof with the VM 1.4.1_02.

#5

I might be totally off track here but… by default the VM can only use a fixed amount of memory (64Mb). The only way I could see a proper windows ‘no virtual memory’ message would be that you’re manually setting the VM to use huge amounts of memory and your page file can’t cope with all of it (or, similarly, you have a ridiculously small pagefile). Assuming that’s not the case (as it seems highly unlikely) a proper memory leak would eventually throw a Java OutOfMemory exception instead.

So I’d guess you’ve either got a memory use from some funky use of native resources (or native code), or something else running in the background is eating up the memory rather than your own app. You did check the memory used by the processes when you got the out of memory error didn’t you? And you’re sure it’s a java process and it is indeed taking up obscene amounts of memory?

#6
Using Perl (haven't tried it with any other language), if you create an obscene amount of threads, each generating alot of garbage, win XP will try and send off your page file through the the internet, over MSN, Yahoo and IRC. Quite funny to watch...

Be warned of Arraylists, Hashmaps and that sort of thing, they store references to the Vector3f generated by the product. If your storing a list of those somewhere, GC cant kick in as the object is still being referenced. Otherwise, i would go with that Orangy Tang said.

DP

#7

Orangy is absolutely right. This in fact is WHY the VM has a fixed heap size, so it wont take a whole system down in the event it uses up all its memory.

It indeed sounds like you need to look at the native level. Either you are calling a native library that has a leak, or something else in your system is swollowing memory.

(One more VERY unlikely thing is that you are using up some system resource, like opening sockets over and over and never closing them, or strating new threads and never ending them, and this is the operating system’s eventual response. That would be very bad OS programming, but this IS Win32, yes? )

#8

Thanks to all for the good suggestions.

Orangy Tang was on the money. I was running the app with an old batch file that set the VM memory min and max to 360. The machine I’m testing on has 256 Mb ram. Still, the page file size is set to 512, so I thought it would be able to handle it. The app doesn’t really need much memory, it’s a fairly simple game loop. No threads or sockets.

I’m testing again with the VM memory set to 128 Mb to see if that fixes it.

Also I’m using LWJGL. It’s possible I’m leaving stray NIO buffers around. I’ll look into that too.

BTW this is Win XP, which may account for some flakiness.

#9

When writing such math methods, I prefer doing something like this:


public Vector3D crossProd(Vector3D v){
  this.x = blahblah
  this.y = ...
  this.z = ...
  return this;
}

That way you can write


someVector.crossProd(otherVect).innerProd(thirdVector);

and so on. This will of course be stored in someVector, but you could create a clone method to create new vectors when you want, thus controlling the amount of garbage.

#10

That works fine. It doesnt need RAM, just virtual memory.

Dont forget the other one, one of my personal gripes against Sun’s JVM - it doesnt close file handles / file input/output streams (IIRC it closes one type, but not the other) when GC’ing. Even with 1.4 (IIRC) and hence probably 5, you still need to manually out.close() and in.close() because one of thsoe wont be done for you even if its clearly impossible for you to access the thing any more.

#11

This now looks like an odder problem than I thought, maybe a Windows problem. My java app now instantiates objects only on load, then reuses, so there’s little/no gc happening. Windows Task manager shows java.exe using 5-6 Mb steadily, and no other app is consuming memory.

The odd thing is that Task Manager shows PF Usage steadily increasing as the app runs. It starts at about 150 Mb and increases over 24-36 hours to over 800 Mb at which point it’s close to the top of the bar chart display (the page file is 512 Mb).

I notice that when I interact with the piece at this point it freezes up for .5 second every thirty seconds or so. Those freezes seem to correspond to page file activity (Windows Performance Monitor shows periodic spikes in page file usage that look like they coincide with the freezes that I see in the app).

Twice the app has crashed out to the OS after 24 hours or so of running.

Can anybody point me to a lucid explanation of what the “PF Usage” display means in the Windows Task Manager? It doesn’t seem to have any relationship to the actual size of the page file.

#12

Have you already profiled you app - it could be that you simply have a memory leak in your code - so theres nothing the GC can do dor you.

#13

I think you should really profile your app.

I suggest using “Reference scanner” http://jb2works.com/index.html. It is easy to use, works great for this type of problem and has a free downloadable version.

Another suggestion would be to search the native libraries you are using. For example, JOGL team detected a bug in an ATI driver that causes exactly the symptoms you are talking about (memory slowly increasing, just like a memory leak).

Good luck

         Vincent
#14

Thanks everyone for the good suggestions.

I’m not sure if it’s good news or bad news, but the problem only occurs on this one machine, with an ATI 9700. I"m pursuing Niwak’s tip about the ATI driver bug. I reduced my code down to a single textured quad and still see a steady increase in app memory usage with the ATI, but with Nvidia I have no problem with any of my apps.

Let’s see what a driver upgrade does.

#15

Then consider your problem solved ;

http://192.18.37.44/forums/index.php?topic=9865.0

A workaround to this driver bug is provided.

Hope it helps

      Vincent