float[] view of direct FloatBuffer + mapped objects (misery)

[Riven]

@princec

That’d depend on the GC algorithm being used. I believe the compacting GC will bugger it right up.

Perhaps you can pin the objects in JNI using getCritical thingies?

Cas

Could you tell me the vm-arguments to enable this compacting GC?

[pepijnve]

@Riven

Could you tell me the vm-arguments to enable this compacting GC?

http://www-128.ibm.com/developerworks/java/library/j-jtp11253/
Lists the different options in a table about halfway the article

[bienator]

@Riven

Could you tell me the vm-arguments to enable this compacting GC?

I think there are three (or four) GCs in Java 6
1 -XX:-UseSerialGC (this would be the default one on desktops)
2 -XX:-UseConcMarkSweepGC (i think it’s just for the young generation)
3 -XX:-UseParallelGC (young) -XX:-UseParallelOldGC (old and young generation)
4 -XX:-UseTrainGC (incremental one don’t know if it still exists)

I’ m not sure but I think 2 and 3 have a compacting part in there algorithm (I never read something about the first one), but If I where you i would try them all
And don’t forget -XX:+PrintGCDetails to trace if your App triggers full or just minor collections.

note: there are more options for the behavior of every collector, at least here are some of them: http://72.5.124.55/javase/technologies/hotspot/vmoptions.jsp

[Riven]

I extended the testcase, to create hundreds of millions of objects, in hundreds of iterations, and got a crash after a few minutes

oh well, was fun while it lasted

Anyway, the question to Ken remains

[zingbat]

@Riven

The only thing that would be required to make this work, is to make the GC never touch any object outside the heap, which seems to be quite easy to implement to me.

I guess that when the java vm becomes open-source there will be a tweaked java-vm project just to be used in java games.

[SluX]

I surely hope so…that is definitely needed if we want a more serious impact on industry…

Btw good job Riven.

[Riven]

The crash is weird though, because at most the relation between the object and the buffer would be gone, if the GC did anything, it should never crash…

[swpalmer]

@Ken Russell

You may want to look at the code GlueGen autogenerates on the Windows platform for JOGL. It creates Java classes which map to C structs. While the memory isn’t overlaid (it uses New I/O internally), the mapping between fields of C structs and methods in the Java classes is very straightforward and easy to deal with. Techniques such as this are legal and are what I’d recommend as an alternative.

Hey Ken, has any work been done on fully documenting GlueGen? I’ve never had the time to look into it but it sounds like something I could really use, as I need to do a lot of JNI work in my day job.
A tutorial on using GlueGen would make a great article for Java.net.

[Riven]

mapping methods to struct-values is dead-easy, just make a buffer backing it, and give the proper offsets. basicly a struct, without fields…

[kbr]

@Riven

While respecting (and accepting) your advice, in the supplied code there was a stress-test which showed the (default?) GC doesn’t copy the mapped objects’s pointers / references to heap-memory. Could you explain this behaviour?

Upon looking again at HotSpot’s serial mark/sweep collector, I think your fake Object references probably won’t get copied, since the relevant phases of the collector operate by iterating the address space of the old generation. However, your mapped objects break invariants in the collector and you would almost certainly see assertion failures if you ran with a debug HotSpot build. Also, if you ran with a different garbage collector such as the Concurrent Mark-Sweep (CMS) collector (-Xincgc) which performs card marks for more than just pointer stores in the heap, you would probably be back to crashing the JVM.

@Riven

Could you please tell me whether Sun is working (or: going to work) on mapped-objects?

There are no plans to do so. Unlike the C language, the Java language and VM spec do not specify the layout of objects in memory, so attempting to overlay them to arbitrary memory regions is meaningless and non-portable. There are ways of using annotations or tools like GlueGen to get Java object views on to C heap memory in a portable way. See the evaluation of bug ID 4820062 for a concrete proposal on how this might be done with no changes to the language or the JVM.

@Riven

The only thing that would be required to make this work, is to make the GC never touch any object outside the heap, which seems to be quite easy to implement to me.

Sorry, but it isn’t that easy. What is the desired data layout in memory? Right now as you’ve noticed you have somewhat meaningless object headers in between your Vec data, which probably makes it difficult to just send this data down to the graphics card. Would these objects support normal Java method dispatch rules? Synchronization? Do they support Object fields, or just fields of primitive type?

If all you want is to treat a large direct FloatBuffer as a sequence of Vecs, you can mimic this with New I/O and pure Java APIs. I think some of the JavaOne talk slides linked from the JOGL home page describe how you might make a VecCollection class which returns Vecs which are basically just integer indices into a backing FloatBuffer and whose getX()/setX() methods just make calls on the backing FloatBuffer. This is similar to what you’re doing but has the advantage of being portable and correct.

[kbr]

@swpalmer

Hey Ken, has any work been done on fully documenting GlueGen? I’ve never had the time to look into it but it sounds like something I could really use, as I need to do a lot of JNI work in my day job.

There’s a manual linked from the GlueGen home page which contains a fairly complete writeup of the functionality and also contains several small, concrete, and hopefully useful examples of how to use it. I’d be glad to hear any feedback, positive or negative, you have on it.

@swpalmer

A tutorial on using GlueGen would make a great article for Java.net.

That’s a good idea. I’ve put it on my list of things to do. Take a look at the examples in the GlueGen manual when you have a chance and see whether they are understandable or not.