There is a well-specified way of creating a direct ByteBuffer which points to a given location in memory: the JNI entry point NewDirectByteBuffer(). This is what JOGL (and, more precisely, GlueGen) uses to interact with C structs necessary for the OpenGL and window system operations behind the scenes.
I am very reluctant to change this mechanism to use hacks like mutating a given ByteBuffer. The specification is very clear on what is and is not supported and I want the core JOGL and GlueGen code to be portable to all vendors’ JVMs.
Yeeeeh, what graphics card and OS are you running on? Have you tried running on another machine with a different graphics card? Are you running the latest version of your vendor’s graphics drivers?
It’s possible that the process size isn’t shrinking because some memory is being allocated on the C heap after the transient storage for the direct ByteBuffer is allocated, so even when the direct ByteBuffer’s storage is freed, the C runtime library can’t return the memory to the OS.
If you think there is a memory leak in the core JOGL code then please file a bug with the Issue Tracker on the JOGL home page and I’ll look into it, though this will have to wait until after JavaOne.