Autogenerated native stub code

Engines, Libraries and Tools Tools Discussion
#1

Hi everyone,

I’ve grown kinda annoyed having to write and recompile the native code for LWJGL for every platform we support (especially for Cas the lazy bum who doesn’t like to get his hands all dirty touching a linux box :-D). And we don’t even have readily access to a real Mac at the moment. So a crazy idea of mine was why not simply autocreate the native code? To do that, I think, we need:

  1. To create an class mapping C structs to an internal ByteBuffer using reflection something like the way Cas wanted the Structs API extension to java. That way, almost every traditional C function argument can be described by a primitive type, a Buffer object or a Struct (at least that’s what I hoped).

  2. To redesign all native parts of LWJGL to use SWT style. That is, have all (OS) native calls be simple and easily generated stubs at the native side and let platform specific java classes handle the work done in native code today. This is already done for OpenGL for obvious reasons, but the real platform dependent code needs a lot of work here.

  3. For each platform, find out how its dynamic libraries are built and then autocreate each and every native stub from the java declarations. Something like:

push the args
call the function pointer (fetched at some library init point)

  1. Now only the jar and openal will have to be distributed along with the game.

Obviously that’s a lot of work, and quite a few details are missing (like how to handle structs within structs? or C arrays that lack explicit length information). The work isn’t pressing, as it won’t introduce any public API changes, but I wonder if anybody had ever tried something like this before? I know almost nothing about dynamic library layouts and only a little (win32/SPARC) assembler, but that doesn’t scare me yet.

  • elias
#2

Have you considered using gluegen from Jogl ? I think it does exactly what you want. On the other hand, if LWJGL would start to use gluegen, it might be easier to just port rest of it on top of jogl :slight_smile:

#3

I’m guessing here, but isn’t gluegen “only” generating source code (and even that is from a gl.h file if I’m not mistaken)? I want to go all the way, creating the dll for System.loadLibrary to load.

  • elias
#4

Yes, it is created also from gl.h, but it is smart enough to handle many non-cryptic headers out there.

What do you mean by ‘only source’ ? It generates both java and C part, it is enough to run gcc with few parameters to turn all c files into dll.

#5

elias is saying he want’s to generate the required machine code for the stubs and write out the DLL, .so, etc. from Java. Since the calls are so simple this won’t need a C compiler… just something that pushes args and calls a method for whatever OS is supported.
It sounds cool, but I don’t know how practical it is.

#6

Maybe something like http://www.excelsior-usa.com/xfunction.html would work ? (it is commercial software, but it should be possible to write it yourself).

#7

The code which xFunction is based on has been publicly available as a JNI example (for x86 at least) for some time now, but it’s not quite what we want, although it’ll do what we need it to. Trouble is it created millions of objects as the primitive arguments have to be wrapped and that’ll never cut it in a performance situation.

What we’re looking at here is actually writing a .dll or .so or .dylib out on first run in the correct binary format for the platform.

A tall order, I suspect :frowning:

Cas :slight_smile: