Jack, Java bytecode -> C++ [Updated to be more descriptive]

I spent the night refactoring the monolithic mess i created over the last couple of days into a mostly sane code base. If someone wants to join the effort let me know. The code should be quite straight forward for the most part. The following things could be areas a contributor can work on:

exceptions should be the simplest task, followed by basic reflection, CNI and threading. Another area i’m sure nobody will want to help out is porting unit tests from other VMs over, e.g. Avian’s small test suite.

I updated the README with instructions and a basic code overview, see https://github.com/badlogic/jack/blob/master/README.md

You are right, it is simpler. You don’t need to implement any interface. The Json class in libgdx or the jsonbeans lib outside libgdx is also simpler, just give it POJOs, done. But I understand change is scary, you know, for old people. :slight_smile:

(Split the topic to avoid Jack thread derailment)

Cas :slight_smile:

Having better GC is quite problem with this simple approach of cross-compiling to other language. For fast accurate GC you need cooperation with the compiler so you know the format of stack and register usage with regard to references (by having extra info about these on certain code points).

You can sidestep that by having second (shadow) stack used just for references where you know exactly it’s format, for references held in registers it would be sufficient to just maintain the reference on the second stack (can be little imprecise, but never having reference in register that isn’t in the stack) while it’s used in registers. This incurs some speed penalty, but is probably the fastest solution.

Other alternative is to use reference counting with cycle detector (could be called manually at “safe” points in application). This would be slowest, esp. because of need of atomic updates for the reference counter for multithread usage.

Maybe some semi-manual approach (similar to real time java) would be preferrable instead of GC.

The problem with a shadow stack is, that if i have a copying GC, i still need to reset the registers, as they’d contained the old address of the object. So, whatever i do, i have to have some platform specific code that checks/rewrites the registers as well. Adding in multi-core, this can get really messy.

Another approach would be to scan the stack and registers conservatively (scanning the stack is almost simple, scanning the registers could be done by dumping them into a jmp_buff via setjmp). The GC would then only rewrite objects that are not on the stack/registers, while still knowing about all the references that are live.

Boehm GC works so far, and i guess i’ll wait with improving the GC until the other things on the task list are (mostly) finished.

Thanks for the input!

You could always be evil and use an object table instead. Yeah that would be slow (and I’m just pulling this out of my butt, so I’m probably missing something obvious).

I’d assume that using the Boehm GC, Weak, Soft and Phantom references wouldn’t be easy (or even possible?) to implement?

The reason I ask is that I like Soft references for caching.

Soft/Weak/Phantom references are possible with Boehm GC.

I never really understood why you would actually need a garbage collector or do memory management. I suppose I might be misguided, but couldn’t one just garbage collect everything at the end of the current block, i.e when the object occupying the memory goes out of scope.

@Sickan methinks you need to read up about how heaps, threads and escape analysis work.

Cas :slight_smile:

Simple example: while(true) { x = new Foo(); x.something(); }

And for Sickan’s benefit:

It is not apparent from the code that Roquen just posted whether the construction of a Foo() or calling .something() places a reference to that Foo somewhere else. For example in the constructor of Foo:


Foo() {
    addFooToSomeListElsewhere(this);
}

So you need something called escape analysis to analyse the scope between the while { and } to see if any reference to that Foo actually escapes outside of it. If it does - or if the stack is full - it’s got to go on the heap. If not - it’s a candidate for on-stack replacement, or at the very least, some sort of quick heap reclamation trick.

Escape analysis is very fast, operating in linear time I’m told; and can be run on ever expanding scopes depending on how far you want to take it. In theory you could end up with the entire heap allocated on the stack if your code fits well enough.

Cas :slight_smile:

Escape analysis breaks down as soon as unsafe operations are involved, e.g. JNI, reflection etc. There are ways around this, but i’m not sure i have the jedi powers to pull that off :slight_smile: conservatively allocating some things on the stack could work though.

Well if you write a C++ game and dont use any pre-done GC implementation, which I think many people do, because GC’s get a lot of hate, you do have to do it all yourself then…

Well, you keep those bits of code to an absolute minimum then. Which you do. So no problem there.

Cas :slight_smile:

You can do that for some variables, but not all of them. The trick is knowing when you can do that. Some references may be shared with other sections of code. If those references get deleted then the second section of code that still contains a reference to the deleted objects won’t have access to them anymore. You can use escape analysis to determine ahead of time which objects are safe to delete and which ones are not. Those that don’t escape can be stored on the stack (or a stack, but normally it’s combined with the call stack used to determine where functions resume after return statements). When the call stack unwinds, the stack space allocated for those objects can be freed at the same time. For objects that may need to stick around longer than the function scope, then those are allocated on the heap (shared memory space.)

It’s slightly off topic, but - just in case you’re asking because you read something from a C++ website - sometimes garbage collectors have a time advantage over scope triggered destructor memory management patterns and almost always have an advantage over reference counting. The overhead of memory management may be large enough for single objects that it’s slower to delete only one at a time. You can budget time to delete and dispose objects in batches and actually use less time overall. It’s also misunderstood that reference counting and scope based destructor behavior provides better real time characteristics. Some programs could create chain reactions that could stall the program longer than a real time garbage collector would.


[quote="princec,post:33,topic:39667"] What do you mean by linear time? Can't you do good escape analysis at compile time? Stack allocation takes constant time. And unless you're interpreting code on the fly, then I can't imagine what time efficiency you're referring to. The analysis is probably fast and only needs to be done once.

You can only do escape analysis at compile time if you have enforcing contracts (in a dynamic language).

Wrote about my motivation, anyone please double check and tell me whether i should give up. Oracle seems to have a plan as well at least for iOS.

http://www.badlogicgames.com/wordpress/?p=2583

I have done a lot of research on this subject as well, but free alternatives to span both Android and iPhone just don’t exist without a price. Jack is a breath of fresh air, but I understand the amount of effort it’ll take to change Java byte-code to C would be grueling. On the other side of the coin, you will learn so much about Java byte-code that it’ll probably make libGDX a lot better in the long run.

To be honest, since you are doing this as a learning experience/hobby, I would suggest not giving up on it. Especially since we have no clue whether or when Oracle will be able to complete the port to iOS. Given the security breaches that we are getting already, they are putting themselves in a bad position when it comes to porting Java anywhere. They have to work on fixing the problems at hand first.

I am excited to see that someone cares enough to keep this idea of a free mobile port running. Even though many programmers are not speaking, there is a staggering amount of programmers that are waiting for something like this to emerge.