Dynamic versus ahead of time compiler

General Discussions
#1

From the article http://www.javalobby.org/articles/java2exe/?source=archives, the author claims that an ahead of time compiler is a big advantage for a client side java app because your app is fully optimized at start up time (by contrast to a VM compiler) and it doesn’t suffer performance penalty caused by agressive compilation of the VM. One question that quicky arise: why hotspot (to name a VM) doesn’t cache on disk the optimized code?

So what are the arguments of Sun to do dynamic compilation since the beginning of Java? I know Java has been much more used on the server side. Sorry if the answers already exist somewhere in this forum.

#2

The simple answer is, because they haven’t figured out how to do it yet. The internals of HotSpot apparently aren’t currently easy to serialize peicemeal. Especially as some bits of code from an object can actually be compiled differently when inlined depending on where they’re called from - so there isn’t a mapping 1:1 of functions to machine code like you get in C.

Having said that - I reckon that the concept of a serialized VM would work very well; that is, you get the VM to serialize its entire runtime state, and you can then subsequently just blat the lot right back in next time you run.

Cas :slight_smile:

#3

IMHO for tiny console applications pre-jitting code could be an enhancement, when start-up time influences the overall duration of such tiny computational algorithms. (Btw, aren’t there any java tools like NGEN ?

But chaching porbably is a bit different:
Correct me if I am wrong, but to my knowledge JVM can jit the same code several times, to best reflect need of the application at the current state. For example it can discard code parts (conditional branches), which do not excuted a dinstict amount of time and recompile the same code if variables change. Assuming, that currently executed native code is influenced by the input to the application, the question would be which dedicated binaries produced by the JIT the Runtime should be cached?

[edit] oops, another one was faster - I have to work on my writing speed :wink: [/edit]

#4

It’s not excactly difficult. If I’d have less obligations to the state I would write a small program that would modify Hotspot to do so. However some smart ass decided that unemployed must write down record about what they did last month to stop being unempoyed so I have a week out to paperwork. Of course if someone would hire me for 4 months as a consultant/programmer I might do something reasonable. :slight_smile:

BTW Cas You don’t need serialization. In fact it might be viewed as harmful.

#5

A simple solution probably doesn’t produce enough benefit to be worth shipping. When you want to reuse compilation cached on disk you have to verify that it is still valid. If the compilation caching is global (common to all java applications using a particular JVM), then the set of applicable optimisations is reduced.
I seem to remember Microsoft touting a (simple) compiler that was so fast that interpretation would never be appropriate. Would it be worth cacheing the output of such a compiler? Probably not. Instead use it as the first stage of a multi stage compilation system (where frequently used methods later get recompiled by a better compiler). But then how do you cache the results of those later stages which may be dependent on a variety of state in a particular JVM instance (for instance that a given non final method is not overridden by any class currently loaded).
You could, I suppose, do something like SmallTalk and save the entire process state with each application.

#6

Well ahead of time is some sort of cached compilation (apart the exe format) so I guess hotspot might be able to do the same thing. My feeling is that Sun hasn’t been focussing on the client for too many years and now they pay the price of it by trying to sqeeze any bit of performance of the VM and libraries.

Well I should say Sun and the JCP. Whatever the optimizations/native compilations done in the VM, the fact is that your code doesn’t start optimized from the beginning and this is a weak point for java games. One might say that you simply warm your code by animating something else before your game begins but this might be just enoying for the player.

And, why is there only one serious java to native compiler (JET)? Seems that many other companies have failed.

#7

Probably more likely that they’d need to recode or replace so much existing code because it wasn’t originally designed with this in mind. Contrast with C# which has this exact feature in since year dot without and such troubles.

#8

[quote]Contrast with C# which has this exact feature in since year dot without and such troubles.
[/quote]
ouch! That’s not good news for Java if one wants to compare both in terms of performance.

#9

Class Data Sharing
http://java.sun.com/j2se/1.5.0/docs/guide/vm/class-data-sharing.html

Although, a somewhat misleading name, it is exactly ahead of time compilation.
It only runs on the system classes right now, but the archive is setting the foundation tech for complete application wide development.

From link:
"When the JRE is installed on 32-bit platforms using the Sun provided installer, the installer loads a set of classes from the system jar file into a private internal representation, and dumps that representation to a file, called a “shared archive”.

Also
“The primary motivation for including CDS in the 5.0 release is the decrease in startup time it provides.”

#10

wow, it seems to do exactly the same as NGEN (see Jason Zander’s Weblog), except that developers don’t have to bother since class data sharing is used automaticly. very nice :smiley:

thanks for pointing out, Shawn.

#11

CDS is not AOT compilation. Instead it just stores the binary representation of the classes’ bytecode after they’ve been loaded into the VM and processed. The big speed increase is actually just because the whole file is mapped directly into memory and bypasses all the jar opening, class finding, class instantiation, etc.

Cas :slight_smile:

#12

As I understand it, that binary is “close” to native executible (is that the same as “whole file is mapped directly into memory” ?), is that not the case?
Or are you saying that it is just some other form of VM bytecode (that still needs JITting)?
If it is native, is that AOT?

Can a Sun VM person answer this?

#13

hmm… CDS reads like a quick-n-dirty serialisation (C-style, raw binary serialisation that is) of a whole bunch of class files. So I assume the JIT just starts from that rather than open each class file manually.

This pretty much supports what I said earlier - Sun’s VM just isn’t designed for proper saving & restoring of already JIT’ed code, so they’ve done a quick hack to avoid one of the loading’s bottlenecks.

#14

Number 1 or 2 complaint from games developers: Java Can’t Do This.

Time savings for loading thousands of objects as a binary dump directly into memory (as modern games do with ALL data - just one big long disk-cached stream-from-disk) are considerable, on the order of reducing load times from several minutes to ten seconds.

IMHO, Java needs better classloaders / resource-loaders…

#15

Err, do you mean millions of objects? It only takes a few seconds to read in over 100000 objects from a stream, provided the minimum heap size is sufficient to contain all the objects being created.

#16

What I really mean is something slightly more complex than I said: there is a problem with object-creation-overhead, but that’s not the biggy (as you pointed out). The huge penalty comes from cache-misses on the deserialize stream. Bearing in mind, of course, that most are thinking in terms of streaming from DVD, exacerbating this kind of caching problem :).

But those cache-misses are caused by a combination of having many objects and of missing the start of each object by a small amount, and having to wait a partial disk-seek to get back to them. If you stream 100Mb of data directly into memory, as one continuous I/O op, and then scan across it in-memory with the CPU creating all necessary pointers then you won’t hacve to pay the penalty of any disk seeks.

but if you only had 100 objects, of 1Mb each, you would effortlessly stream each object into memory in one go (or I’m asssuming you would, given a non-stupid deserializing algorithm). So it’s a combination of having many objects and of them taking up a large amount of memory.

#17

If everything is compiled to native code, then application’s input has no influence on that code in most cases.

JIT Caching has been present in Excelsior JET since 2001. If a class is loaded that does not match the cached version for any reason (bytecode change, different classloader, import dependency, etc.), it is simply recompiled. You may also recompile the cached classes into a single DLL for better performance and shorter startup.

#18

Hey,

There are lost of ways to argue this back and forth but, as they say, the proof of the pudding is in the eating.

Our VM guys, who are some of the best in the world, went the way they did for reasons they considered important. Othres have gone other ways.

My advice to ANY develoepr is this. If you think thre is a better solutio nfor running your code then TEST it and see. But test it on your real app, not some arbitrary and easily misleading benchmark.

Thats the beauty of Java, you can run your code without modification on ANY compliant VM.

I know of a number of these types of tests and, so far, I have yet to hear of any case other then those domi9nated by start-up time (little comamnd line utilities, for example), where pre-compilation has produced a superior result. But hey, if it does, then use it!

#19

Jet got me quite an advantage in Alien Flux :slight_smile: As fast as or faster than the 1.4.2 Hotspot server VM, and instant startup. There’s a real world example. It’s a great product but you need a good case for actually buying it - it’s not just a toy for us amateurs really.

Cas :slight_smile: