How to search for bugs between jvm impelmentation?

Miscellaneous Topics
#1

I have recently come back to my pet project of developing a lossless video codec, and have actually have the preliminary version working! yay! :slight_smile:

Ah, i thought, i can speed things up by using the -server jvm as i dont mind a longer start up if it means faster compression over all. So i then tried my program with the -sever jvm… but it causes my program to crash with an array out of bounds error.

What distresses me even more is that the time and point at which the program crashes seems to be arbitary! In my efforts to locate the error i introduced some println statements, with the addition of these statements the program now crashes on the 5th frame instead of the 3rd frame of a sequence!

This arbitary crashing seems to be like there is some sort of threading issue… however my program is only the main process. Does the -server jvm make my non threaded program threaded?

I then tried compiling the program to native win32 binaries via excelsior JET. The program compiles and runs perfectly (and faster).

It will be about 10 hours before i am able to respond to any replies.
How do i go about finding and correcting the bug causing this? (assuming it is a bug on my end, which i dont doubt for a second!)

The code for the video compresser is http://www.geocities.com/budgetanime/hdiffindex.html

some frames that will cause the error are also included.

to run the compressor:

java -server hdiffC clock -alt -scan -autoDelta 3

assuming that the clock bitmaps are in the same directory as the program.

The program will run perfectly if you use the client jvm… i.e. remove the -server from the above line.

#2

also, i cannot respond to any reply for the next 9-10 hours.

#3

hmm, must be a tricky problem :wink:

#4

It seems there is a bug in the server VM’s dynamic compiler, i.e. your code works ok on the interpreter but the compiled code fails. To prove this, I have run your program on HotSpot 1.4.2_03 three times: as you suggested, with -Xint (interpreter mode) option added and with -XX:CompileThreshold=2 (compile on second call/branch, default is 10000 for the server VM) option added:

> java -server hdiffC clock -alt -scan -autoDelta 3
Frame 0 as KeyFrame.
Frame: 1 as Key Frame: 95017 as Delta Frame: 109123
Frame: 2 as Key Frame: 95057 as Delta Frame: 108831
Frame: 3Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -777
        at deltaC.run(deltaC.java:363)
        at hdiffC.encode(hdiffC.java:228)
        at hdiffC.run(hdiffC.java:201)
        at hdiffC.mainInst(hdiffC.java:131)
        at hdiffC.main(hdiffC.java:42)

> java -server -Xint hdiffC clock -alt -scan -autoDelta 3
Frame 0 as KeyFrame.
Frame: 1 as Key Frame: 95017 as Delta Frame: 109123
Frame: 2 as Key Frame: 95057 as Delta Frame: 108831
Frame: 3 as Key Frame: 95558 as Delta Frame: 108781
Frame: 4 as Key Frame: 95441 as Delta Frame: 108194
Frame: 5 as Key Frame: 95763 as Delta Frame: 107765
Frame: 6 as Key Frame: 95413 as Delta Frame: 106442
Frame: 7 as Key Frame: 95033 as Delta Frame: 106478
Frame: 8 as Key Frame: 95251 as Delta Frame: 107457
Frame: 9 as Key Frame: 95244 as Delta Frame: 108264
Frame: 10 as Key Frame: 95785 as Delta Frame: 108130
No of Frames: 11
avg time taken: 92510
avg time taken per frame: 8410

> java -server -XX:CompileThreshold=2 hdiffC clock -alt -scan -autoDelta 3
Frame 0 as KeyFrame.
Frame: 1 as Key Frame: 95017 as Delta Frame: 109123
Frame: 2 as Key Frame: 95057 as Delta Frame: 108831
Frame: 3 as Key Frame: 95558 as Delta Frame: 108781
Frame: 4 as Key Frame: 95441 as Delta Frame: 108194
Frame: 5 as Key Frame: 95763 as Delta Frame: 107765
Frame: 6 as Key Frame: 95413 as Delta Frame: 106442
Frame: 7Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -777
        at deltaC.run(deltaC.java:504)
        at hdiffC.encode(hdiffC.java:228)
        at hdiffC.run(hdiffC.java:201)
        at hdiffC.mainInst(hdiffC.java:131)
        at hdiffC.main(hdiffC.java:42)

As you may see, you program runs ok on the interpreter and fails at different points depending on when it gets natively compiled by HotSpot. The server VM uses a substantially different compiler than the client VM, this is why you program does not fail on the latter.

I’d suggest that you file a bug report at Sun and use the client VM for now. You may also wish to try IBM VM which is often faster than HotSpot Server on this type of applications.

Out of curiosity, how much faster is the JET-compiled version?

#5

[quote]It seems there is a bug in the server VM’s dynamic compiler, i.e. your code works ok on the interpreter but the compiled code fails. To prove this, I have run your program on HotSpot 1.4.2_03 three times: as you suggested, with -Xint (interpreter mode) option added and with -XX:CompileThreshold=2 (compile on second call/branch, default is 10000 for the server VM) option added:

> java -server hdiffC clock -alt -scan -autoDelta 3
Frame 0 as KeyFrame.
Frame: 1 as Key Frame: 95017 as Delta Frame: 109123
Frame: 2 as Key Frame: 95057 as Delta Frame: 108831
Frame: 3Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -777
        at deltaC.run(deltaC.java:363)
        at hdiffC.encode(hdiffC.java:228)
        at hdiffC.run(hdiffC.java:201)
        at hdiffC.mainInst(hdiffC.java:131)
        at hdiffC.main(hdiffC.java:42)

> java -server -Xint hdiffC clock -alt -scan -autoDelta 3
Frame 0 as KeyFrame.
Frame: 1 as Key Frame: 95017 as Delta Frame: 109123
Frame: 2 as Key Frame: 95057 as Delta Frame: 108831
Frame: 3 as Key Frame: 95558 as Delta Frame: 108781
Frame: 4 as Key Frame: 95441 as Delta Frame: 108194
Frame: 5 as Key Frame: 95763 as Delta Frame: 107765
Frame: 6 as Key Frame: 95413 as Delta Frame: 106442
Frame: 7 as Key Frame: 95033 as Delta Frame: 106478
Frame: 8 as Key Frame: 95251 as Delta Frame: 107457
Frame: 9 as Key Frame: 95244 as Delta Frame: 108264
Frame: 10 as Key Frame: 95785 as Delta Frame: 108130
No of Frames: 11
avg time taken: 92510
avg time taken per frame: 8410

> java -server -XX:CompileThreshold=2 hdiffC clock -alt -scan -autoDelta 3
Frame 0 as KeyFrame.
Frame: 1 as Key Frame: 95017 as Delta Frame: 109123
Frame: 2 as Key Frame: 95057 as Delta Frame: 108831
Frame: 3 as Key Frame: 95558 as Delta Frame: 108781
Frame: 4 as Key Frame: 95441 as Delta Frame: 108194
Frame: 5 as Key Frame: 95763 as Delta Frame: 107765
Frame: 6 as Key Frame: 95413 as Delta Frame: 106442
Frame: 7Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -777
        at deltaC.run(deltaC.java:504)
        at hdiffC.encode(hdiffC.java:228)
        at hdiffC.run(hdiffC.java:201)
        at hdiffC.mainInst(hdiffC.java:131)
        at hdiffC.main(hdiffC.java:42)

As you may see, you program runs ok on the interpreter and fails at different points depending on when it gets natively compiled by HotSpot. The server VM uses a substantially different compiler than the client VM, this is why you program does not fail on the latter.

I’d suggest that you file a bug report at Sun and use the client VM for now. You may also wish to try IBM VM which is often faster than HotSpot Server on this type of applications.

Out of curiosity, how much faster is the JET-compiled version?
[/quote]
wow! I knew my programming was bad… but bad enough to crash a well developed jvm? :wink:

Thanks for all your help! I really appreciate it. Its a load off my mind. I was fretting that i had a major latent bug in my code.

I have tried obtaining the IBM’s jvm… however i have had no luck finding a download for it. (and no amount of snazzy search queries on the IBMs site has revealed where i can get a copy :frowning: )

I will take your suggestion of placing a bug report… once i figure out how to do that. Time to browse suns’s “wonderful” website.

The JET compiled native code is about 1.2x-1.5x as fast as the sun’s client JVM. not a huge improvement, however over a lot of frames it is noticable.

#6

[quote]I have tried obtaining the IBM’s jvm… however i have had no luck finding a download for it. (and no amount of snazzy search queries on the IBMs site has revealed where i can get a copy :frowning: )
[/quote]
IBM 1.3 JDK used to be available from http://www.ibm.com/developerworks/java/. As for 1.4, some posts at JavaLobby.org suggested the following:

  1. Download WebSphere MQ Evaluation (free, requires registration.) from http://www-306.ibm.com/software/integration/wmq/v53/

  2. Start the installation program and click Next repeatedly until it displays a message box with animation asking whether you want to start WebSphere MQ installation now.

  3. The IBM 1.4.0 JDK is now in

system-drive:\Program Files\IBM\Source\WebSphere MQ t_en_us\Prereqs\JDK\ibm-java2-sdk-140.exe

Copy it to the desired location.

  1. Voila! Now you may cancel WebSphere MQ installation (unless you are interested in evaluating it.)

[quote]The JET compiled native code is about 1.2x-1.5x as fast as the sun’s client JVM. not a huge improvement, however over a lot of frames it is noticable.
[/quote]
Well, not that much indeed. If your code is CPU-intensive, esp. floating-point intensive, give IBM JRE a try.

#7

heh, tricky!

no wonder i had little luck finding it then :slight_smile:

Cheers for the info!

Actually most of the the processing is on integers and bytes… i wonder if i will achieve any improvement using IBM jvm.

The java version of the compressor is more of a proof of concept, so speed is not a major issue… tho faster is always better :slight_smile: I (or some other person(s) ) will then migrate it in to C and then optimize the code such that it can be used as a standard vfw video codec.

and wouldnt you know it?! Suns bug submission form has a bug! (oh the irony!) and so i cant submit the bug. Will try again tomorrow.

#8

I did download the IBM’s jvm and the code runs fine on it as well.

I optimized the code a bit and ran some simple bench marks. I ran the compressor over 67 frames on each jvm type 5 times:


     IBM       IBM        Sun     JET
     -Xcomp Vanilla Client  

1    425       531        304     225
2    449       461        302     209
3    433       498        301     209
4    494       456        300     209
5    494       500        301     209

Avg  459       489.2      301.6   212.2

These are the average time to compress each frame. (in msec)

Hmm, it seems my optimizing really helped with the JET compiled native code! it is nearly 100 msecs faster than Sun’s client JVM!

I am a little dissappointed with IBM’s jvm I expect it to perform on par or better than suns client JVM.

I tired it wil the -Xcomp option and it is moderately faster than the vanilla IBM jvm call.

Maybe someone who knows the IBM jvm can tell me what options are optimal for programs that perform lots of array access and computation?

On related news… I sent an email to Sun saying their bug submission page has errors… but no repsonce and I still cannot submit a bug :frowning:

#9

We would be very interested in using your codec as a benchmark. Can we have snapshots of (i) your Java sources, taken right before you start rewriting them in C, and (ii) of your C sources, taken right after they are rewritten?

#10

Well, I would say yes, however it will be a long time in comming… This codec is my Honours Thesis so it will not be finished until at least October, September… The conversion to C will start after that time. I am considering making it an open source project as i am not proficient in C/C++ as i am in java and i feel that i will not be able to do the conversion by my self.

In the process of being converted there are parts of my java code which are not as efficent as they can be and so I (or others) will probably make enough changes to make a direct comparision between the java version and the C version to not meaningful.

I am trying to make my java code “referance” code for myself or others to make more efficient C/C++ code with.