JPCT vs JME vs Xith3D vs ... benchmark

General Discussions
#1

Thought this might come of some interest of people who don’t read the JPCT forum.
A basic benchmark(Maybe someone could double check the results), I was shocked of the winner. Original thread: http://www.jpct.net/forum2/index.php/topic,809.45.html

Latest tests:

[quote]Now for the tank…i’ve converted it to OBJ format first to ease testing. So i could reuse my existing test applications without changing anything except the filename and maybe the scaling.

I wasn’t able to load it with Java3D (it bombed out with a nullpointer exception when loading the materials), so i skipped this. The numbers for JME are taken with the build-in stats tool and backed up by FRAPS (had to write the fps to a file, because FRAPS’ onscreen display stops at 999). I’ve added 5% to the JME score to compensate for the stats tool’s display. This seems to be reasonable judging from the results with the car. Of course, lockMeshes() has been used.
For 3DzzD, i’ve tested it in IE8 and Firefox3 and got the same results. An applet isn’t a very good environment to test performance IMHO but it’s all i got, so…

Here are the numbers:

jPCT: 3500fps
3DzzD: 2100fps
JME: 2000fps
xith3D: 1950fps
[/quote]

[quote]I’ve managed to load the model into Java3D…i have no idea, if there is something like lockMeshes() in Java3D too, but anyway…
So on the Core2 Quad@3Ghz, ATI Radeon HD 4870, Java6, Vista Ultimate, we now have:

jPCT: 580fps
JME: 525- 530fps
xith3D: 510fps
Java3D: 120fps (with some visual glitches, but that could be a bug in the loader)
[/quote]
Old tests:

[quote]At first, the results of rendering that high polygon car model that paul provided…

…on a Core2 Duo@3Ghz, some lowly NVidia Quadro GPU:

jPCT: 78 fps
jME: 34 fps
xith3D: 22 fps

The same thing rendered on my main machine, a Core2 Quad@3Ghz, ATI Radeon HD 4870:

jPCT: 580 fps
xith3D: 222 fps
jME: 30 fps

And finally, the famous Quake3-level from the xith/jME-benchmark…but just loaded as a model…i haven’t converted the whole benchmark thing and only loaded it into xith and jPCT on the Quad core:

xith3D:
fps view: 91 fps
level view: 69 fps

jPCT:
fps view: 1000 fps
level view: 1000 fps
[/quote]

[quote]The car-model loaded on a Core2Duo @ 2.66Ghz, Intel GMA 950 onboard graphics:

jPCT: 6fps
JME: 4fps
xith3D: crashed due to absence of glActiveTextureARB on that machine (which is a bogus message, because jPCT uses that too and runs fine)

The same model loaded on a Pentium 4M @2.2Ghz, Geforce 2MX Go:

jPCT: 3fps
xith3D: 1fps
JME: unknown (but feeled like 1+fps)…i wasn’t able to enable the fps stats, just got a gray overlay instead

and finally on an Athlon X2 @2.2Ghz, Radeon HD 3650 AGP:

jPCT: 200fps
xith3D: 110fps
JME: 28fps
[/quote]

#2

Hi!

I’m highly suspicious. Can someone double-check?

#3

No need to be suspicious. I haven’t added a Thread.sleep(xxx) or something.

I’ve used this model: http://www.paulscode.com/Models/triantObj.zip

This is the jPCT-app that displays it (an older version with a small bug, but that doesn’t affect performance): http://www.jpct.net/download/beta/car.exe (it’s a self-extracting exe for Windows, i was lazy…)

For xith3D and jME, i’ve simply used the obj-loader-demos that come with the engines and replaced the model. I’ve used the latest stable releases for both.

#4

great, dont know really why but I am not surprised,

where can we check the car model (the first test case ?), I want to get my stats too ! :slight_smile:

#5

There’s a link to it in my previous post.

#6

Very interesting - would love to see the results for 3DzzD as well. With JME looking increasingly ropey jPCT and 3DzzD are looking like the prefered alternatives right now IMHO.

#7

Why do you say that JME is looking increasingly ropey? There are still some developments on it, some people are even preparing JME 3.0, someone else is repairing the JOGL renderer and the AWT input system (as I have not enough time to do it myself), I am still working on the portal culling and some other people are submitting other features.

#8

That is really impressive.

Any chance of jPCT going open source?

#9

Watch this:
http://www.jmonkeyengine.com/jmeforum/index.php?topic=10879.msg82316#msg82316

[quote]Well, you have to wonder which tests he is running (where’s the code and the models, etc.?) and what versions of xith and jme, etc. Easy to claim a benchmark when you are the engine’s author and you don’t give out the exact code for others to try themselves.

That said, the 91 vs 1000 Q3 test thing is not very surprising if you follow that thread along… He added a feature to jpct that compiles everything down to just the fewest number of batches needed to render that scene. In other words, at a very elementary level imagine the Q3 level is a big triangle soup… go through that soup and separate out all the triangles that are using the same states. Combine those into one big mesh. Now instead of several thousand “bsp faces” you have a couple dozen meshes. Your polygon chomping card loves you. Smiley

Great idea except it changes the nature of the “benchmark” since jme and xith are purposely processing it as an old fashioned BSP with visibility calcs, etc.

Then again, those numbers are kind of odd:

Quote
xith3D:
fps view: 91 fps
level view: 69 fps

jPCT:
fps view: 1000 fps
level view: 1000 fps

maybe i just don’t know what “fps view” is, but on my older PC here at home (P4 2.4G, nVidia 7800) I get in the 200-500FPS range flying through the level for both Xith and jME.

edit: maybe those numbers are from altering the xith test to not use visibility sets. That would fall in line with the numbers.
[/quote]

#10

Because there’s no quality control any more. I may be wrong but as far as I know:

  • main developers have gone, so there’s lots of code no-one really has a deep understanding of.
  • No proper releases, just instructions to “get the latest from svn”
  • No nightly builds or automated testing
  • Allowing anyone to check in to the source repository means that there’s zero guarantees that it’ll actually work.
  • All the demos on the website are still JME 1.0
  • Broken links in javadoc (which I’m not even sure is recent, nor if it’s 1.0 or 2.0).

It’s a shame because it used to be a production quality engine, but it seems like it’s being run by amateurs now.

#11

It is wrong. We have to show our source code before submitting it and it has to respect the coding conventions.

It is wrong too, I had to ask for the right to check in.

At least lots of demos are still in JME 1.0.

#12

Manually eyeballing code is not quality control, and still leaves all of the other points unaddressed. Individually they’re not insurmountable problems, but they’re indications of the decay and break down of the project as a whole.

#13

I agree that the Quake3-benchmark isn’t an apples-to-apples comparison. That’s why i only ran it once on one machine. On the other hand, the visual outcome is the same, just faster, so it doesn’t really matter to the user. The “fps-view” simply is a defined view inside the level like a player would see it. “Level view” means, that the whole level is visible. And that view gave me the results that i’ve posted. I would have tried a newer version of xith3d than the latest stable, but the link to it was dead.

#14

From http://www.jmonkeyengine.com/jmeforum/index.php?topic=10879.msg82325#msg82325

[quote]- The car model is really nice grin

  • It weighs in at about 1.1million triangles, so let’s all be happy for Java engines showing these framerates.
  • I get around 20FPS with jME just loading the car and dumping it in the scene.
  • Just by doing model.lockMeshes(), that jumps to about 250FPS
  • This is on a single core AMD Sempron 3400 with a 3650 AGP graphics card.
    [/quote]
#15

That’s a little better heh. Though would be nice to now compare it with the other 2 engines.

#16

Yes, that looks better. As said, i have no clue of JME or xith3D. But i expected the demos to be written in a way that shows off the engines’ full capabilities. Obviously, this isn’t the case… ???

#17

Aviatrix3D is also a good alternative. Not very well known on this forum, but I think it is the best java scenegraph at the moment.

As for the jPCT benchmark. A large model should be rendered at the same speed on all engines that supports some kind of compiling. It boils down to a few display list or VBO calls. The Q3 model is a different matter. It has thousands of objects with an average of 10 triangles and around 100 unique states. Combining the objects can improve the performance. Although you don’t always want to merge objects as culling the objects separately might be better. It all depends on the scene.

#18

I’ve retested the JME-version with a call to lockMeshes() on the loaded object (tested on the Quadcore mentioned in the first post). It’s @520 fps now, which is almost on par with jPCT. When compiling the object in dynamic mode on jPCT (which seems to be similar to what JME does without lockMeshes()), jPCT puts out 30fps…exactly the same as JME does. So i guess it’s almost a draw now… ;D
Does anybody know if xith3D has something similar like lockMeshes() in JME? Is there some OBJ-loader in JAVA3D to include this in the benchmark?

#19

I’ve managed to load the model into Java3D…i have no idea, if there is something like lockMeshes() in Java3D too, but anyway…
So on the Core2 Quad@3Ghz, ATI Radeon HD 4870, Java6, Vista Ultimate, we now have:

jPCT: 580fps
JME: 520fps
xith3D: 220fps
Java3D: 120fps (with some visual glitches, but that could be a bug in the loader)

#20

Java3D BranchGroup has a compile() method. Javadoc:

http://download.java.net/media/java3d/javadoc/1.5.0/javax/media/j3d/BranchGroup.html#compile()