Performance & Benchmarks

Engines, Libraries and Tools Xith3D Forums
#1

In the topic “About 3D Model Loaders” Java-Cool-Dude has posted some FPS numbers. These numbers have been based on his Quake3-Model loader, which animated the model and has been done in three implementations for Java3d, Xith3d and Jogl.

Say the FPS numbers would be avarage. Then Dude’s FPS numbers look like this:


Jogl   683 fsp: 100%
Xith3d 504 fps:  74%
Java3d 306 fps:  45%

I’ve to add that these numbers aren’t comparable on a direct base. The way the Xith3d and the Jogl implementation work are different, and so on.

Also it’s a basic task; add more complexity and the Jogl implementation’s FPS will very probably approach the one of the Xith implementation. Like Kev said in another thread.

#2

As Dude said in that thread, its not really valid to compare the JOGL version to the other two, since they’re not implemented in the same manner.

A generic scenegraph (in general terms) will not be as fast as an custom application written directly against OpenGL. The scenegraph will always be performing some extra work through its nature.

Therefore I’m not sure how valid it is to benchmark Mr X’s pure JOGL impl of anything against a Xith/Java3D version.

To be honest there was alot of time spent in Java3D threads trying to justify its existence when comparing performance to native OpenGL apps. I’m sure that in part this was what gave Java3D its bad performance name and contributed to the state its currently in.

Wouldn’t it be best to let the applications written using Xith talk for themselfs in terms of performance. I realise this is going to be difficult initially, however Dude is doing good work in producing lots of demos and its seems there are at least a couple of projects on the go.

Reading back over this it sounds a bit crappy, I guess performance is a valid thing to be considering, maybe we just need to let it become the be all and end all.

Kev

#3

In the other mentioned topic Yuri wrote:

[quote]Short note regarding performance: Xith3D is not performance-optimized right now. Some of shaders are coded straight-forward, and some memory allocations have to be fixed, too. So I believe we can come much closer to plain JOGL performance.

From the other point, Xith3D shader optimization may provide big benefit on complicated scenes, where optimizations in JOGL will be not obvious.
[/quote]
This is a good note.
Is there already a date when the speed optimization will be started? Or is it too early to ask for such a thing.

I think this is a quite important topic, because one of the main differences between Java3d and Xith3d is and should be the execution speed.

For example currently I’m programing in my spare time a little 3d game based on Jogl. Well, I see that I re-invent the wheel several times because Xith3d does provide all of what I need and much more (hurray). Well, I thought maybe I could use Xith3d and join its user base. However this is only possible for me if my game type fits to the architecture of Xith3d and the performance of it will be roughly the one of a comparable Jogl implementation.

Well, of course the Xith3d project is “young” and there are so many other important tasks in the beginning. So my article’s intended in a very “general speaking” way.

PS: Maybe there will be some notes about “performance hints” for Xith3d/Java3d including “do never do this” etc.

#4

[quote]As Dude said in that thread, its not really valid to compare the JOGL version to the other two, since they’re not implemented in the same manner.
[/quote]
Yes, this a good and important note and I am happy you say it again.
Unfortunately there are no other numbers yet to get a clue how these two APIs (Xith3d and Jogl) peform on a certain task. Well it’s not really possible to compare a low level API like Jogl with a high(er) level API like Xith3d - just their output of a similar task can be compared…

[quote]A generic scenegraph (in general terms) will not be as fast as an custom application written directly against OpenGL. The scenegraph will always be performing some extra work through its nature.
(…)
To be honest there was alot of time spent in Java3D threads trying to justify its existence when comparing performance to native OpenGL apps. I’m sure that in part this was what gave Java3D its bad performance name and contributed to the state its currently in.
[/quote]
I see. My problem is that I don’t know the Java3d API (well I saw its demos and how slow they’ve been - relatively. So it’s not been that interesting for me.)

I like Jogl but the direct OpenGL way is pretty low level and I’m still unsure if I like this.

[quote]Wouldn’t it be best to let the applications written using Xith talk for themselfs in terms of performance. I realise this is going to be difficult initially, however Dude is doing good work in producing lots of demos and its seems there are at least a couple of projects on the go.
[/quote]
Since Xith3d is intended as a gaming 3d API: what types of games are well suited for Xith3d and which are not? If it’s possible at all to group. I know, yet there aren’t to many finished ones. Maybe some people who use Xith3d for their games right now would like to comment? Thanks.

Would it be silly to use Xith3d for an action game comparable to http://www.pompom.org.uk/spacetripper.htm ? (Not concerning the gameplay, but rather the way how it’s being done.)

#5

From a short look at space tripper:

  1. What spec would you be expecting to run your game on? They say any OpenGL card (no mention of memory or type). I’d be interested to see how well it ran on a Voodoo 2 :slight_smile:

  2. Technology wise, nothing really clever by the looks of it. Seems to be a good way to go at the moment, don’t push the boundaries of 3D tech just do it simply but really really well (see Alien Flux)

  3. Action Based - Looks like there are only a few enemies on the screen at at a time (<20) and the models looks pretty light on polygons (<100). Depends how many you’re going to be pushing around.

If you just wanted to duplicate the game IMO it would be possible in Xith but as to whether you’d get good performance on old cards, I doubt it. By old I mean pre-GF series. You might be able to squeeze out an extra few FPS if you wrote it direct to JOGL but you’d spend quite a while (weeks) implementing stuff that Xith already has.

However, if its a hobby project (is it?), just do it in whatever suits you best.

Kev

#6

Preston,

Those stats are far from hard evidance. They are but one example. Don’t forget the project is only alpha, don’t go basing any conclusions on that one test.

If you have the time to write directly to the lower layer then that is your choice. Hell if you are that concerned then you can always skip java, use C++ or even assembler. Isn’t it nice to have some insulation from those layers? Personally I don’t see any need to waste my time reinventing the wheel.

Comparing Xith3D to JOGL is not a true comparison anyway. It’s interesting - but you can’t say “Xith3D is slow just because it isn’t as fast as JOGL”. If you want to do an accurate survey then compare the Xith3D Scenegraph to other scenegraphs (Java and otherwise).

Looking at your stats another way you have:


Java3D: 306 fps:  100%
Xith3D: 504 fps: 164%

Basied on your stats (which as said I don’t hold with a huge amount of credit as they are derived from but one demo, run on but one computer) Xith3D is 1.6 times faster than Java3D

Will.

#7

Hi Kev, thanks for your nice article.

[quote]From a short look at space tripper:

  1. What spec would you be expecting to run your game on? They say any OpenGL card (no mention of memory or type).
    [/quote]
    Geforce (and comparable) and better should be OK. I mean right now I use a TNT2 so it should run on it, too, but the user base of such old cards is shrinking - luckily.

[quote]2) Technology wise, nothing really clever by the looks of it. Seems to be a good way to go at the moment, don’t push the boundaries of 3D tech just do it simply but really really well (see Alien Flux)
[/quote]
Yes, Spacetripper is no Quake3-engine-type game but it does its job very well, like Alien-Flux, I think. Simple but addictive gameplay, done in a technically impressive way. That’s the way to go (for independent devs).

[quote]3) Action Based - Looks like there are only a few enemies on the screen at at a time (<20) and the models looks pretty light on polygons (<100). Depends how many you’re going to be pushing around.
[/quote]
I think they use more polys?
Anyway, my plan is to use about 100 3d models whith a total of 30 000 to 50 000 polygons (on older 3d cards I could decrease the number optionally). About half of these models will have (alpha-) transparent surfaces. So fast sorting is important, too. Xith3d does already support such alpha transparent surfaces according to some demos I’ve seen.

[quote]If you just wanted to duplicate the game IMO it would be possible in Xith but as to whether you’d get good performance on old cards, I doubt it. By old I mean pre-GF series. You might be able to squeeze out an extra few FPS if you wrote it direct to JOGL but you’d spend quite a while (weeks) implementing stuff that Xith already has.
[/quote]
I see. While I won’t duplicate the gameplay I intend to do similar effects (also Alien Flux has such nice OpenGL effects). Like Spacetripper my game is bascially a 2D game idea but being done in 3d (with moveable camera etc).
Pre-GF 3d cards shouldn’t be too important but it should run well on Geforce upwards.

[quote]However, if its a hobby project (is it?), just do it in whatever suits you best.
[/quote]
At the moment it’s a hobby project but if it should become nice maybe I could release it as budget game. A friend of mine does the models/art (he’s a professional 3d artist).

#8

[quote]Preston,
Those stats are far from hard evidance. They are but one example.
[/quote]
Hi Will. You’re absolutely right. Probably I didn’t manage to emphasize enough on this fact (with my basic English knowledge), that these numbers are very “thin” to do any comparison at all.

[quote]Don’t forget the project is only alpha, don’t go basing any conclusions on that one test.
[/quote]
Yes, the project is very young and I think it’s a good point to thank all you people who contribute to it. It looks very nice.

Still I need to have some kind of comparision in order to decide whether to go for Xith3d or “direct” Jogl. Probably I need patience because we’ll see more numbers in the future and I’ll have to learn about scenegraphs (as said, I don’t know the Java3d API but just how to plot models/polygons via Jogl.)

[quote]If you have the time to write directly to the lower layer then that is your choice.
[/quote]
Well, I think I could have the time but … First I am no good low level programmer (anymore) and second if possible I don’t want to re-invent the wheel.

[quote]Hell if you are that concerned then you can always skip java, use C++ or even assembler.
[/quote]
I like Java. :slight_smile:
Also I am not concerned that Xith3d is slow by what it is performing. I am asking and writing here to find out if it’s a good idea to do a game comparable to Spacetripper (in some way) with a scenegraph based engine like Xith3d is.

[quote]Isn’t it nice to have some insulation from those layers?
[/quote]
Absolutley! This is why Xith3d looks so nice to me.

However, I don’t know why Java3d has been so slow. When I saw some Java3d demos or showed them to some 3d artists, nobody has been impressed. Other than Jogl stuff, like its demos, or Wurm, or Alienflux (ok, no Jogl, but Ogl). Also the little Xith3d demos do look very nice.
So my impression currently is: you can do cool 3d graphics with Xith3d without the need to use low level Ogl API calls.

I am sorry that my direct comparision of the FPS numbers of a loaded and animated Quake3 model (in Jogl and Xith3d) caused confusion. These are no comparable tasks, indeed. Let’s concentrate on real world games then.

#9

Damn! You’re so lucky to have a pet artist like that. Really hard to find. I’m thinking my 2D (in 3D) platformer is going to be my current project for a while but I get to rely on community art (which well being very nice) isn’t tailorable. I wish there was somewhere hobby developers could get hold of hobby artists.

Anyway, good luck with the game…

Kev

#10

Fair enough. I think you should decide first whether you want to use a scenegraph or not. If you do use a scenegraph, then your options currently in Java arn’t many but Xith3D IMHO is a good candidate.

Essentailly it’s a trade off between development time and performance. I have used OpenGL (in the form of GL4Java) and Java3D/Xith3D - using a scenegraph is so much better to program with, the end result is a lot better because you can concentrate your work on making the game good. Also being a standard API - there are heaps of tools out there and other people have a better chance of helping you.

For complex stuff (which that demo wasn’t) a scenegraph can even increase performance sometimes by doing optomisations for you (without much intervention).

Will.

#11

Oh yes.
On the other side it’s difficult for a hobby 3d developer (3d newbie) to fulfill his expectations. Well, at work his colleagues use commercial engines so he wonders why it takes so long until my small Jogl app imports and renders his nice Lightwave models… :wink:

[quote]I’m thinking my 2D (in 3D) platformer is going to be my current project for a while but I get to rely on community art (which well being very nice) isn’t tailorable. I wish there was somewhere hobby developers could get hold of hobby artists.
[/quote]
That would be fine, indeed.
Your project looks nice so far.

[quote]Anyway, good luck with the game…
[/quote]
Many thanks! Same applies to your project.

#12

I’m reading the Xith3d tutorials and will look into some Java3d doc, too. If possible I’m going to use your nice engine.

[quote]I have used OpenGL (in the form of GL4Java) and Java3D/Xith3D - using a scenegraph is so much better to program with, the end result is a lot better because you can concentrate your work on making the game good.
[/quote]
That’s the point, yes.

[quote]For complex stuff (which that demo wasn’t) a scenegraph can even increase performance sometimes by doing optomisations for you (without much intervention).
[/quote]
That sounds very well.
Now I’ve to figure out if a (small) 3d action game fits nicely to a scenegraph renderer like Xith3d. For example like the mentioned budget game http://www.pompom.org.uk/spacetripper.htm

#13

I don’t have facts which show how good the performance of Xith3D is, but my personal opinion is the following:

You want the game to run on cards starting from GeForce 1 (about 4 years old) up to the current cards of ATI and GeForce. The current cards are about 3 to 10 times faster (depending on many factors) than a GeForce 1. The tradeoff between Jogl and Xith3D is an order of magnitude smaller (30%?). I think if there’s no particular reason why you need Jogl, you can always use Xith3D.

#14

[quote]Is there already a date when the speed optimization will be started? Or is it too early to ask for such a thing.
[/quote]
I personally think that for now the major goal is to add functionality to Xith3D, so it is at least the same level [of functionality] as Java3D. Then we switch to agressive optimizations.

BTW, some people already started working on performance enhancements, so we have progress also in this area.

Regarding the performance question in general, I see major speed-ups on strategical engine desing, rather than on code fine-tuning [which does not mean we allowed to write poor code].

Now on performance tests. I think we should have as many tests as possible, at least to figure out the problematic points in design and implementation. This will help to enhance the entire engine, as well as write the performance tips, how-to’s etc.

Yuri

#15

Unless you are going to have a lot of free time to work on your project I’d definitely say to go with Xith3D. You’ll get a lot more done in less time. If you have perf problems, you can then profile your program and take time to optimize any problem areas. Probably by the time you are getting near to being done with it, Xith3D will be much more mature and optimized anyway :smiley:

#16

Lemmie write a real benchmark…
Brb

#17

quote The current cards are about 3 to 10 times faster (depending on many factors) than a GeForce 1. The tradeoff between Jogl and Xith3D is an order of magnitude smaller (30%?).
[/quote]
Good hint.

#18

A very reasonably roadmap.

[quote]BTW, some people already started working on performance enhancements, so we have progress also in this area.
[/quote]
Excellent, too.

[quote]Now on performance tests. I think we should have as many tests as possible, at least to figure out the problematic points in design and implementation. This will help to enhance the entire engine, as well as write the performance tips, how-to’s etc.
[/quote]
Yes.

The more I learn about Xith3d the more it looks interesting to me. Thanks Yuri, David, Jens, William and all fellow developers and helpers for your work and in particular the helpful articles here in the forum.

#19

[quote]Unless you are going to have a lot of free time to work on your project I’d definitely say to go with Xith3D. You’ll get a lot more done in less time. If you have perf problems, you can then profile your program and take time to optimize any problem areas. Probably by the time you are getting near to being done with it, Xith3D will be much more mature and optimized anyway :smiley:
[/quote]
Also a good point. (Many very good points during the last few days here, :-).
Open-Source impresses me.

#20

Yay I’m finally done, all mah work babeh ;D

http://www.realityflux.com/abba/finalProject.jpg

Lemmie talk to BMyers if I can post the source and write a tutorial about it :stuck_out_tongue: