Xith source & javadoc

[eMKa]

Wow ! What a mess !
I just dug into Xith javadoc, and ehhh there’s a lot of packages, and some of them deprecated, bad-named, duplicated… some classes are not in the good packages (IMHO) and… eh there’s a lot of work.
Here is the list of packages and my comments

Collision System :
com.xith3d.collider
com.xith3d.collider.forces
These should be placed in a separate jar, I think. It’s now broken, buggy and incomplete

com.xith3d.common
Package name could be clearer. It contains only one class, SystemHelper, which is used for debug messages !!! Should be removed if not used. If it is, should be renamed OR moved.

com.xith3d.datatypes
Nothing to say, except that there’s no naming conventions. One is named ExtSomething3f and another Something4f. I’d say, all ExtSomething or all Something, but not two conventions

com.xith3d.gui
The DisplayOptions class seems completely useless… it justs contains two fields : USE_JAVA3D_ENGINE or USE_XITH_ENGINE. Don’t see the use here.

com.xith3d.image
com.xith3d.imaging
I’d like to see the difference here… In fact xith3d.image is for “pixel processing” and imaging is in fact some image loaders. I don’t understand their use… I thought it was java.io (or any JDD class) that was doing the image loading job ?

com.xith3d.io
Totally unknown package. I didn’t know Xith was an I/O library !! I think it’s outdated and should be removed.

com.xith3d.loaders
com.xith3d.loaders.ase
com.xith3d.loaders.texture
The xith3d.loaders package contains some strange classes (Loader and Scene) that seems not to be used. I don’t what we should do about them.
I don’t understand why the ASE loader is part of the core, and not the OBJ, MD2, 3DS, and COLLADA ones.

com.xith3d.occluder
So we have occluder culling ? I thought it was broken…

com.xith3d.picking
Arne’s classes for advanced picking would be welcome here

com.xith3d.render
com.xith3d.render.jogl
com.xith3d.render.lwjgl
Seems OK

com.xith3d.render.lwjgl.test
Should be com.xith3d.test.render.lwjgl

com.xith3d.scenegraph
Contains far too much classes IMHO. Creating sub-packages seems a good idea.

com.xith3d.scenegraph.behavior
Should be com.xith3d.behavior and, in fact should even be external to Xith3D. I didn’t know Xith3D was a behavoring library either. And some behavior classes are in the toolkit, IIRC

com.xith3d.scenegraph.utility
com.xith3d.scenegraph.utility.general
Packages name not clear… After investigations, utility contains a timer (should then be in com.xith3d.timer, or external to the core) and utility.general contains some comparator thingies I never heard of before.

com.xith3d.sound
com.xith3d.sound.drivers.javasound
com.xith3d.sound.drivers.joal
com.xith3d.sound.loaders
Seems right.

com.xith3d.spatial
com.xith3d.spatial.bounds
com.xith3d.spatial.clipping
com.xith3d.spatial.octree
com.xith3d.spatial.octree.triangle
com.xith3d.spatial.spheretree
Is it used ? Is frustum culling functional ? (We don’t have test cases for occlusion and frustum culling)

com.xith3d.terrain
Javadoc filled with “Created with IntelliJ IDEA” ;D And seems magicosm-specific. Well I didn’t succeeded to figure out what it was doing exactly.

com.xith3d.test
com.xith3d.test.lesson37vs
com.xith3d.test.tcm
.test ok, but .lesson37vs and .tcm don’t need seperate package ! or we must be doing sepearate packages for everything ! (model and texture loaders, renderers, lighting, and so on.) Test cases don’t need to have “Xith3D” at the beginning of their name

com.xith3d.transients
Classes should be in datatypes

com.xith3d.userinterface
What’s the difference with xith3d.gui ?

com.xith3d.utilities
Useless, contains one class only, the “ModelRenderer”, that renders a simple cube. Should be removed.

com.xith3d.utility.cache
Don’t understand what it’s used for

com.xith3d.utility.cmdline
What’s the use ? And the syntax of the comand-line commands ?

com.xith3d.utility.events
What does this has to do with Xith ? Is it used somewhere ?

com.xith3d.utility.exception
Useless, contains one class, Check, that checks wether a value is true or false… To be removed

com.xith3d.utility.general
To be splitted in utility.math & utility.compression. And the ZipUtility shouldn’t be in Xith3D (and the JDK, IIRC, has classes that does that job)

com.xith3d.utility.geometry
com.xith3d.utility.geometry.nvtristrip
Is it used ? Classes with unclear names

com.xith3d.utility.image
Classes with unclear names

com.xith3d.utility.interpolate
OK

com.xith3d.utility.logs
OK, but don’t we use an external lib for logging (Log4j / java.util.logger ?)

com.xith3d.utility.memory
OK

com.xith3d.utility.noise
OK

com.xith3d.utility.profile
Seems OK

com.xith3d.utility.properties
Hmm… is this serialization ?

org.xith3d.geometry
Some classes’ names to be changed for the sake of clarity, and conventions to be defined for geometric primitives

org.xith3d.loader3ds
org.xith3d.loader3ds.chunks
org.xith3d.loader3ds.data
OK

org.xith3d.loaders.md2
org.xith3d.loaders.md2.util
org.xith3d.loaders.md2.util.cio
org.xith3d.loaders.md2.util.pak
org.xith3d.loaders.md2.util.pcx
OK, but for PCX loading shouldn’t it be handled elsewhere ? And do we need all these packages ?

org.xith3d.loaders.obj
OK

org.xith3d.loaders.tds
org.xith3d.loaders.tds.model
OK. Could it be in one and only package ?

org.xith3d.test
OK, but redundancy with Xith core tests (com.xith3d.test)

org.xith3d.test.jcd
Why JavaCoolDude does have a seperate package ??

That’s it ! Pheww… I know it’s a bit long to read but I think a clean wouldn’t be a luxury, in our case…

And I still really doesn’t see a clear difference between the core and the TK, sometimes you have an “artistic blur” when you want to know were to put your class(es).

[arne]

[quote]The xith3d.loaders package contains some strange classes (Loader and Scene) that seems not to be used. I don’t what we should do about them.
[/quote]
They actually are supposed to define a common interface for Model-Loaders. And for example croft uses them for his collada loader.

[quote]com.xith3d.spatial
com.xith3d.spatial.bounds
com.xith3d.spatial.clipping
com.xith3d.spatial.octree
com.xith3d.spatial.octree.triangle
com.xith3d.spatial.spheretree
Is it used ? Is frustum culling functional ? (We don’t have test cases for occlusion and frustum culling)
[/quote]
maybe we’ll only have to fix it?

[quote]com.xith3d.terrain
Javadoc filled with “Created with IntelliJ IDEA” ;D And seems magicosm-specific. Well I didn’t succeeded to figure out what it was doing exactly.
[/quote]
for displaying terrains, but it doesn’t support textures AFAIK. As far as I can remember there should also be example using it.

But on the whole you are pretty right I think, Xith is really very messy and also inconsistent. But you’ll also have to know, that most of the changes you’re suggesting would mean great work for everybody using Xith3d (for example everybody would have to change most of the import statements, if we would split com.xith3d.scenegraph into several subpackages)

[croft]

@arne

But on the whole you are pretty right I think, Xith is really very messy and also inconsistent. But you’ll also have to know, that most of the changes you’re suggesting would mean great work for everybody using Xith3d (for example everybody would have to change most of the import statements, if we would split com.xith3d.scenegraph into several subpackages)

I think a cleanup could be done but it would need to progress incrementally.

[eMKa]

@arne

[quote]The xith3d.loaders package contains some strange classes (Loader and Scene) that seems not to be used. I don’t what we should do about them.

They actually are supposed to define a common interface for Model-Loaders. And for example croft uses them for his collada loader.
[/quote]
So the best to do is probably to adapt already existing loaders to this interface, eh ? But for the MD2/3DS/COLLADA loaders, where we have animation, I don’t know how to do. Maybe we can define a common datastruct for animation data and make a shared “frame interpolator” (and a “skeleton interpolator” when we’ll have Cal3D support ;D)… But that’s a lot of work. Loaders are hack or ports…

@croft

@arne

But on the whole you are pretty right I think, Xith is really very messy and also inconsistent. But you’ll also have to know, that most of the changes you’re suggesting would mean great work for everybody using Xith3d (for example everybody would have to change most of the import statements, if we would split com.xith3d.scenegraph into several subpackages)

I think a cleanup could be done but it would need to progress incrementally.

@Croft : hmmm… people won’t like to see 50 different revisions, each time having to changes 20 imports…
@Arne : maybe we can do a tool that refactor a source code to change imports correctly ^ ^

[eMKa]

So here’s a quick summary, after the big dug.
The items striked are done or removed, italic are subject to polemic, and the others are proposals.
For each item or group of item, I’ll launch a poll and apply changes if the majority is for it.
Easy to do

• 001 Move all com.xith3d.collider packages to xith-cs.jar
• 002 Remove com.xith3d.gui package
• 008 Fix occlusion culling if broken. If not, say aloud it’s a feature
• 009 Add arne’s classes for Picking in com.xith3d.picking (I can submit a patch)

Break API compatibility (but easy to do)

• 003 Rename com.xith3d.imaging to com.xith3d.utility.image

• 004 Remove the com.xith3d.io package, if it doesn’t bother anybody

• 005 Move com.xith3d.loader.ase to the toolkit, or change Core/Toolkit policy

• 010 Rename com.xith3d.render.lwjgl.test to com.xith3d.test.render.lwjgl
  Split the com.xith3d.scenegraph package
  [i]- 011 Move AmbientLight, DirectionalLight, Light, PointLight, SpotLight to com.xith3d.scenegraph.light

• 012 Move BoundingBox, BoundingLeaf, BoudingPolytope, BoundingSphere, to com.xith3d.scenegraph.bounds

• 013 Move ColoringAttributes, LineAttributes, PointAttributes, PolygonAttributes, RenderingAttributes, TextureAttributes, and TransparencyAttributes to com.xith3d.scenegraph.attributes

• 014 Move BackgroundSound, PointSound, Sound, MediaContainer to com.xith3d.scenegraph.sound

• 015 Move GeomContainer, GeomData, Geometry, GeometryArray, GeometryStripArray, GeometryTranslocator, GeomFloatData, GeomIndexedContainer, GeomNioFloatData, GeomNioIntData, IndexedGeometryArray, IndexedGeometryStripArray, IndexedLineArray, IndexedLineStripArray, IndexedQuadArray, IndexedTriangleArray, IndexedTriangleFanArray, IndexedTriangleStripArray, LineArray, LineStripArray, PointArray, QuadArray, TriangleArray, TriangleFanArray, TriangleStripArray to com.xith3d.scenegraph.geometry

• 016 Move Background, Billboard, ExponentialFog, Fog, Foreground, LinearFog, and Morph to com.xith3d.scenegraph.nodes

• 017 Move TexCoordGeneration, Texture, Texture2D, Texture3D, TextureCubeMap, TextureUnitState to com.xith3d.scenegraph.texture

• 018 Move FragmentProgram, ShaderProgram, VertexProgram to com.xith3d.scenegraph.program

• 019 Move ImageComponent, ImageComponent2D, ImageComponent3D to com.xith3d.scenegraph.image

• 020 Move BranchGroup, DecalGroup, OrderedGroup, SharedGroup, Link, Switch to com.xith3d.scenegraph.group
  [/i]

• 021 Rename com.xith3d.scenegraph.behavior to com.xith3d.behavior

• 022 Move & rename com.xith3d.utility.JavaTimer to com.xith3d.timer.Timer

• 023 Split com.xith3d.test

• 024 Move com.xith3d.transients classes to com.xith3d.datatypes

• 025 Remove com.xith3d.utilities

• 026 Merge org.xith3d.loaders.md2, org.xith3d.loaders.md2.util, org.xith3d.loaders.md2.util.cio, org.xith3d.loaders.md2.util.pak and org.xith3d.loaders.md2.util.pcx into org.xith3d.loaders.md2

• 027 Merge org.xith3d.loaders.tds and org.xith3d.loaders.tds.model into org.xith3d.loaders.tds

• 028 Merge org.xith3d.loader3ds, org.xith3d.loader3ds.chunks and org.xith3d.loader3ds.data into org.xith3d.loaders.tds2

Heavy changes

• 006 Use java.util.Logger for logging, and remove com.xith3d.common package
• 007 Adapt the model loaders to the com.xith3d.loaders.Loader interface and find something for animation

[eMKa]

(PS : three-digits numbers are used for quickly referencing a TODO item)
(PPS: I wish I had the core developer access, I would do these things myself…)

[arne]

ok first of all I’m against the merging of the sub-packages of the org.xith loaders packages (026, 027, 028) I think, if the designers of those packages decided to have parts in subpackes it’s best to keep it like that.

about my picking algo: should it go to xith or to the xith-tk (dunno where, but it should go somewhere)
I would also check it myself an do some dokumenting if you submit a patch for it. (009)

also +1 for splitting up that huge scenegraph package.

[arne]

016: I would name it com.xith3d.scenegraph.leafs and move Billboard to com.xith3d.scenegraph.geometry (it extends TriangleArray)

[divzero]

Valid comments, but backwards compatability goes out the window

005 has already been done, the files just need removal from CVS (I was waiting a while so everyone has a chance to move over)
006 is actually not that hard to do, and doesn’t break backward compatability. There is an IssueZilla logged on it and it is in progress (commons logging has been added to the third-party directory)

I’m not a big fan for splitting up the scenegraph package, as I think it would violate the principle of tight cohesion and loose coupling. It is possible to take these organisational changes too far and I wonder if this would possibly be an instance off that. What would actually be gained by doing this?

While I havn’t looked into all the cases to confirm, I agree in principle that dead code should be removed and packages with only a few underused classes could be merged into other packages.

The most important thing with non-backwards compatable changes (which cause Xith3D users pain) is that each change is justified and that the benifits outweigh the inconveniences.

On the topic of the Javadoc, I encourage everyone to take a class that they understand, javadoc it and submit a patch/changed file to IssueZilla (we could potentially setup a general Javadoc bug which can be used to track this) .

Cheers,
Will.

[eMKa]

@arne

ok first of all I’m against the merging of the sub-packages of the org.xith loaders packages (026, 027, 028) I think, if the designers of those packages decided to have parts in subpackes it’s best to keep it like that.

Okay, Items removed.

@arne

about my picking algo: should it go to xith or to the xith-tk (dunno where, but it should go somewhere)
I would also check it myself an do some dokumenting if you submit a patch for it. (009)

Well that’s IMHO the core/toolkit division problem, don’t know where to put it.
You will be able to check my patch if you want. I just tidied it up and made it game-independant.

also +1 for splitting up that huge scenegraph package.

@William Denniss

I’m not a big fan for splitting up the scenegraph package, as I think it would violate the principle of tight cohesion and loose coupling. It is possible to take these organisational changes too far and I wonder if this would possibly be an instance off that. What would actually be gained by doing this?

Well, what would be gained ? Clarity. Actually it’s really difficult to distinguish (/classify) classes in the com.xith3d.scenegraph package. Took me nearly a half hour to create seperate packages and put the classes into them. So yeah it may take time to changes imports for Xith users but that the price to pay if you wanna have a clean API. Anyway it isn’t absolutely needed. Hmm… I’ll look deeper into that… (By the way : Can we consider Billboard as a primitive ? In fact I was surprised to see that it derived from TriangleArray, as I rather saw a Billboard as a Group that as Primitive)…

@William Denniss

On the topic of the Javadoc, I encourage everyone to take a class that they understand, javadoc it and submit a patch/changed file to IssueZilla (we could potentially setup a general Javadoc bug which can be used to track this) .

OK. Doing that.

[divzero]

Still on the debate of breaking up the scenegraph package: take the Java3D API packages for example: http://java.sun.com/products/java-media/3D/forDevelopers/J3D_1_3_API/j3dapi/index.html

Apart from vecmath, everything is in the one package.

A package should be mostly self-contained, which is the idea behind loose-coupling – that it’s better to reduce the package interdependancies and have tight-cohesion within the package. Xith3D isn’t a shining example of the loose-coupling principle as it is, you gave a few examples of seemingly meaningless packages.

Perhaps what is needed is simply better non-javadoc non-source documentation explaining what the different features of the scene graph is? Java3D had this (e.g. the Java 3D API Jump Start book). We have the getting started guide which I think is quite good (disclaimer: I did write some of it :)), but perhaps could have a more detailed explanation of the scenegraph.

Billboard is a type of node in Java3D which is probably why Xith3D treats it similarly. The actual implementation in Xith3D is a bit messy and could probably be optimised and cleaned up.

Will.

[eMKa]

Hmm… yeah I see… so we just need :
com.xith3d.scenegraph
com.xith3d.render
com.xith3d.sound
com.xith3d.utility
com.xith3d.picking

And occluder should be in the scenegraph package. And spatial* things too.

[dsellars]

Just a quick point regarding the api. Xith was originally created with the idea in mind of having the same api as Java 3D (as at that time Java3D was in an indefinate holding pattern). If maintaining the same api as Java 3D is still a goal then sure the organisation should continue to reflect this? or be brought in line where it has drifted?

I think this has also lead to some of the missing parts in Xith as the api was based on Java3D, but only the bits needed for magicosm were written.

It’s strange how long ago this all seems now…

[yvg]

Hi,

OK, I will also make some comments on this topic… Still have not too much time on this, but I think it is good to know how the situation was developing in the past.

Originally, big part was based on Java3D API and was created by David Yazel. There was absolutely no idea to develop Java3D replacement, but rather to ease proting of Java3D code to Xith3D - many developers started theit 3D efforts with Java3D, and after [even few years] recognized that Java3D has major problems (not discussing them here).

Maintaining 100% Java3D compatibility was never a goal, it was “good to have”.

After I joined the development, (adding parallel projection, extra transparency sorting modes, some other significant additions) and in parallell decided that all the 3D projects being developed in my company move to Xith3D. Approximately that time other devs also made valuable contributions - Java Cool Dude aka jcd added shaders support (as far as I can recall), Artur Besiadovski aka abies added very important port of NVidia Stripifier, which allows to double up performance in some cases, etc. (I also have to mention William Dennis as community maintainer and our informal “PR chief” ).

Since the original release, I was trying to ensure that everybody follows internal concept of Xith3D - as I mentioned in other thread, state sorting mechanism in Xith3D is quite sophisticated and leaves a huge space for enhancements and optimizations. This is one of the reasons why the public API is as it is - it nearly did not change from the very beginning.

Another point that all the developers were (and are) focusing on the topics they are interested in - for example, I do not use BillBoards [now], so I did not check. Parts touched several times are working fine, others - not.

Unfortunately, Xith3D had never good ARB (Architecture Review Board). Having ARB is very important for keeping clean architecture evolving in correct direction. It is even not possible to design more or less big API and make an implementation of it without ARB coordinating the efforts. Even I with my not so big company of bit more than 40 developers have to have ARB. But ARB means strict following the formal procedures, and it slows down progress a lot - so I will pessimistically tell that it is not possible to develop good API fast. Look at JCP if you ever tried to take part as an expert - it is very very slow but 100% proper way how Sun is doing this.

Every developer pretends that he will improve the architecture, but it is only possible after really deep understanding the previous one [really no critics to anybody - just a general statement; I really really appreciate the efforts of Xith3D community].

If you ask me about the future of the API, I would suggest to create Xith3D API Generation 2, which will have more clean design, better organizing of the source code, etc. We should push the practice of commenting the changes, javadoc, etc. etc. etc.

…but you have to consider sometimes a big - really big - efforts of migrating to new API (for example, I have here more than a hundred thousand lines of code depending on existing Xith3D API now - I already migrated once from Java3D to “compatible” Xith3D, so I exacly know what does it mean for more or less big project).

Now on particular items:

[quote]Collision System :
com.xith3d.collider
com.xith3d.collider.forces
[/quote]
Answer is generalized physics interface, or, even better, generalized scenegraph modification API -> just cleaner API design. Agree to be moved, but then we should make a coordinated repository of “Xith3D Components” (xith-tk is now such repository).

[quote]Is it used ? Is frustum culling functional ? (We don’t have test cases for occlusion and frustum culling)
[/quote]
Yes, it is working. I can guarantee this, because of I was extensively debugging it. We just don’t have test case.

Regarding all the utilites.* package, it is a megrer from xith_utilites.jar dependency introduced by David Yazel long long time ago. I think most of these packages should be factored out or moved (for example, nvstripifier should be moved, but it can be also one of components.

OK, I have to run…

Yuri

[eMKa]

OK Yuri so seems like we have a lot of work.

Personally I don’t remember saying I was going to change Xith’s architecture to make it better ^^ I studied a bit the architecture (mainly via Javadoc and Source sometimes for obscure parts…) but didn’t changed the core (yet).

So you think we have to do an API redesign, make an ARB (= global spec, right ?)… so we should release a Xith version 1 and maintain it “correctly”, and make an ARB for a Xith version 2 ?

I would be the first to be for this approach, but I’m afraid I don’t have the software engineering competences to do that.
But I will follow the process if it’s engaged.

I just hope some competent people will collaborate, since IMHO it’s not doable by one only person.

[yvg]

Hi,

[quote]Personally I don’t remember saying I was going to change Xith’s architecture to make it better
[/quote]
As I mentioned - was no critics to anybody, so please even don’t try to take it on your own account just because of you raised the topic.

[quote]So you think we have to do an API redesign, make an ARB (= global spec, right ?)… so we should release a Xith version 1 and maintain it “correctly”, and make an ARB for a Xith version 2 ?
[/quote]
Well… ARB is just a process - The Board should review all the proposed changes and ensure they fit the concept. Now looks like I am playing the role of ARB - I guess you feel on nearly all my replies a kind of conservative opinion.

As of complete redesign, I don’t think it is a good idea. I prefer evolutionary development rather than revolutionary - introducing a big architectural change is a game that never ends.

I think the first target we should concentrate on is a source code organizing and other efforts, that will help people to use AS MUCH AS POSSIBLE of Xith3D WITH MINIMUM EFFORTS.

It should be easy to start exploring the source code, easy to understand it even if it is big and complicated. I think that strategically more people understand the open-source system is better for the system. My idea (still not implemented) is to convince developers that it is easier for them to change/fix/add features in Xith3D than waiting for somebody else to do this.

Now it is too big effort to start contributing to Xith3D core - only few people can really do this correct way, unfortunately.

To improve this, if I would sign to do this [which is no chance - sorry] I would create a tutorial for beginners that teaches them how to use Xith3D, and in parallel by very small bits feeds them with an information about Xith3D internals with pointing to Xith3D source code, so the barrier of “It is too complicated to even start thinking about this” can be lowered slowly in the heads of developers.

Yuri

[eMKa]

@Yuri Vl. Gushchin

Hi,

[quote]Personally I don’t remember saying I was going to change Xith’s architecture to make it better

As I mentioned - was no critics to anybody, so please even don’t try to take it on your own account just because of you raised the topic.
[/quote]
OK, didn’t feel offensed at all, but just tried to remember in a constructive manner if I did any mistakes in my obscure past…

@Yuri Vl. Gushchin

As of complete redesign, I don’t think it is a good idea. I prefer evolutionary development rather than revolutionary - introducing a big architectural change is a game that never ends.

I agree.

@Yuri Vl. Gushchin

source code organizing

That’s what I tried to do in this topic ;D

@Yuri Vl. Gushchin

that will help people to use AS MUCH AS POSSIBLE of Xith3D WITH MINIMUM EFFORTS.

Well, a database of “Mini HOWTOs” should help.
I can do that on Xith new website.

@Yuri Vl. Gushchin

It should be easy to start exploring the source code, easy to understand it even if it is big and complicated. I think that strategically more people understand the open-source system is better for the system.

And how to achieve that goal ? I don’t mind working if I know what to do

[yvg]

Hi,

[quote]And how to achieve that goal ?
[/quote]
At least, provide a kind of ready-to-use source setups for users of different IDEs, so, say, if one uses Eclipse, he can with single click install Xith3D sources, so his app will be already linked to Xith3D source via Ctrl+Click. This way we will provocate people to look at the sources more often, which leads to more knowledge about the source in general. For many people CVS checkout is a headache…

See how vendors of different microcontrollers do - they provide starter kits and sell development boards, so one can start playing with the chips in a minute.

[again, this is just a very first step, more to come]

Yuri

[eMKa]

I can do a tutorial with screenshots, really easy to follow, to add xith sources to Eclipse, but I don’t know how to do a script for installing that with a single click. I will investigate that a bit more…