I have started work on input/output of Xith3D scenegraphs from/to XML format. It appears to be working well so far with the demo programs. I have a few questions:
1> Shared objects. What should I be looking for here. I need suggestions on how to handle shared data.
2> Texture image data. I was hoping there would be some way to tie the images in Textures to the TextureLoader however there doesn’t appear to be a way to connect a Texture2D/3D object back to the image used in the TextureLoader. Any suggestions?
3> Currently I’m thinking about using base64 encoding to place the texture bytes into the XML. If I could get all of the texture image info (filename, directory, etc.) I could avoid including the data in the XML. Thoughts?
Below is the output from the CubeTest demo with the number of cubes reduced to 2 instead of 300.
edit: I know it is a bit hard to read, the forum screwed up the indenting.
EDIT: I REMOVED THE XML POSTS TO CLEAN UP THE THREAD.
As for shared objects , I don’t know exactly what you mean by that (being the j3d/xith3d clewbie that I am :)) so I might be totally off track here, but maybe you could use XPath expressions to reference other nodes in the XML where those shared objects are described?
Forget this post if I’m totally misunderstanding you ;D
XML could be excellent for defining Xith3D shape’s or transformgroup’s physical attributes.
How about adding an physics extension for your XML?
For each object one could define:
identity (or name) of object
collision primitive(s)
-geometry data like sizex,y,z (e.g. box)
-local position (x,y,z)
-complex composite objects have multiple primitives
mass distribution
-e.g. simply mass size which is centered
surface parameters
-frictions, bounciness…
There should be also common physics XML definition for the world itself that contains e.g. gravity or common friction values etc. Also joint node’s would be good, these bind two different objects to each other (e.g. leg to body) and define their attributes (how much can I move my leg related to my body) and so on.
After these definitions, not only Xith3D objects but also their physic engine objects could be created automatically using XML definition, both sides would be neatly in the same place.
Here’s one thing I’d like to clarify for people that haven’t used physic engines yet. Why to define primitives two times per object, once for scenegraph (Xith3D) and once for physics engine? Think for example a human, it’s quite complex object for 3D renderer, but for physics engine it consists of a sphere (head), cylinders (neck, arms, legs) and a torso (box). As physics calculations are quite complex it’s best to simplify objects as much as possible on the physics engine side.
The reason I am writing this is that I’ve been thinking of creating some kind of physics definition for Odejava project (physics engine for Java, based on ODE). This definition would be perfect place for this and XML sure is the right tool to accomplish this.
My take on this would be that you would need some sort of generic resource loading. You would give it a list of directorys, zip files or whatever, and it would serve up files for you on demand. You could reference this in your xml.
A good idea of what i’m trying to say is in ogre, which reads a list of locations for files, models, textures, etc at start up.
Of course this means writing a resource loader, and your idea of what it should do is probably different to mine.
[quote]1> Shared objects. What should I be looking for here. I need suggestions on how to handle shared data.
[/quote]
I was mainly worried about things like: GeometryTranslocator and SharedGroup which I don’t really understand the implications/use of.
I think I can use the TEX_STATE_ID_SEQ member of the Texture class to identify textures that have been used across more than a single object. I will need to add a method to allow me to get the value, any objections?
It would be very handy if ALL objects had a common unique ID that could be referenced. It only needs to be readable. Can this get implemented, what issues might this cause? I think an interface that defines a single method “long getRuntimeUniqueID()” or somthing like that would be nice to implement in the low level classes.
[quote]My take on this would be that you would need some sort of generic resource loading. You would give it a list of directorys, zip files or whatever, and it would serve up files for you on demand. You could reference this in your xml.
[/quote]
Okay, I tried this and it simply isn’t going to work with large textures. So I’m going to try to load the textures via the TextureLoader and the XML will only contain attributes not image data. The getSearchPath() method will allow me to store the image paths in the XML and register them during read. I still need one of two things:
1> Add a method to TextureLoader to allow me to get the file name (mean other mods to the class) of a texture based on either the TEX_STATE_ID_SEQ of a Texture object or the Texture object itself.
or
2> Add the file name information to the Texture class. I like this idea better, keep the the data with the object that it belongs to. Thoughts?
XPath is an iteresting idea, I’ll take a look at it. Right now if I could get the unique ID mentioned in number 2 above the problem basically solves itself. I would traverse the graph and write out an entity for each object, keeping track of the unique IDs already written. Then traverse the graph again writing out the actually graph structure using only ID references to point to the original elements.
[quote]XML could be excellent for defining Xith3D shape’s or transformgroup’s physical attributes.
How about adding an physics extension for your XML?
[/quote]
Would be glad to if I knew anything about it, perhaps you could help on that side. I need to get the basic scene graph stuf working first though.
It occured to me that I might be able to use the java.lang.Object.hashCode() method to get my unique object identifiers for the graph objects. This would remove the need for adding code to every object type. However the javadoc description has me a bit worried.
Don’t those two statements contradict each other? ALso look at this statement from the equals() method.
System.identityHashCode, not Object.hashCode. But it is not guaranteed to be unique. Comment about it being mostly unique due to being pointer address is not true in presence of copying garbage collector (which is a standard currently AFAIK).
Yep that method will have the same problems, I have to have unique values. However I think I may have found another way around this.
Writting the XML File:
Run the graph and for each object:
0> init objectCount variable to 0
1> Check an Object[] to see if the object is in already in the array using the == operator to compare the memory addresses.
2> If the object is already in the array skip it, go to step 1.
3> If the object is not in the array add it to the array
4> Increment objectCount variable and add the value to an array called objectIDs in the same position as the object was added to the Object[]
5> Write the object to the file with the current value of the objectCount variable as its ID
6> Run the graph again to write the graph structure to the file.
7> For each object find the reference in the Object[] using == operator and get the ID from the objectIDs array in the same position.
8> Write simple XML element for the Object type that contains only the ID
Reading the XML:
0> Read each of the objects elements in and create the Xith3D objects adding the Object ref to an Object[] and the ID to a long[] objectIDs in the same positions.
1> Read the Graph structure in.
2> For each graph xml element get the Object reference fromt he Object[] by finding the ID in the objectIDs array and getting the reference from the same position in the Object[].
3> Do what ever processing is needed on the Object, for instance adding a Locale to the VirutalUniverse.
Would it make since to write the XML out to multiple files?
I was originally thinking that I would write the entire graph out to a single file. Now I’m wondering if it makes more scence to split them like the following.
You supply a root name for the files, such as “landscape” and pass in the VirutalUniverse object that contains the graph. The code would then create:
landscape_graph.xml <- just the structure with references to the other files
landscape_typenname.xml
Where: type is the Object type n is the unique ID I create during graph traversal name is the name attribute of the object if it is set
This would allow me to add methods to the reader/writter classes for reading/writing indiviual objects. This way you wouldn’t have to create the entire scene graph from a file. You could for instance create a tree object write it to a single file by itself (actually possibly two or more files tree_shape3d_1_bigoak.xml and tree_texture2d_1_bigoakbark.xml) and use it in other projects by having the reader load it and pass you back the Shape3D object which you could then add to a dynamically created graph.
Another up side to this would be we would have a DTD to conform to for each object type. This means a published “standard” format for programmers to use to write converters, for instance OBJ to Xith3D.
With xml you are already paying big performance penalty. But if you will split it to file per node, overhead of file/xml processing will be even bigger. Please keep it in mind - it would be bad to spend a lot of time only to later discover than non-trivial scene requires 10 minutes to deserialize…
I see your point. I guess I’m wondering if splitting them up (and taking the perf hit) would give the Xith3D community an opportunity to create some really nice tools. For instance imagine the following scenario happening over the next couple of years:
1> The XML formats get defined and readers/writers are coded.
2> Someone creates a tool to allow you to visually create textures and write out the XML files using #1.
3> Texture libraries become commonly available because of #2.
4> Someone creates a tool to allow you to visually create appearance objects and write out the XML files using #1.
5> Appearance libraries become commonly available because of #4.
6> Someone creates a tool to allow you to visually create object geometry and write out the XML files using #1.
7> Geometry libraries become commonly available because of #6.
8> Someone combine the tools and we have a full fledge visual modeling environment to work with like Lighwave3D Modeler.
The key here, at least in my mind , is to break things into small enough pieces to allow small tools to be created quickly. Another interesting possible side effect might be loaders that function over the network.
I realize this is all “pie in the sky” stuff at this point. I’m simply trying to think through the implications of the possilbe design choices.
I understand the idea, but I’m not sure if it is a way to go. We need following things - loaders for various formats, plus ability to serialize and deserialize scenegraph from/to xith3d. You seem to think about writing plugins for various programs which would allow them to output serialized version of scenegraph directly. Is it needed ? Loaders need to be written anyway, to work with legacy data, is there a reason to write separate plugin for each possible modelling program out there ?
I do not see major benefit of designing yet another public model format for xith3d. Tools out there work a lot better with their native formats. Scenegraph serialization is needed, because sometimes preprocessing/converting data takes a lot of time. But personally, I don’t think that anything outside xith3d should be aware how this format looks like. I also don’t think that backward/forward compatibility of file format is really needed. To achieve that, you would either need to abstract away xith3d internals (this creating generic scenegraph format and there are plently of them out there, most notably http://www.web3d.org/x3d.html, with java3d loader at http://www.web3d.org/TaskGroups/source/xj3d.html) or stop evolving xith3d in any major way.
I do see a need for one kind of xith3d-specific file format - this is model geometry which can be directly memmapped from disk. But this is certainly different thing from what you want to achieve, as it would be not portable even between bigendian and littleendian machines.
OK, I was already using XML to dump Xith3D scenegraph to file, but this was never intended to act as serialization/deserialization, but was a kind for debugging tool used to visualize actuial programmatically-constructed scenegraph in XML viewer.
The idea was to get an easy overview of the scenegraph in human-readable format.
[quote]I was mainly worried about things like: GeometryTranslocator and SharedGroup which I don’t really understand the implications/use of.
[/quote]
GeometryTranslocator is lamost nothing different than Shape3D with actual geometry constructed at run-time basing on reference geometry and set of transform groups.
For SharedGroup, it always accessible via Links, and should act as any other shared object (texture, appearance, etc.)
As of object identification, you can use scenegraph path concept to identify any piece of scenegraph. Think about groups as indexed arrays of their childs, and node components as attributes [which may also have attributes]. In this case you can serialize the 1st occurence of node/component in-place, mark it, and for all subsequent occurences place references in form of path, like
coloring=root.childs[2].childs[1].childs[2].appearance.coloring
[quote] In this case you can serialize the 1st occurence of node/component in-place, mark it, and for all subsequent occurences place references in form of path, like
coloring=root.childs[2].childs[1].childs[2].appearance.coloring
[/quote]
Maybe XPath can be used ? It was designed exactly for that reason http://www.w3.org/TR/xpath
small change and you could be compatible with another standard (which has an benefit of being able to use already written libraries for parsing/searching xml).
I would personally suggest sticking to numbers Simple implementation, easy to check and search in file by hand without counting children order, IMHO faster to save and load.
As of numbers, in case of human-readable forms, I would prefer to stay with names (even for child indexes within the groups), so xml then can be human-editable (not only readable).
[quote]As of numbers, in case of human-readable forms, I would prefer to stay with names (even for child indexes within the groups), so xml then can be human-editable (not only readable).
[/quote]
Exactly for the reason of human editability, numbers are better way
Imagine situation where you have 20 groups below the root. Now you want to delete one of these groups by hand, by just deleting entire node with subnodes. Simple ? Not, because in every group after your current one, you need to renumber child number by one down. With numbers, you don’t care - they will be just not continious now, which is not a problem.
Do you want to add another node ? With numbers, you can add it anywhere, with paths it has to be at end - but it can be done. But now imagine that you have found geometry data you want to duplicate. With numbers, you just copy/paste it’s id number. With path, you need to go from geometry to root, counting every subnode and determining at what place it is in group, constructing long and typo-prone string.
Additionally, if you allow string ids (with number ones being just a special case for automatic generation of missing ones), it becomes even more clear - “AppearanceGoldMetal” is a lot better than “/root/childs[2]/childs[1]/childs[2]/appearance”. But even with just integers, it is IMHO a lot easier to find 82378912 with editor in file to see appearance data then to count childs[2]/childs[1]/childs[2].
All this is just about hand editing, which IMHO is not a design criterium which is important, but IF it is, then I would suggest going for at least integer ids and even better string ids.
Actually, the question we are discussing is just namespace/aliasing question. As in Java, system may allow accessing children either by path or by alias. The basic idea behind path concept was to be able to define shared node in-place, nested into other components, and be able to reference to it.
But all of this introduces additional complexity, and maybe it makes sense to look at X3D (XML-based alternative for VRML), and maybe just dump Xith3D scenegraph to X3D-compatible document.
See http://www.web3d.org/x3d.html for more details. Also I know they were working on Xj3D, Java3D-based player for X3D, so porting it to Xith3D would be really nice and practical step for wider Xith3D recognition as possible replacement for Java3D.
Yuri
P.S. I was not visiting that site for relatively long time and now recognized that X3D already promoted to be ISO standard as ISO/IEC 19775:200x, 19776:200x, 19777:200x.
Okay I’ve got the first version working with shared objects. What I need to know now is which of the following list of objects should be able to be shared. I’m sure there isn’t anything in Xith3D stopping any of them but I want the XML to produce something that isn’t “wrong”.
The items that have “true” next to them are the objects I currently have picked as sharable.
Note I’ll worry about a version that outputs X3D XML format when I get this all working with my format.
You will be able to produce XML starting any place in the graph. This means you could produce an entire scene graph or specialized pieces like, Shape3Ds, Lights, Appearances, Textures.
You have the choice of inlining all objects or making them shared. Shared produces much smaller files when you have objects used in more than one location in the graph.
, is to break things into small enough pieces to allow small tools to be created quickly. Another interesting possible side effect might be loaders that function over the network.