Picking enhancement

[robertMerkle]

Hi,

I’ve been tracking/toying with Xith for some time now,
and like it a lot.
I would like to propose to enhance Xith3D picking somewhat.

Currently Xith3D uses the standard OpenGL picking mechanism with a select buffer,
and picking by Node.
PickRenderResult.getNodes() lists the (I think: single) Node picked
and a zmin, zmax value.

This use of OpenGL picking is quite specific to picking Node’s and:

• does not allow identification of primitives (which quad of the
  immense quadarray has been picked?)
• does not allow the application to provide (multiple) pick names
  (the cell picked has this cellId, this segmentId, this clusterId etc.
  Admittedly I haven’t seen a number of identification codes > 3,
  and we packed them into a single int. Works, but caused a bit of a range
  limitation )
• any future node predraw hooks for DIY OpenGL drawing cannot get
  custom pick results back.

The proposal below attempts to remedy this and keep on using standard OpenGL picking.

• add to Node:
  // when true, the first name on the stack will be the primitive index.
  // this allows getting (close) to the picked coordinates, for applications that do
  // not want to deal with custom names.
  
  // geoArray.getValidVertexCount()/# vertices_per_primitive for normal geometry
  // igeoArray.getValidIndexCount()/# vertices_per_primitive for indexed geometry ???
  </not thought through>
  boolean pickPrimitiveIndex = false;

  // the number of names that this node can use for identifying its pick result
  // value 0 means that only the default node picking is available.
  int customNameStackDepth = 0;

• change PickRenderResult
  private Node node; // should be just one Node + all of its sub-identification,
  private int[] names; // so complete contents of name stack at pick time - 1
  // (the name used to identify the node itself)

• to allow multiple custom names per primitive of a Node, add to GeomContainer
  // array of size nameStackDepth, containing the pick names per primitive
  
  // the pickNames could be java int arrays, subject to GC
  </not thought through>
  GeomDataInterface[] pickNames;

• add GeomNioIntData (similar to GeomNioFloatData) for use as pickNames

The biggest change is with rendering: with custom names per primitive, we can’t use
glDrawArrays or glDrawElements for drawing.
When in pick mode && custom picking is enabled, drawing needs to revert to drawing
per primitive:

• if (pickPrimitiveIndex) push current primitive index
• push all custom (customNameStackDepth) names on the stack
• draw primitive, using the appropriate glArrayElement() call
• pop all custom names off the stack
• if (pickPrimitiveIndex) pop

Thus, ShapeAtomPeer needs to be aware of the pickMode.
Additionally proper checking of the pickNames data should be done,
customNameStackDepth + #internal names should be checked to be < GL_NAME_STACK_DEPTH,
(allthough this is not likely to be a problem as GL_NAME_STACK_DEPTH >= 64 …)
convertSelectBuffer would need to be changed etc…
But in the end this would provide a very flexible (configurable per Node) picking mechanism.

What do you think?
Does this conflict with other developments in the pipeline?

Cheers,
Robert

[DavidYazel]

I have always thought that opengl screen picking was a sort of stop gap for a more advanced scene based picking system, which is currently under development. I am not sure I am entirely comfortable with the type of changes you are suggesting, but Yuri is the OpenGL picking guy, so we should get his opinion.

[joo]

I don’t want to comment on the exact changes, but shochu and I had problems with implementing a movement system using Xith3D (see this thread). Hopefully a more sophisticated implementation of picking can address these issues.

[yvg]

Hello,

I confirm David’s information that OpenGL picking is just one of picking techniques to be available in Xith3D and was implemented as fast [in development time sense] solution for object picking in Xith3D.

Next versions of Xith3D will include alternate techniques of object picking, and primary task is to include Java3D-style picking capabilities, including the ability to get geometry-specific parameters, such as vertex and texture coordinates, normals and colors at the object intersection point. Non-ray picking shapes are to be supported, too. Additionally, such programmatic picking techniques will allow to perform pick tests on separate thread and eliminate issues related to OpenGL thread management and rendering thread synchronization.

I already was starting once an attempt to port Java3D picking code to Xith3D, but after figuring out the amount of work decided to implement OpenGL-based picking, which is enough for many [but not all] applications.

Identification of specific geometry primitives is to be implemented using programmatic (non-OpenGL) picking techniques, at least because of OpenGL picking anyway has a lot of limitations: a) MUST be executed on rendering thread, causing jitters and slowdowns in animations; b) has numerous driver-specific problems, especially on non-Windows operating systems; c) does not solve a problem of getting 3D coordinates of the intersection point.

The idea with supplying custom nodeIds for every specific elementary component [quad, triangle, line, point] is nice, but I see it causing too big changes in rendering process - as Robert already mentioned, renderer will not be able to use glDrawArrays, etc., and will MUST render all the geometry per individual primitive during picking pass, which, being executed on the Rendering thread and between rendering visible frames, will [OK, may] cause [probably] bigger delays and frame skips, resulting in non-smooth animations during the mouse clicks. Also this technique may cause some questions when used in conjunction with OpenGL evaluators and Vertex Programs [which are to be added to Xith3D].

So, besides of I developed OpenGL-based picking code, I treat is as a solution targeted to simple Node-based picking applications [OK, and also as a temporary solution], and will vote for sophisticated programmatic picking functionality.

Anyway, all the contributions to Xith3D are really welcome, and I think we are not limited to having one and only picking technique in Xith3D.

Yuri

[robertMerkle]

Thanks for your explanation Yuri; I can see why you’d prefer the picking not to clutter
up the rendering thread.
My particular interest is volume (simply outer hull) and cross-section visualization of lots of
cells, where, after having created the shapes for rendering, the only connection back to the
underlying data structures is really only the index of each primitive in the shape’s
geometry array.
Based on a pick result intersection point, and/or closest vertex alone it is very
clumsy and time-consuming to determine the exact primitive by doing all sorts of ‘isInside’
tests
Will the programmatic picking you guys are developing support giving back the primitive
index in the pick result? Would solve all my picking issues, and actually there’d
be no need to bother the renderer with custom pick names.

Remains possible gl picking support for DIY jogl drawing in future shape predraw hooks.
It may still be a good idea to alter PickRenderResult as mentioned above, so
to contain a single Node + the pick names, minus the one used for node id.
I like the idea of toying will jogl knowing that the bulk of the transforms is dealt with by
the scene graph.

Thanks,
Robert

[yvg]

[quote]Will the programmatic picking you guys are developing support giving back the primitive index in the pick result?
[/quote]
I hope so [read: I am sure]. Anyway, picking logic have to inspect all the primitives for intersection, so I don’t see the reason of why it should not return the primitive index.

Only question will be to exactly specify how it should work for non-plain geometries, i.e. indexed arrays, strip arrays and fan arrays.

Yuri

[yvg]

Another note:

[quote]Remains possible gl picking support for DIY jogl drawing in future shape predraw hooks.
[/quote]
If you mean programmatic [Java-level] pre-draw hooks, I see no pbs. If you mean Vertex Programs and Evaluators - this is something different… and I don;t know how GL_SELECT interacts with that. Could be that GL_SELECT is only a way of picking geoms generated/transformed by Vertex Programs… but we will see this very very soon.

Yuri