2D over 3D mouse question

[ahottev]

Hello,

I have this Canvas3D (using Java3D) with my 3D scene in it, with a PickMouseBehavior, and then a JPanel on the side for debugging. Thing is, i also have some 2D images on top of my canvas3D (using postRender(), think of it as an overlay menu) which also has to be able to read the mouse input, but only if the mouse is over a 2D image, so that it won’t go through the 3D picking instead.

How do you do this?

I figured i could somehow hack the picking behavior, but i’m not sure it’s the best way. Any help or suggestion appreciated!

Thanks!

edit: I’m thinking of creating a class called Button2D containing the position and rectangle of each buttons in my overlay, add them in an ArrayList, pass it to the mouse picking and test it against the cursor’s position before the actual 3D picking. If the cursor is withing bounds, return the Button2D’s ID.

I guess it could work?

edit2: Well, it’s working fine I guess i just needed to think “aloud”, heh. Anyway, if there’s better solutions, i’d still like to hear them.

[SimonH]

@ahottev

edit: I’m thinking of creating a class called Button2D containing the position and rectangle of each buttons in my overlay, add them in an ArrayList, pass it to the mouse picking and test it against the cursor’s position before the actual 3D picking. If the cursor is withing bounds, return the Button2D’s ID.

That would work. Another option would be to place your 2D images in the 3D scene as planes facing the camera (so they appear 2D). That way the poly-picker could find them.

[ahottev]

@SimonH

That would work. Another option would be to place your 2D images in the 3D scene as planes facing the camera (so they appear 2D). That way the poly-picker could find them.

Oh yeah, i thought about that, but it would have been complicated to keep the plane at exactly the right position since you can move, rotate and zoom in every directions…

[lhkbob]

See if Java3D has something to help with billboarding, if they do, it should be pretty easy to set it up so that an object always faces the camera.

[tom]

You can add it to the view transform group. Then it will follow the camera. However placing the object at the correct position can be tricky. Especially if you want to support the ultra complex Java3D view model. Also you will have problems with the hud being rendered behind the geometry in world. You’d have to make the hud elements transparent and disable depth test. Even then it can be drawn behind transparent objects in the world. That can be fixed by creating your own transparency sorter (com.sun.j3d.utils.scenegraph.transparency).

The problem with the way you’re doing it now is that:

1. it is slow
2. the size of the hud your rendering is fixed to the size of the window when it started. If you try to resize the window you will see that it stretces. This makes it impossible to create pixel perfect huds.

Creating a hud in Java3D is impossible without it having some negative side effects.

[ahottev]

@tom

You can add it to the view transform group. Then it will follow the camera. However placing the object at the correct position can be tricky. Especially if you want to support the ultra complex Java3D view model. Also you will have problems with the hud being rendered behind the geometry in world. You’d have to make the hud elements transparent and disable depth test. Even then it can be drawn behind transparent objects in the world. That can be fixed by creating your own transparency sorter (com.sun.j3d.utils.scenegraph.transparency).

The problem with the way you’re doing it now is that:

1. it is slow
2. the size of the hud your rendering is fixed to the size of the window when it started. If you try to resize the window you will see that it stretces. This makes it impossible to create pixel perfect huds.

Creating a hud in Java3D is impossible without it having some negative side effects.

Ouch, yeah i saw what you meant. Maybe i’ll just set it to non-resizable for now, though. OTOH, is post-rendering that slow? Is is slower to draw lots of pixels each frame than calculating a few vertices + material of the billboard each frame (should still be the same number of pixels…) ? Currently, i’m not noticing any FPS drop, but i guess that’s due to not having an FPS counter, which i’ll be adding right away.

Honnestly, i’ll probably switch to openGL for my next project, but right now, i guess it’s just too late.

[tom]

“draw lots of pixels” also means uploading it to the graphics card. This is slow. When using a billboard the texture is uploaded ahead of time and this is fast.