Mouse-to-World Coordinates and Pick Rays???

OpenGL Development JOGL Development
#1

:-[ I need help.

I’ve looked for many hours trying to find information on how to convert Mouse coordinates (AWT) into World Coordinates (JOGL). I’ve tried OGL’s selection meathods, I’ve tried gluUnProject(), and none of it works. I’ve been looking lately on how to mathmatically do it my self with basicaly no success (I can move the ‘camera’ around, or any other object based on mouse movement in a direction (vectors) , but I can’t actually pick a point in space. Which is what I need to do.)

I then heard about what sounds like a most wonderful technology. Raytracing. Supposedly after getting a mouses (x,y) coordinates you can convert these to world coordinates (x,y,z) and then shoot a ray in a certain direction. After which you can check for intersection with objects, among other things.

Here is my current problem. I can’t find any relevant information on how to convert mouse-to-world coordinates. I would like to do this so I can use ray picking rather then the other meathods. Ray picking sounds more powerful, flexible, and better suited to what I want to do.

If someone would be so kind as to point me in the right direction or better(post some code ;D), then I would be most grateful. Ofcourse I will keep working on it myself, it seems to be a well guarded secret (almost like nobody wants to share :-[ ). I’ve been under the impression it has something to do with the inverse of the model view matrix but I’m not sure how to get that.

Thanks for your help(in advance). I hope to bug you in the future.

#2

Here is two threads about this topic:
http://www.java-gaming.org/cgi-bin/JGNetForums/YaBB.cgi?board=LWJGL;action=display;num=1067556350
http://www.java-gaming.org/cgi-bin/JGNetForums/YaBB.cgi?board=LWJGL;action=display;num=1067194724

#3

Thank you tom.

Oddly enough I usually skip over information that isn’t either directly OGL or JOGL related so I wasn’t looking in the LWJGL forum, eh. Unfortunatley I seem to get some errors with the code. As a reminder I am usuing JOGL not LWJGL.

I believe some things in JOGL do not function the same way in other bindings and vice versa. If you could assist me with JOGL specific code I would be most apreciative.

#4

I’ve personally never used gluUnProject or the selection buffers but the sources to the gleem library are shipped in the jogl-demos sources on java.net. This library performs ray casting and picking using its own linear algebra library rather than relying on OpenGL and is used to provide all of the mouse interaction in the JOGL demos.

#5

I’ll check gleem out.

I have discovered that gluUnProject just isn’t working for me. There has to be another way. I mean if gluUnProject is the only way then I am going to have a problem.

Thanks for the help so far.

#6

All gluUnproject does is calculate the inverse matricies for you and then multiply your supplied point by them to get the resulting world position. Its works fine and is the ‘proper’ mathmatical way of doing this kind of conversion.

If you’re getting wrong results out, it’ll be because you’re putting the wrong results in, or calling it at an inappropriate time. Post some code, ‘it doesn’t work’ is not a constructive answer. :stuck_out_tongue:

Edit: and typical usage of gluUnproject simply involves grabbing the relevant values from glGet*() commands and piping them straight though, its about as foolproof as it gets. :slight_smile: If thats not what you’re doing and you’re trying to supply the values yourself thats probably whats tripping you up…

#7

Orangy,

I think I am trying to pass values in myself. Problem is I’m not totally sure. Since I don’t know what gluUnProject’s job is (actual math behind it) I can’t know if what I pass to it is going to work.

I don’t think I am calling it correctly either. Every time the mouse moves on the screen I want to have a ray being projected from the mouse (x, y). I am assuming the code goes in the mouse move function.

I will post code: but don’t laugh I am new to this stuff, never got into 3d input.
Sorry about the mess.

private CVector3 unProject(int mouseX, int mouseY, int mouseZ)
      {
            // I call this mess everytime I move the mouse to a new position on screen.
            int[] viewport = new int[4]; 
            double[] modelview = new double[16];
            double[] projection = new double[16]; 
            float winX, winY, winZ; 
            double[] posX = new double[1], posY = new double[1], posZ = new double[1]; 

            gl.glGetDoublev( GL.GL_MODELVIEW_MATRIX, modelview ); 
            gl.glGetDoublev( GL.GL_PROJECTION_MATRIX, projection ); 
            gl.glGetIntegerv( GL.GL_VIEWPORT, viewport ); 

            winX = (float)mouseX;
            winY = (float)viewport[3] - (float)mouseY;
            winZ = mouseZ; // mouseZ is 0.0

            // In JOGL aparently I need double[] to be passed where posX, posY, posZ are passed
            // This apears to make gluUnProject work but then when I get the return values
            // They are all 0. I know this probalby looks silly to someone who knows what gluUnProject
            // does. And although someone might explain what I'm doing wrong I would like to know
            // how to do it without gluUnProject. I know it might be pointless, but I am not doing it
            // to get something done.  I am doing it to learn.

            glu.gluUnProject( (double)winX, (double)winY, (double)winZ, modelview, projection, viewport, posX, posY, posZ);

            CVector3 mouseTemp = new CVector3((int)posX[0], (int)posY[0], (int)posZ[0]);

            // No matter what I do, these values are always 0.
            System.out.println("Check Values: " + posX[0] + ", " + posY[0] + ", " + posZ[0]);

            return mouseTemp;
      }
#8

Looks ok for the most part, although I’d be slightly suspicious of using a winZ of 0, trying it with 1 or larger might be an idea.

More likely however is where you’re calling it from. Any and all operations that use the current GL state (ie. glGet*) have to be performed in the rendering thread (probably the one created by the Animator you’re using or similar). The AWT event thread that triggers the mouse move/clicked events isn’t allowed to tinker with the GL state.

Most of the time this isn’t a problem, but mouse interaction is one of the most annoying situations. You’ll have to queue the mouse event somehow and process it in your rendering loop after your model and projection matrices are setup.

Hope that helps.

#9

Two things were preventing coordinate translation from working. I changed winZ to 1.0 and called the unProject from the rendering thread.

Now I have a new problem. Although the conversion was successful the coordinates generated shoot way off my screen. Is there any way I can correct this?

#10

Can I hazard a guess that you’re using one projected mouse coord and creating a ray from the camera position to the returned world coord? That’ll get the angle wrong, try projecting two points with winZ of (say) 1f and 2f, then create a ray passing though the two.

If you’re already doing that, then I’m out of ideas. ???

#11

I knew I’d have to bug you guys again. In JOGL I don’t know if I’m dreaming or somthing but If you put a value higher or lower than 1f for winZ in gluUnProject then it just doesn’t return any coordinates? Is there something I should be doing?

Here is some code:

      private void unProject(CVector3 mouseTemp, int mouseX, int mouseY, int winZ)
      {
            // I call this mess everytime I move the mouse to a new position on screen.
            int[] viewport = new int[4]; 
            double[] modelview = new double[16];
            double[] projection = new double[16]; 
            float winX, winY; //winZ; 
            double[] posX = new double[1], posY = new double[1], posZ = new double[1];
            //CVector3 mouseTemp = new CVector3(0,0,0);

            gl.glGetDoublev( GL.GL_MODELVIEW_MATRIX, modelview ); 
            gl.glGetDoublev( GL.GL_PROJECTION_MATRIX, projection ); 
            gl.glGetIntegerv( GL.GL_VIEWPORT, viewport ); 

            winX = (float)mouseX;
            winY = (float)viewport[3] - (float)mouseY;

            //winZ = 1.0f; // mouseZ is 1.0

            gl.glPushMatrix();
            glu.gluUnProject( (double)winX, (double)winY, (double)winZ, modelview, projection, viewport, posX, posY, posZ);
            gl.glPopMatrix();

            mouseTemp.x = (int)posX[0];
            mouseTemp.y = (int)posY[0];
            mouseTemp.z = (int)posZ[0];

            //System.out.println("unProject: " + mouseTemp.x + ", " + mouseTemp.y + ", " + mouseTemp.z);
      }

I need this to work so I can do what Orangy said. I need 2 winZ values to get my pick ray to work.

#12

What appears to be happening actually is that every time I call gluUnProject it sets all my mouseLoc[x] to the latest coordinate collection. I am lost. I need to find out how to get a second winZ without resetting my last mouseLocation in 3d.

#13

What do gluUnProject return at different winZ?

And pretty please, don’t store the values as ints. Far all we know, they might be less than 1 and are all cast to 0.

#14

At any winZ above or below 1.0f it returns all zero’s except for the z value which is usually returned as 10. I believe I may be misunderstanding what is going on.

However if it’s doing it’s job correctly I don’t think I am getting the result I want.

In any case I have a bigger problem. Every time I call gluUnProject even if I store the result in a different variable it simply returns the last result for all the the variables I called gluUnProject for.

for example:

I have two variables which are vectors with 3 values stored x, y, z called mouse1 and mouse2.

when I call gluUnProject like this (Using psuedo code)



display(){
// do open gl stuff here

drawstuff(); // draw a quad or something

afunction(mousex, mousey);

glFlush();

}

afunction(mousex, mousey){

Create needed variables;

gluUnProject winZ set to 1.0 to get (x,y,z);

store result in mouse1(x,y,z);

// Do I need to do somthing here???

gluUnProject winZ set to 2.0 to get (x,y,z);

store result in mouse2(x,y,z);

}

I get the results from the last gluUnProject call stored in both mouse1 and mouse2. This makes no sense to me.

This is happening using the code I posted above. And I can post any relevent (actual) code if requested.

#15

I think the defferance between the values returned from gluUnproject is so smal that when they are cast to int they are the same.

DO NOT use CVector3 to return the result from your unProject function. Use this instead (or something else as long as it uses floats or doubles):


public class Vector3f {
   public float x, y, z;
}

This is very important!!!

If it still don’t work, post the real code. It sounds that you are not using 2 different objects, and that the first one is overwritten by the second.

#16

Well I found a solution, though annoying. I simply store the values into another variable after calling each gluUnProject so that they don’t become 0, 0, z(varies).

Thanks for all your help guys. I’ll be picking your brains later. Believe it… :smiley: