Does anyone know how to do this or know of a good tutorial? I’m trying to implement a very basic version of neverwinter nights movement system. It might be a little too tough for me at this point but I’d like to give it a shot. THank you very much. (I know I’ve been posting a lot recently sorry bout that)
Have you read the GSG tutorial about picking? I don’t know anything about the neverwinter nights movement system, but learning about picking may be a good start.
I think picking will now be big help with converting screen coordinates to world coordinates (if we are speaking about picking implemented using GL_SELECT as it done in Xith3D for now). Picking passes screen coordinates to OpenGL and then renders scene in GL_SELECT mode to get objects that match specified pick area.
To convert screen coordinates to world coordinates it is necessary to unproject screen coordinates using current projection matrix. But this will not give you the point in world coords, but ray - when you click on the screen you are missing 3rd coordinate (Z), so infinite number of points in 3D space correspond to single point in 2D space.
BTW, if you want to pick an object and come closer to it, you definitely need picking - here Jens is 100% right.
I also have no idea about neverwinter nights movement system (“neverwinter” sounds like I heard it somewhere but can not recall).
Please describe task in more details - it will be useful.
Yuri
Neverwinter nights is a 3D third person view. In 2d it would be very similar to an isometric view. THe camera is relatively locked but you can move it just a little bit up and down. The way movement works is you click on the screen, then you’re character runs to that point (in the 3d world). Probably the same principle that any 3d RTS game uses such as Warcraft III. Character movement is done by point and click.
The two main parts of this are determining where the user clicked and where the character needs to move to.
I’ve been able to find lots of great stuff on pathfinding but determing where in the world coordinate system to move the character based on the mouse click is stumping me.
I was thinking about it a little more. Would it work if I converted the 2d mouse click coordinate into the ray that it represents then figure out where the ray intersects the terrain to get my z coordinate? (In my coordinate system the the y axis is “up” and “down”, with z going “in” and “out” and x being left to right. My terrain is super simplistic and is just a flat plane for now.
To be honest though if thats what I need to do I’m not quite sure how to do it.
[quote]To convert screen coordinates to world coordinates it is necessary to unproject screen coordinates using current projection matrix. But this will not give you the point in world coords, but ray - when you click on the screen you are missing 3rd coordinate (Z), so infinite number of points in 3D space correspond to single point in 2D space.
[/quote]
Can you explain a bit more detailed how this can be done in Xith3D? :
The technique should be quite similar to use of gluUnProject function, that maps window coordinates to object coordinates:
[quote]The gluUnProject function maps the specified window coordinates into object coordinates using modelMatrix, projMatrix, and viewport. The result is stored in objx, objy, and objz.
[/quote]
I don’t remember if Java implementation of gluUnProject exists in Xith3D, but this can be easily implemented because of C source of GLU is available, and proting should be trivial.
The only question will be choosing Z window coordinate, which should be I think either 0 or 1 - it’s necessary to check which one is more appropriate.
I also sure that there are examples of using gluUnProject for plain OpenGL programming, which will be useful in this case.
Yuri
Also following thread will be useful:
http://www.java-gaming.org/cgi-bin/JGNetForums/YaBB.cgi?board=jogl;action=display;num=1069089567
Thread “Mouse-to-World Coordinates and Pick Rays???” in JOGL forum.
Yuri
I’m still not fully understanding what to do and how it works (probably I misunderstood it completely). There are actually two possible tasks. The first is that I want to know where I clicked on a certain Shape3D (no matter if it’s visible or not). The second is that I want to find out which Shape3D I clicked and where. Let’s assume I want to do the first.
I have the (x,y) coordinates of a mouse click and a Shape3D. I want to compute where I just clicked on that Shape3D. A mouse position doesn’t have a z coordinate, so I actually compute the intersection of a ray and a Shape3D object. I call this a picking ray. However gluUnProject returns (x,y,z) coordinates (which is obviously not a ray). Does it return the coordinates where the picking ray hits the near clipping plane or what exactly? Why do I need to test which z mouse coordinate is more appropriate (0 or 1)?
If I want to implement this without using Jogl I need the current view and the getLocalToVWorld()-Transform3D of my Shape3D. Is this correct?
(Sorry for the late reply. I don’t have much time these days.)
Hi Jens,
Here’s what I think you need to do. You actually need to call gluUnProject twice. The first time you call it with a value of 0 for winz. This will give you the intersection with the near plane. THen, you call gluUnProject again with a value of 1 for winz. This will give you the intersection with the far plane. Then you just subtract the two values to get the “picking ray”. The origin of the picking ray will just be the location of the view. Once you have the ray you need to check what objects in your scene it intersects. I’m not quite sure how to do this part yet but if I figure anything out I’ll be sure to post it. Hope that helps a little.
Yes, this makes it a bit clearer.
z=0 => near plane
z=1 => far plane
Why didn’t I find this written down somewhere?
Actually there are still two problems: The first is to compute the intersection of a ray and a Shape3D. The second is that I don’t know how to access the modelview, viewport and projection. I thought CanvasPeerImpl.getGL() and CanvasPeerImpl.getGLU() can be used to get GL and GLU objects and I can call “gl.glGetDoublev( GL.GL_MODELVIEW_MATRIX, modelview );” (similar for the other variables), but this just returns zeros.
[Btw. I personally think such a common function should be implemented in Xith3D. I don’t really like having to use Jogl, because I want to be able to exchange Jogl and LWJGL when the LWJGL port is ready.]
I started working on this and I’m having problems figuring out how to get the ModelViewMatrix, ProjectionMatrix and ViewPort. Does anyone know how I can get those values? THanks for the help.
I tried the following but keep getting a null pointer exception on the gl.glGetDoublev line
double [] model = new double[16];
CanvasPeer canvasPeer = universe.getCanvasPeer();
GL gl;
gl = ((CanvasPeerImpl)canvasPeer).getGl();
gl.glGetDoublev(GL.GL_MODELVIEW_MATRIX, model);
Okay so I checked and the gl I’m getting back is null which would explain the problem in the glGetDoublev line.
Of course now I’m stuck trying to figure out why I’m getting a null value returned for gl.
You can access valid GL only from within display(…) method of CanvasPeerImpl. But, anyway, I don’t think this approach is good because of you are trying to use renderer-specific functionality.
You can deduce the projection/model-view matrices from the following fact:
Xith3D JOGL renderer uses following schema to set matrices before rendering:
gl.glMatrixMode(GL.GL_PROJECTION);
gl.glLoadIdentity();
view.getProjection().get(trans);
gl.glMultMatrixf(trans);
view.getTransposeTransform().get(trans);
gl.glMultMatrixf(trans);
gl.glMatrixMode(GL.GL_MODELVIEW);
gl.glLoadIdentity();
and then for every shape sets its Model-View transform using shape.getLocalToVworld().
Yuri
try view.getProjectionTransform().
Thanks for the response Yuri. I just wanted to check to make sure I understand.
For the Projection Matrix:
- I would create an identity matrix A
- Multiplay matrix A by the result of view.getProjection() to get matrix B
- Multiply matrix B by the result of view.getTransposeTransform() to get matrix C
- Matrix C would be the projection matrix
For the modelview matrix
I can take any object in the world and get the model view matrix by using shape.getLocaltoVworld
Is this understanding correct? Thanks for all the help!
Just read your post David … there is no view.getProjectionTransform() but there is a view.getProjection which returns a Tranform3D and I’m trying that out right now. Thanks for the tip.
~shochu
[quote]I just wanted to check to make sure I understand
[/quote]
Looks like OK, but definitely you have to test this
Yuri
I’ve been working on porting the gluUnProject function from the mesa version of OpenGL into my xith program and I’m getting a little confused when it comes to the modelview matrix and using gluUnProject.
As taken from the openGL redbook gluUnProject maps the specified window (screen) coordinates into object coordinates.
From what I’ve read it seems that you use gluUnProject to convert screen coordinates obtained through mouse clicks into world coordinates.
I assumed we could do the following
Call gluUnProject twice to get the picking ray
Once you have the picking ray check where the picking ray intersects the objects in your scene. (In this case I want to know where it intersects the terrain)
However, if you are really getting object coordinates then how does this help you in determing if the picking ray intersects anything? Do you have to use the model view matrix for the object you want to test intersection with?
Thanks in advance for any clarification. ~shochu
gluUnProject will convert the screen coordinates into the space given by the modelview matrix. So if the modelview matrix contains only the camera transformation, you will get what most people would think of as “world” coordinates. If you call gluUnProject just before rendering a mesh, you will get the coordinate in the meshes object space.
A brute force way of checking if your ray hits a mesh is to check if the ray hits any of its triangles. A better way is to use a bsp-tree, but thats off topic When doing this test the ray and the mesh must be in the same space. If your ray is in world space, you would either have to transform the ray into object space, or transform the mesh coordinates into world space. The first option is obviously the fastest in most situations.
So it’s a good idee to have the ray in object space. This means you have to create a ray for each modelview you use. So don’t think of it as one ray in world space. But one ray for each object.
Hope this help.
I’ve got kind of a unique use so let me elaborate (Thanks for your reply by the way it helped a lot)
I’m making a 3D isometric game. (I think that’s an oxymoron) (Think Neverwinter Nights or any 3d RTS ala Warcraft 3) The game is just for learning purposes of course.
For now the terrain is just a perfectly flat plane.
defined by Point(0,0,0) Normal (0,1,0)
When the user clicks on the screen I want to figure out where they are clicking on the terrain so I can decide where to move the avatar.
When the user clicks on the screen with the mouse I grab the screen coordiantes. I then pass these to gluUnProject first with winZ set to 0 then winz set to 1.
From this I can get the picking ray.
I’m a little confused as to what model view matrix I should use. The bottom right corner of the terrain is at point (0,0,0) and then grows from there. Thus I thought that my world coordinates would coincide with the terrain coordinates.
But it seems that I need to use the Modelview matrix of the terrain? If I use the modelview matrix of the terrain then is this giving me a ray which goes from the camera through the point clicked on by the user?
If so then I was planning to then figure out where the ray intersected the plane. When you say that if the modelview matrix only contains the camera transformation would this mean that it the same as the projection matrix? Thanks for your generous help! ~shochu
There is another approach you can take. Using the view transform and the perspective transform you should be able to translate the screen coordinates into world coordinates. Then you could calculate a point which was behind the view but centered on it (eye point) and calculate a ray which went from there and passed through the point you first calculated. I think this would result in a ray you could use to calculate an interection with your terrain. I really havn’t thought deeply about this, but I think it would work.
Hi David,
Isn’t your suggestion pretty much the same as what I’m trying to do now? I’m still having problems but will keep working on it.
On another note,
I ported over the gluUnProject code for anyone that might want to use it. I think it’s pretty useful. I’m pretty sure it works without any problems (the problems I’m having are related to how I use the gluUnProject … i think) I called it xithUnProject.
/**
* Multiplies the 4x4 matrix 'M' by the column vector 'in.' This is a helper function for the xithUnProject method
* @param M
* @param in
* @return out a column vector with 4 elements
*/
private float[] transform_point(Matrix4f M, float[] in)
{
float [] out = new float[4];
out[0] =
M.getElement(0, 0) * in[0] + M.getElement(0, 1) * in[1] + M.getElement(0, 2) * in[2] + M.getElement(0, 3) * in[3];
out[1] =
M.getElement(1, 0) * in[0] + M.getElement(1, 1) * in[1] + M.getElement(1, 2) * in[2] + M.getElement(1, 3) * in[3];
out[2] =
M.getElement(2, 0) * in[0] + M.getElement(2, 1) * in[1] + M.getElement(2, 2) * in[2] + M.getElement(2, 3) * in[3];
out[3] =
M.getElement(3, 0) * in[0] + M.getElement(3, 1) * in[1] + M.getElement(3, 2) * in[2] + M.getElement(3, 3) * in[3];
return out;
}
/**
* Converts the given screen coordinates into object coordiantes based on the given projection and modelview matrix.
* @param winX screen coordinate x
* @param winY screen coordinate y
* @param winZ screen coordinate z
* @param model modelview matrix
* @param projection projection matrix
* @param viewPort viewport
* @param results puts the final object coordinates into the results array where results[0] = x, results[1] = y results[2] = z
* @return boolean stating whether the unprojection was succesful or not
*/
public boolean xithUnProject(float winX, float winY, float winZ, Matrix4f model, Matrix4f projection, int []viewPort, float[] results){
float in[] = new float[4];
float out[] = new float[4];
Matrix4f A = new Matrix4f();
in[0] = (winX - (float)viewPort[0]) * 2.0f / (float)viewPort[2] - 1.0f;
in[1] = (winY - (float)viewPort[1]) * 2.0f / (float)viewPort[3] - 1.0f;
in[2] = 2.0f * winZ - 1.0f;
in[3] = 1.0f;
A.mul(projection, model);
A.invert();
out = transform_point(A, in);
if(out[3] == 0)
return false;
results[0] = out[0] / out[3];
results[1] = out[1] / out[3];
results[2] = out[2] / out[3];
return true;
}
Please let me know if you use it and think something isn’t working properly.