I thought i had this working… but drawing it to the screen is not yield the sort of line i would expect.
Unfortunately this is a pre-cursor to raycasting.
I have been working on it for a few days with little to no progress.
I would welcome if anyone can see something that is wrong with this unproject method.
thanks!
j.
public static Vector3f unproject(int mouseX, int mouseY, float depth){
Vector3f mouse3DPosition = new Vector3f();
IntBuffer viewport = BufferUtils.createIntBuffer(16);
FloatBuffer modelView = BufferUtils.createFloatBuffer(16);
FloatBuffer projectionView = BufferUtils.createFloatBuffer(16);
float winX = mouseX;
float winY = mouseY;
FloatBuffer position = BufferUtils.createFloatBuffer(3);
// poplulate matrixies
GL11.glPushMatrix();
GL11.glGetFloat(GL11.GL_MODELVIEW_MATRIX, modelView);
GL11.glGetFloat(GL11.GL_PROJECTION_MATRIX, projectionView);
GL11.glGetInteger(GL11.GL_VIEWPORT, viewport);
winY = viewport.get(3) - mouseY; // due to inverted coords
GLU.gluUnProject(winX, winY, depth, modelView, projectionView, viewport, position);
//position.rewind();
mouse3DPosition = new Vector3f(position.get(0), -position.get(1), position.get(2));
GL11.glPopMatrix();
return mouse3DPosition;
}