Trouble converting 2d mouse coordinate into 3d ray.

Newbie & Debugging Questions
#1

Hello!

I am trying to convert 2d mouse coordinate into a 3d ray to do some object-picking.

However, I am having some issues. With my current implementation, it appears as though the direction of the ray changes when the position of the camera changes.

For example. If I look straight down, and click in the center of the screen at world-position ( 0, 0 ), then the ray vector is close to ( 0.01, 0.01, -0.998 ) :point: Just an example

If I re-test at world-position (200,200), then the ray vector is close to ( 0.03, 0.03, -0.95 ).

It’s not much of a change but it’s there and leads to annoyingly imprecise object selecting.

Here is my current code:

int mx = Mouse.getX();
int my = Mouse.getY();

// Calculate mouse coords in NDC space
Vector3f mCoords = new Vector3f( mx / (float)Display.getWidth(), my / (float)Display.getHeight(), 1.0f );
mCoords.x *= 2;
mCoords.y *= 2;
mCoords.x -= 1.0;
mCoords.y -= 1.0;

// Calculate inverse matrices
Matrix4f projMat = cam.getProjectionMatrix();
Matrix4f viewMat = cam.getViewMatrix();
Matrix4f iProjMat = (Matrix4f) new Matrix4f( projMat ).invert();
Matrix4f iViewMat = (Matrix4f) new Matrix4f( viewMat ).invert();

// Put coords from NDC space into view space
MatrixUtils.mulProject(mCoords, iProjMat, mCoords);

// Put coords from view space into world space
MatrixUtils.mulProject(mCoords, iViewMat, mCoords);

// Normalize
mCoords.normalise();

// Print coords
System.out.println(mCoords.toString());

MulProject:

public static Vector3f mulProject(Vector3f left, Matrix4f right, Vector3f dest) {
	float invW = 1.0f / (right.m03 * left.x + right.m13 * left.y + right.m23 * left.z + right.m33);
	dest.set((right.m00 * left.x + right.m10 * left.y + right.m20 * left.z + right.m30) * invW,
			(right.m01 * left.x + right.m11 * left.y + right.m21 * left.z + right.m31) * invW,
			(right.m02 * left.x + right.m12 * left.y + right.m22 * left.z + right.m32) * invW);
	return dest;
}

I am certain that my matrices are the same ones used for rendering, and I am certain that the rotation of the camera has not changed between the two points.

#2

Figured it out. I was being stupid!

I forgot to subtract the cameras position from the coordinates before normalizing :slight_smile: