gluUnProject works, but depth is inaccurate?

(apologies, this is really an OpenGL rather than a jogl question,
but I’m using Jogl)

I’m trying to use gluProject/UnProject to determine if a particular vertex is occluded:

  1. Project to the screen, record screen x,y
  2. use glReadPixels to read the depth at this x,y
  3. use this x,y,depth in UnProject to get a world xyz.
  4. Compare this to the original position, if it is sufficiently far away declare the original to be occluded.

[ Comment, it occurred to me that an alternate approach would be to compare the projected depth
available in step 1 to that in the Zbuffer. I did not do this because it would entail figuring out what
a meaningful depth difference on the zbuffer 0…1 scale is, whereas in world coordinates I know this
already ]

This is sort of working, in that the unprojected positions come out
somewhat close to the originals. However it is not working well!
The X,Y are usually quite close, but the Z is off by as much as 2 units,
for a model that is roughly 20 units in size.

I initially assumed that this may be due to quantizing the
projected screen sx,sy location to a pixel location in order to read the depth.
I tried bilerping the depth, but this actually made things very slightly
worse, in terms of the summed reprojection difference over a large
number of pixels. Rounding rather floor in the double->int conversion of the
sx,sy also made things worse (slightly).

In summary, this approach is not accurate enough to determine occlusion!

Can you think of anything I can try to debug this, or suggest other
approaches?

The clipping planes are 0.2,800.
Here are relevant code fragments:



      double[] mmat = new double[16];
      double[] pmat = new double[16];
      int[] viewport = new int[4];
      gl.glGetIntegerv(GL.GL_VIEWPORT, viewport);
      gl.glGetDoublev(GL.GL_MODELVIEW_MATRIX, mmat);
      gl.glGetDoublev(GL.GL_PROJECTION_MATRIX, pmat);
      double[] pproj = new double[3];

	// project this model point to the screen, record its location
	glu.gluProject(v[0],v[1],v[2], mmat,pmat,viewport, pproj);
	//int sx = (int)(pproj[0]+0.5);
	//int sy = (int)(pproj[1]+0.5);
	int sx = (int)(pproj[0]);
	int sy = (int)(pproj[1]);

	// unproject back to the model.  make sure the depth is
	// roughly the same.  If not -> point is OCCLUDED
	float[] adepth = new float[1];
	gl.glReadPixels(sx, sy, 1, 1, GL.GL_DEPTH_COMPONENT,
			GL.GL_FLOAT, adepth);
	double depth = adepth[0];
	//double depth = glUtil.bilerpDepth(gl, pproj[0],pproj[1]);
	double[] xyz = new double[3];
	glu.gluUnProject(pproj[0],pproj[1], depth,
		      mmat, pmat, viewport,
		      xyz);

	// compare original and unprojected
	double dx = xyz[0] - v[0];
	double dy = xyz[1] - v[1];
	double dz = xyz[2] - v[2];

You should read up on the contents of the depth buffer. I’m not sure exactly what it contains but it may contain normalized values between the near and far clipping planes. You might want to use the selection buffer for what you’re trying to do. See this example (may require slight modification) for an example of using the selection buffer from JOGL.

Thanks!

using selection sounds like a better approach.

If I don’t reply, let’s assume it worked out :slight_smile: