Cube outline is see-through

OpenGL Development
#1

I was working on outlining 3D cubes in OpenGL and I wanted depth testing to be enabled when drawing the outlines, so that other cubes wouldn’t render on top of the outline. When I look to the right using the virtual camera, the stencils are drawn perfectly with the depth testing enabled.

However, when I look to the left, the depth testing for the outlines doesn’t really work and you can kind of see through the outlines. I’m using stencil testing for the outlines. The outlines are black.

This is how it looks when I look to the right:

And this is how it looks when I look to the left:

EDIT: The depth testing for the outlines doesn’t work if I look down (X:0, Y:-1, Z:0) or backwards (X:0, Y:0, Z:1).

#2

Do you need to sort the drawing of the cubes based on distance to the camera?

#3

orange451 I draw all the cubes, I don’t sort them based on the distance to the camera.

The depth testing for the cubes themselves works. But the outlines are see-through.

#4

How are you trying to draw outlines? From the looks of it you’re drawing first a cube and then a slightly scaled up black cube for the outlines, correct? I have an idea what the problem could be, but I’ll need to see your code to be sure.

#5

You’re right! That’s exactly what I’m doing, theagentd.

#6

Still need to see code to see what stencil function and draw order you’re using.

#7 

GL11.glEnable(GL11.GL_CULL_FACE);
GL11.glCullFace(GL11.GL_BACK);
GL11.glEnable(GL11.GL_DEPTH_TEST);
GL11.glEnable(GL11.GL_STENCIL_TEST);
GL11.glClear(GL11.GL_COLOR_BUFFER_BIT|GL11.GL_DEPTH_BUFFER_BIT|GL11.GL_STENCIL_BUFFER_BIT);
GL11.glStencilOp(GL11.GL_KEEP, GL11.GL_KEEP, GL11.GL_REPLACE);
GL11.glClearColor(backgroundColour.getRed(), backgroundColour.getGreen(), backgroundColour.getBlue(), 1);

for(Block b : world.getBlockList()){
				if(b != null){
					GL11.glStencilFunc(GL11.GL_ALWAYS, 1, GL11.GL_TRUE);
					GL11.glStencilMask(0xFF);
					renderer.render(b, blockProgram, cam);
					if(!b.getColour().equals(goalColour)){
						b.scale(b.getScale().getX() + 0.00002f, b.getScale().getY() + 0.00002f, b.getScale().getZ() + 0.00002f); //this is intentional
						GL11.glStencilFunc(GL11.GL_NOTEQUAL, 1, GL11.GL_TRUE);
						GL11.glStencilMask(0x00);
						b.scale(b.getScale().getX() + 0.002f, b.getScale().getY() + 0.002f, b.getScale().getZ() + 0.002f);
						Colour colour = b.getColour();
						b.setColour(Colour.BLACK);
						renderer.render(b, blockProgram, cam);
						b.setColour(colour);
						b.scale(b.getScale().getX() - 0.002f, b.getScale().getY() - 0.002f, b.getScale().getZ() - 0.002f);
					}
				}
			}

I’m looping through the blocks in the world, as you can see. And also, the blocks are supposed to increase in size every frame, so the outlines are adjusted as well. I really need a solution :’(

EDIT: I am actually clearing the colour, depth and stencil buffer bits, I just forgot to include that. I’m enabling the depth and stencil test too.

#8

GL_TRUE is not a “valid” input to glStencilFunc(), but in this case it works since GL_TRUE=1. I’m gonna assume you’re setting glStencilOp() to something sane before that loop. My guess at what’s happening is that you mark pixels belonging to a cube with the value 1 in the stencil buffer and then try to draw the outline on pixels that are not equal to 1. This means that every time you draw a cube, the pixels it’s covering will be marked to 1 for all following cubes too, so even if the outline of a later cube is closer to the camera it will still be removed since the pixel is marked with a 1. What you need to do is to reset the stencil buffer between each cube so that it is all 0s again.

You can do this by adding a glClear(GL_STENCIL_BUFFER_BIT) after you’ve drawn each box, but this will get slow. Still, try this to confirm that you’re on the right track. It’d be faster to use the entire 8-bit precision of the stencil buffer and first mask with 1, then 2 for the second box, 3 for the third, etc until you reach to 256, when you clear the stencil buffer to 0 again. That way you could draw 255 boxes between every clear, but you’d still have a rather big number of clears if you draw lots of boxes.

A faster but less precise way of doing this would be to reset the stencil buffer to 0 when drawing the outline. In the marking pass you mark the pixels with a 1, and in the outline drawing pass you reset them back to 0 again. Since the outline is bigger than the original model, all pixels will get changed back to 0. This could be done like this:


//glStencilMask() is always set to 0xFF here.

//Render cube
glStencilFunc(GL_ALWAYS, 1, 0xFF);
glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE); //Mark all pixels that the original box touches the box regardless of depth testing.
renderCube();

//Render outline
glStencilFunc(GL_NOTEQUAL, 0, 0xFF);
glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE); //Reset all pixels to 0
renderCube();

This has some major drawbacks. It will have artifacts if the box and/or its outline intersects the near plane (pixels won’t be reset back to 0), and it also won’t work with complex (concave) models that overlap themselves in any way as the first overdraw will reset it to 0 and the second overdraw will be allowed to place the outline over the model itself. For cubes however this is the fastest way.

The best solution by far would be to just use a depth-detecting shader to compute an outline per pixel. This is what most games nowadays (like Borderlands) do.

#9

You are awesome. I have nothing else to say. Thanks for taking the time to type all that out :smiley: I really appreciate it…geddit?

Anyway, I tried your different methods but none of them really seemed to work. I played around with the stuff, tried your different solutions, but they either gave the same results, or the entire thing just started bugging out and lagging until it became really slow.

I’m going to keep trying to solve it from different angles. I figure it’s something got to do with either the projection matrix or the depth buffer. Thanks, anyway. I really, really appreciate it, because most people don’t take as much time as you do to help people out.

#10

Adding a stencil clear between every cube+outline draw should solve the problem. Hmm. What do you get if you add that?

#11

It works now. Thanks, man!