Shadows flicker (Shadow Mapping)

Newbie & Debugging Questions
#1

I’m looking to improve my shadows again. Basically the shadows I have work really well (good detail) when I’m only rendering small portions of my scene. Problem is I have a big world so obviously if I try to render the shadow map over the world I get very boxy shadows.

I thought a decent way to remedy this would be to only render the shadow map to a 40x40 area around the camera (I don’t have any decent frustum logic to help with sizes):

		glOrtho(-20.0f + main_cam.getPosition().m_x, 
					20.0f + main_cam.getPosition().m_x, 
				   -20.0f + main_cam.getPosition().m_z, 
				    20.0f + main_cam.getPosition().m_z, 
				    0.1f, 1000.0f);

However, this leads to the shadows flickering when the camera is moving, why does the flickering occur?

#2

I have no experience with shadows or OpenGL, but, when I think of what you’re trying to do, I think that the reason why the shadows flicker is either that the checking takes too long (which I don’t see why it should) OR the following scenario is occurring constantly :

Red circles are out of range objects, the Green one is within range.
If the camera moves up, then the Red circle halfway on the line will become shaded and if this doesn’t happen smoothly, it might look like a flicker.

I might be totally wrong, but I gave it a shot.

#3

Nope, the flickering occurred when objects were all the way in view. Turns out that it had to do with me adding the camera position to glOrtho() and the glTranslate() as well as the near plane being positive. I guess it thought objects were popping in and out of the bounds?

Anyway now I just have flickering at the edges, I assume it’s because the inaccuracy of the shadow map at lower resolutions.

#4

I was right(kinda), probably cause I read this article - https://msdn.microsoft.com/en-us/library/ee416324(VS.85).aspx - within the past month, but I don’t understand this:

“The vWorldUnitsPerTexel value is calculated by taking a bound of the view frustum, and dividing by the buffer size.” The buffer size is the shadow map size, but what is the “bound of the view frustum”? Would that be the size sent to glOrtho()? Basically I just don’t understand how to calculate texel sizes…

 FLOAT fWorldUnitsPerTexel = fCascadeBound /
        (float)m_CopyOfCascadeConfig.m_iBufferSize;
        vWorldUnitsPerTexel = XMVectorSet( fWorldUnitsPerTexel, fWorldUnitsPerTexel, 0.0f, 0.0f );

vLightCameraOrthographicMin /= vWorldUnitsPerTexel;
        vLightCameraOrthographicMin = XMVectorFloor( vLightCameraOrthographicMin );
        vLightCameraOrthographicMin *= vWorldUnitsPerTexel;
        vLightCameraOrthographicMax /= vWorldUnitsPerTexel;
        vLightCameraOrthographicMax = XMVectorFloor( vLightCameraOrthographicMax );
        vLightCameraOrthographicMax *= vWorldUnitsPerTexel;

Edit: FYI here’s what I have so far, if change x and z to ints it helps with the flickering, but either way this still occurs…

		glViewport(0, 0, SHADOW_MAP_SIZE, SHADOW_MAP_SIZE);
		glMatrixMode(GL_PROJECTION);
		glLoadIdentity();
		glOrtho(-30.0f, 30.0f, -30.0f, 30.0f, -50.0f, 50.0f);
		
		glMatrixMode(GL_MODELVIEW);
		glLoadIdentity();

		Vector3 translate = main_cam.getPosition();
		float factor = 60.0f / SHADOW_MAP_SIZE;
		float x = Math.round(translate.m_x / factor) * factor;
		float z = Math.round(translate.m_z / factor) * factor;
		glTranslatef(-x, 0.0f, -z);
#5

I recently fixed the shimmering myself. You basically need to snap the position of the shadow map rendering in light space to shadow map texels.

  1. Calculate a light view matrix. This is easily done using a look-at matrix looking in the light’s direction. You’ll need to make sure the up-vector is always valid.

  2. Calculate the main camera’s view frustum bounds in light view space. Take all 8 combination of (±1, ±1, ±1), multiply by inverse projection matrix, inverse view matrix, then the light view matrix. Then calculate the min and max XYZ of these 8 points. You have your bounds.

  3. Usually the XY bounds aren’t square like the shadow map is. It’s a good idea to calculate the center of the bounds (centerXYZ = (maxXYZ+minXYZ)*0.5), and an “extent” (extent = max(maxX-minX, maxY-minY) * 0.5). At this point, the problem is that both centerXYZ and extent change with very high subpixel resolution while the shadow map is very limited in resolution. This is what causes the flickering.

  4. To fix the centerXYZ, we snap to the nearest shadow map texel. Snap function:

	public static float snap(float f, float multiple){
		return Math.round(f/multiple)*multiple;
	}

The snap multiple you want to use is (2.0f * extent / shadowMapResolution). However, a problem here is that the flickering reoccurs when the extent varies as well. This only works if the camera is only moving, not rotating. The best solution for extent is to calculate the maximum extent over time (extent = max(newExtent, extent)) and always use the saved one, if even if the current frustum happens to have a lower extent.

  1. Compute the light projection matrix: ortho(centerX-extent, centerX+extent, centerY-extent, centerY+extent, -maxZ, -minZ). The extent is only 2D extent, so we use the raw bounds calculated in point 3 for depth.

Done! That was really easy, wasn’t it? >____>

#6

Yeah totally no problem…

OK so

  1. My light doesn’t have an up vector (that I define), would it be possible to use the modelview matrix after rotation but before translation or should I include the translation?

  2. To clarify we are passing centerXYZ as float f?

#7
  1. Any up-vector as long as it’s not parallel to the direction vector is fine.
  2. centerXYZ is just the center point of the frustum bounds cuboid.
#8

It’s still swimming :(, I guess I’m missing something?

To get the inverse view matrix I can just use the camera transform matrix correct?

“1. The Camera Transformation Matrix: The transformation that places the camera in the correct position and orientation in world space (this is the transformation that you would apply to a 3D model of the camera if you wanted to represent it in the scene).
2. The View Matrix: This matrix will transform vertices from world-space to view-space. This matrix is the inverse of the camera’s transformation matrix described above.” - http://www.3dgep.com/understanding-the-view-matrix/

The inverse projection matrix I’m getting from openGL after applying the camera’s projection and then inverting…

I’m calculating the light matrix like so(up is defaulted to +y):

Vector3 zaxis = (look.subtract(eye)).normalize();
Vector3 xaxis = (up.crossProduct(zaxis)).normalize();
Vector3 yaxis = zaxis.crossProduct(xaxis);
		
m_mat[0][0] = xaxis.m_x; m_mat[0][1] = xaxis.m_y; m_mat[0][2] = xaxis.m_z; m_mat[0][3] = -xaxis.dotProduct(eye);
m_mat[1][0] = yaxis.m_x; m_mat[1][1] = yaxis.m_y; m_mat[1][2] = yaxis.m_z; m_mat[1][3] = -yaxis.dotProduct(eye);
m_mat[2][0] = zaxis.m_x; m_mat[2][1] = zaxis.m_y; m_mat[2][2] = zaxis.m_z; m_mat[2][3] = -zaxis.dotProduct(eye);
m_mat[3][0] = 0.0f; 	 m_mat[3][1] = 0.0f; 	  m_mat[3][2] = 0.0f; 	   m_mat[3][3] = 1.0f;

and then to find the bounds I should just be able to do the following for each point:

points[0] = new Vector3( 1.0f,  1.0f,  1.0f);
points[0] = light_matrix.multiply((inv_view.multiply((inv_proj.multiply(points[0])))));
  1. After finding the centerXYZ I am snapping like so (max_extent is a float member value, also I think this is where I screwed up):
float extent = Math.max(max_xyz.m_x - min_xyz.m_x, max_xyz.m_y - min_xyz.m_y) * 0.5f;
if(extent > max_extent) max_extent = extent;
float multiple = 2.0f * max_extent / SHADOW_MAP_SIZE;
		
center_xyz.m_x = Math.round(center_xyz.m_x / multiple) * multiple;
center_xyz.m_y = Math.round(center_xyz.m_y / multiple) * multiple;
center_xyz.m_z = Math.round(center_xyz.m_z / multiple) * multiple;
  1. and finally my light projection matrix:

gl.glViewport(0, 0, SHADOW_MAP_SIZE, SHADOW_MAP_SIZE);
gl.glMatrixMode(GL2.GL_PROJECTION);
gl.glLoadIdentity();
		
gl.glOrtho(center_xyz.m_x - max_extent, 
			center_xyz.m_x + max_extent, 
			center_xyz.m_y - max_extent, 
			center_xyz.m_y + max_extent, 
			-max_xyz.m_z, 
         -min_xyz.m_z);

This also makes the shadow map area very small between 3 and 6 units wide as the max_extent it usually just larger than 1.59.

#9

Snapping to texel is great with translate only but does not completely solve problem with orientation change. Only way to fix that is to push screen pixel to shadow texel ratio high enough and use wider sampling kernel. So you want to push your effective shadow resolution as high as possible. For Hardland I use standard cascade shadow maps but I also clip sub frustums to ground plane(I just use min of height map for frustum area). Then I try to fit this clipped frustum to minimal projection area. I just use 60 different shadow camera up vectors and I choose smallest projected area. Usually worst up vector produce twice as high area.

Remember also put far plane to edge of frustum.

#10

Keeping the extent constant and the shadow map square is supposed to fix that.

#11

Some how I skipped that part. But this basically waste lot of effective resolution.

Luckily my temporal antialiasing filters out rest of the noise.

More advanced techniques.
http://www.realtimeshadows.com/sites/default/files/Playing%20with%20Real-Time%20Shadows_0.pdf

#12

It does not. You almost certainly want to have a square shadow map area anyway, so expanding to the largest extent of XY is necessary to “squarify” it. The extent does not vary significantly over time. Using the max extent is simply a solution to avoid flickering due to small changes in the extent.