Questions about GLSL and shadows

CopyableCougar4 · July 1, 2014, 12:51am

Hi

In my current project I would like to hide everything except in a certain radius around the player. Essentially I want an unseen light source around the player that fades to almost complete darkness around the player. I don’t even know where to start searching, researching, or coding, so any code samples, links, or keywords to google would help

CopyableCougar4

Rayvolution · July 1, 2014, 12:54am

“Fog of War” or “Line of sight” tutorials may help.

CopyableCougar4 · July 1, 2014, 1:22am

Hi

As of now, I have a really hacky and laggy way to achieve what I want, but as of now it’s to broken to be considered a solution. However… progress

CopyableCougar4

CopyableCougar4 · July 1, 2014, 1:40am

Hi

I had an idea about how to accomplish this, but it has some flaws.


   public void renderMap(){
		int radius = 0;
		float alpha = 1.0f;
		while(radius < Toolkit.getDefaultToolkit().getScreenSize().width){
			radius += 50;
			GL11.glColor4f(0.0f, 0.0f, 0.0f, alpha);
			GL11.glBegin(GL11.GL_TRIANGLE_FAN);
			GL11.glVertex2f(0, 0);
			for(int i = 0; i <= 360; i++){ 
			    double angle = Math.PI / 180.0 * i;
			    GL11.glVertex2f(x + (radius * (float)Math.cos(angle)), y + (radius * (float)Math.sin(angle)));
			}
			GL11.glEnd();
			alpha -= 0.05f;
			alpha = Math.max(0.0f, alpha);
			GL11.glColor3f(1.0f, 1.0f, 1.0f);
		}
	}

This starts dark in the center and gets lighter going outward, which is the opposite of what I want. Any thoughts?

CopyableCougar4

EDIT: I fixed that inversion of alphas by disabling 2d textures, so I think my issue is mostly solved
EDIT 2: My problem still isn’t fixed

CopyableCougar4 · July 1, 2014, 2:59am

Hi

I have been trying some possible solutions, and the one I posted above is the closest to the desired result, unfortunately.

A few flaws in my latest idea:

It should get darker as it goes out, not lighter
I would like the light level to change more smoothly.

CopyableCougar4

PandaMoniumHUN · July 1, 2014, 4:45am

Pass the player’s position to your fragment shaders as a uniform and calculate the distance from the player’s position to the fragment’s position.
Divide the received value by your max view distance thus normalizing the value between 0 and 1.
Now flip the value by subtracting it from one. (<- This is needed so that further objects are being darker and not lighter)
The received value will be your color intensity factor which you can use to multiply your output color channels with.

This should be extremely efficient and pretty easy to implement.
If you need further help or a GLSL example code just tell me.

Edit: I forgot to mention that although you didn’t talk about it I’m expecting you to be doing this in 2D. What I wrote is a 2D approach to your problem.
Also forgot to mention that you should clamp the value to 1 after you divide it by your max view distance so further away fragments will return 1 as well and not 1+ values.

CopyableCougar4 · July 1, 2014, 2:17pm

Hi

From what I can see PandaMoniumHUN, your idea would work well for me, however, I will need some examples as I am a complete newbie with GLSL. Thanks

Edit: I was looking around for any GLSL tutorials or explanations I could find, and I was giving GLSL a feeble (at best) attempt. Here is what I have so far (note: I may be completely wrong, so please point out any problems with what I have so far):


#version 110

uniform vec2 player;
in vec4 gl_FragCoord;
out vec4 outputF;

void main(){
	// calculate the distance
	double distance = sqrt(((gl_FragCoord.x - player.x) * (gl_FragCoord.x - player.x)) + ((gl_FragCoord.y - player.y) * (gl_FragCoord.y - player.y)));
	// divide by the max view distance
	double received = distance / 1000;
        if(received > 1.0){
            received = 1.0;
        }
	double inverted = 1.0 - received;
	outputF = vec4(0.0, 0.0, 0.0, inverted);
}

CopyableCougar4

CopyableCougar4 · July 1, 2014, 4:17pm

Hi

Okay, so I was writing some more experimental stuff in GLSL, and now I have a shader that compiles (I think), but when I use it it makes the whole screen black.

Pastebin: http://pastebin.java-gaming.org/c4542899495

Any help would be appreciated

CopyableCougar4

Longarmx · July 1, 2014, 5:22pm

Just some thoughts:
-Player position can be a uniform, since it’s not changing from fragment to fragment
-distance / 1000.0
-[icode]inverted = 1.0 - clamp(distance / 1000.0, 0.0, 1.0);[/icode]

PandaMoniumHUN · July 1, 2014, 11:25pm

Not a bad start but sadly I forgot to mention in my previous post that this could be best implemented as a post-processing effect.
That means you have to do the following:

Create an FBO with it’s own texture
Render everything to your FBO using your regular shaders
Render out your FBO to the screen using your lighting shader

You got me inspired with this and I was wondering if I could make it work, here’s the result I’ve got:

Also here’s a gorgeous (well, not so much, thanks to the recording program) GIF of the effect in action: [Link]
The GIF turned out to be huge (7+MB) so if it’s lagging for you give your browser time to load it properly. Also it is NOT as ugly as in the GIF, it looks like exactly on the picture above.

This is all calculated in the fragment shader, no premade mask has been used.
Another approach to this kind of lighting would be to use a circle texture from GIMP or Photoshop as an alpha mask but that could end up looking worse than this (although that is probably more efficient).

And to post some actual code too:

Vertex Shader:


#version 330 core
uniform mat4 projectionMatrix;
layout(location=0) in vec2 in_Position;
layout(location=1) in vec2 in_TexCoord;
out vec2 pass_TexCoord;
void main(){
	pass_TexCoord = vec2(in_TexCoord.x, 1.0-in_TexCoord.y);
	gl_Position = projectionMatrix*vec4(in_Position, 0.0, 1.0);
}

Fragment shader:


#version 330 core
uniform sampler2D texUnit;
uniform vec2 playerPos;
uniform float viewDistance;
uniform vec4 color;
in vec2 pass_TexCoord;
layout(location=0) out vec4 out_Color;
void main(){
	vec2 fragPos = vec2(ceil(gl_FragCoord.x), ceil(gl_FragCoord.y));
	float lightFactor = 1.0-clamp(distance(playerPos, fragPos)/viewDistance, 0.0, 1.0);
	out_Color = texture(texUnit, pass_TexCoord)*color*vec4(vec3(lightFactor), 1.0);
}

This is the shader that I use to draw my FBO texture to the screen.
As you can see the uniforms that you have to pass in before rendering are:

projectionMatrix <- The projection matrix
playerPos <- The player’s x and y position
viewDistance <- The player’s view distance
color <- You can change this value to affect the output texture color. Set it to (1; 1; 1; 1) to get normal output.

If you have further questions feel free to ask.

CopyableCougar4 · July 1, 2014, 11:40pm

Hi

I’m gonna need the java half of the code too, as I am not only a complete newbie in GLSL, but quite a newbie in LWJGL altogether

CopyableCougar4

Slyth2727 · July 2, 2014, 12:15am

Well that’s the point isn’t it? If you have to at least code some of it yourself, you’ll learn loads more.
Give it a shot.

CopyableCougar4 · July 2, 2014, 12:24am

Hi

Yeah but with his GLSL code I need to know what variables were passed to the shader so that I can do the code parts that I am capable of.

CopyableCougar4

Slyth2727 · July 2, 2014, 12:29am

Which variables don’t make sense to you?

CopyableCougar4 · July 2, 2014, 12:33am

Hi

I don’t really know where basically all the uniforms were passed in except for the ones that were passed between the fragment and vertex shaders.

CopyableCougar4

Slyth2727 · July 2, 2014, 12:39am


#version 330 core
uniform sampler2D texUnit; //the current texture
uniform vec2 playerPos; // ......
uniform float viewDistance; //probably the radius/diameter of the circle
uniform vec4 color; // ..
in vec2 pass_TexCoord; // once again..
layout(location=0) out vec4 out_Color; // the output color, replaced by gl_FragColor in some cases
void main(){
   vec2 fragPos = vec2(ceil(gl_FragCoord.x), ceil(gl_FragCoord.y));
   float lightFactor = 1.0-clamp(distance(playerPos, fragPos)/viewDistance, 0.0, 1.0);
   out_Color = texture(texUnit, pass_TexCoord)*color*vec4(vec3(lightFactor), 1.0);
}

PandaMoniumHUN · July 2, 2014, 8:50am

Okay, I’m back.

The thing is I can’t teach you all of GLSL, you should learn that yourself. It could probably take up like 1 or 2 hours to get the grasp of it.
The shader that I wrote is for modern OpenGL, that means there are no built-in matrices (that’s why you have to send the projection matrix to the shader as a uniform) - note: I don’t need to multiply with a modelView matrix here since I always want my FBO texture to be rendered at (0; 0) - , no gl_FragColor (instead we have to send out a vec4 that’ll be our color, that’s out_Color). PlayerPos and ViewDistance should be obvious. A nice side note here is that if you’re using orthographic projection you can specify the viewDistance in pixels (in my program I set it to 300 = The view distance is 300 pixels from the center of our player).

In the vertex shader my two inputs (in_Position and in_TexCoord) are per-vertex attributes that are being filled up from data from my buffers using vertex attribute pointers. But because I need the texture coordinate in the fragment shader instead of my vertex shader I have to pass it to the fragment shader. That’s what pass_TexCoord is for. An output that the fragment shader receives as an input. You might be wondering why I do this:
[icode]pass_TexCoord = vec2(in_TexCoord.x, 1.0-in_TexCoord.y);[/icode]
Instead of this:
[icode]pass_TexCoord = in_TexCoord;[/icode]
That’s because my FBO texture is upside down for some reason (probably because I’m fighting the convention and my orthographic projection is “upside down” to make (0; 0) my bottom-left coordinate) so I have to flip it on the Y axis. Don’t care about it too much, if it’s upside down for you too that’s how you flip it back.

Okay so about the calculations:
I calculate the current fragment’s position by ceiling the gl_FragCoord’s x and y values. You have to ceil them because they contain the ‘middle position of a pixel’. So for the bottom left pixel it’s going to be [icode](0.5; 0.5)[/icode] instead of [icode](0; 0)[/icode] or [icode](1; 1)[/icode]. With ceil I turn [icode](0.5; 0.5)[/icode] to [icode](1; 1)[/icode].

I’ve already explained how to calculate the lightFactor in my previous posts but here’s it again:
You have to calculate the distance between the player’s position and the fragment’s position and divide the received value with your viewDistance. After that you should clamp the value to min. 0 and max. 1 and finally subtract the value from 1. As for the why check out my post above.

Finally I set the output color to be a mix of 3 things:

My FBO texture: [icode]texture(texUnit, pass_TexCoord)[/icode] (<- That’s where we use the passed texture coordinate in our fragment shader)
A passed uniform color that you can use to manipulate the output’s color: [icode]color[/icode]
The lightFactor: [icode]vec4(vec3(lightFactor), 1.0);[/icode] Note that [icode]vec4(vec3(lightFactor), 1.0)[/icode] is just a shortcut to [icode]vec4(lightFactor, lightFactor, lightFactor, 1.0)[/icode].

Now that I’ve explained everything you should be able to port this to OpenGL 2.0 or to whatever version you’re using.