Blinn-Phong is a slight adjustment of the Phong model you saw in the lighting section. It provides more plausible or realistic specular reflections. You'll notice that Blinn-Phong produces elliptical or elongated specular reflections versus the spherical specular reflections produced by the Phong model. In certain cases, Blinn-Phong can be more efficient to calculate than Phong.
Instead of computing the reflection vector, compute the halfway or half angle vector. This vector is between the view/camera/eye and light direction vector.
The specular intensity is now the dot product of the normal and halfway vector. In the Phong model, it is the dot product of the reflection and view vector.
\n
\n\n
\n\n
The half angle vector (magenta arrow) will point in the same direction as the normal (green arrow) when the view vector (orange arrow) points in the same direction as the reflection vector (magenta arrow). In this case, both the Blinn-Phong and Phong specular intensity will be one.
\n
\n\n
\n\n
In other cases, the specular intensity for Blinn-Phong will be greater than zero while the specular intensity for Phong will be zero.
Adding bloom to a scene can really sell the illusion of the lighting model. Light emitting objects are more believable and specular highlights get an extra dose of shimmer.
These parameters control the look and feel. size determines how blurred the effect is. separation spreads out the blur. threshold controls which fragments are illuminated. And the last parameter, amount, controls how much bloom is outputted.
\n
// ...\n\n vec2 texSize = textureSize(colorTexture, 0).xy;\n\nfloat value = 0.0;\nfloat count = 0.0;\n\n vec4 result = vec4(0);\n vec4 color = vec4(0);\n\nfor (int i = -size; i <= size; ++i) {\nfor (int j = -size; j <= size; ++j) {\n// ...\n }\n }\n\n// ...
\n
The technique starts by looping through a kernel/matrix/window centered over the current fragment. This is similar to the window used for outlining. The size of the window is size * 2 + 1 by size * 2 + 1. So for example, with a size setting of two, the window uses (2 * 2 + 1)^2 = 25 samples per fragment.
\n
// ...\n\n color =\n texture\n ( colorTexture\n , ( gl_FragCoord.xy\n + (vec2(i, j) * separation)\n )\n / texSize\n );\n\n value = max(color.r, max(color.g, color.b));\nif (value < threshold) { color = vec4(0); }\n\n result += color;\n count += 1.0;\n\n// ...
\n
For each iteration, it retrieves the color from the input texture and turns the red, green, and blue values into a greyscale value. If this greyscale value is less than the threshold, it discards this color by making it solid black. After evaluating the sample's greyscale value, it adds its RGB values to result.
After it's done summing up the samples, it divides the sum of the color samples by the number of samples taken. The result is the average color of itself and its neighbors. By doing this for every fragment, you end up with a blurred image. This form of blurring is known as a box blur.
\n
\n\n
\n\n
Here you see the progression of the bloom algorithm.
The need to blur this or that can come up quite often as you try to obtain a particular look or perform some technique like motion blur. Below are just some of ways you can blur your game's imagery.
\n
Box Blur
\n
\n\n
\n\n
The box blur or mean filter algorithm is a simple to implement blurring effect. It's fast and gets the job done. If you need more finesse, you can upgrade to a Gaussian blur.
Like the outlining technique, the box blur technique uses a kernel/matrix/window centered around the current fragment. The size of the window is size * 2 + 1 by size * 2 + 1. So for example, with a size setting of two, the window uses (2 * 2 + 1)^2 = 25 samples per fragment.
To finish computing the mean, divide the sum of the colors sampled by the number of samples taken. The final fragment color is the mean or average of the fragments sampled inside the window.
\n
Median Filter
\n
\n\n
\n\n
The box blur uses the mean color of the samples taken. The median filter uses the median color of the samples taken. By using the median instead of the mean, the edges in the image are preserved—meaning the edges stay nice and crisp. For example, look at the windows in the box blurred image versus the median filtered image.
\n
Unfortunately, finding the median can be slower than finding the mean. You could sort the values and choose the middle one but that would take at least quasilinear time. There is a technique to find the median in linear time but it can be quite awkward inside a shader. The numerical approach below approximates the median in linear time. How well it approximates the median can be controlled.
\n
\n\n
\n\n
At lower quality approximations, you end up with a nice painterly look.
The size parameter controls how blurry or smeared the effect is. If the size is at or below zero, return the current fragment untouched. From the size parameter, calculate the total size of the kernel or window. This is how many samples you'll be taking per fragment.
i and j are used to sample the given texture around the current fragment. i is also used as a general for loop count. count is used in the initialization of the colors array which you'll see later. binIndex is used to approximate the median color.
The colors array holds the sampled colors taken from the input texture. bins is used to approximate the median of the sampled colors. Each bin holds a count of how many colors fall into its range when converting each color into a greyscale value (between zero and one). As binsSize approaches 100, the algorithm finds the true median almost always. binIndexes stores the bins index or which bin each sample falls into.
total keeps track of how many colors you've come across as you loop through bins. When total reaches limit, you return whatever bins index you're at. The limit is the median index. For example, if the window size is 81, limit is 41 which is directly in the middle (40 samples below and 40 samples above).
These are used to covert and hold each color sample's greyscale value. Instead of dividing red, green, and blue by one third, it uses 30% of red, 59% of green, and 11% of blue for a total of 100%.
Loop through the colors and convert each one to a greyscale value. dot(colors[i].rgb, valueRatios) is the weighted sum colors.r * 0.3 + colors.g * 0.59 + colors.b * 0.11.
\n
Each value will fall into some bin. Each bin covers some range of values. For example, if the number of bins is 10, the first bin covers everything from zero up to but not including 0.1. Increment the number of colors that fall into this bin and remember the color sample's bin index so you can look it up later.
\n
// ...\n\n binIndex = 0;\n\nfor (i = 0; i < binsSize; ++i) {\n total += bins[i];\nif (total >= limit) {\n binIndex = i;\nbreak;\n }\n }\n\n// ...
\n
Loop through the bins, tallying up the number of colors seen so far. When you reach the median index, exit the loop and remember the last bins index reached.
Now loop through the binIndexes and find the first color with the last bins indexed reached. Its greyscale value is the approximated median which in many cases will be the true median value. Set this color as the fragColor and exit the loop and shader.
\n
Kuwahara Filter
\n
\n\n
\n\n
Like the median filter, the kuwahara filter preserves the major edges found in the image. You'll notice that it has a more block like or chunky pattern to it. In practice, the Kuwahara filter runs faster than the median filter, allowing for larger size values without a noticeable slowdown.
The Kuwahara filter works by computing the variance of four subwindows and then using the mean of the subwindow with the smallest variance.
\n
// ...\n\nvoid findMean(int i0, int i1, int j0, int j1) {\n\n// ...
\n
findMean is a function defined outside of main. Each run of findMean will remember the mean of the given subwindow that has the lowest variance seen so far.
Similar to the box blur, loop through the given subwindow and add up each color. At the same time, make sure to store the greyscale value for this sample in values.
Now calculate the variance for this given subwindow. The variance is the average squared difference between each sample's greyscale value the mean greyscale value.
If the variance is smaller than what you've seen before or this is the first variance you've seen, set the mean of this subwindow as the final mean and update the minimum variance seen so far.
As stated above, the Kuwahara filter works by computing the variance of four subwindows and then using the mean of the subwindow with the lowest variance as the final fragment color. Note that the four subwindows overlap each other.
Make sure to locate where the Panda3D headers and libraries are. The headers and libraries are most likely in /usr/include/panda3d/ and /usr/lib/panda3d/ respectively.
\n
Next clone this repository and change directory into it.
With the object file created, create the executable by linking the object file to its dependencies. You'll need to track down where the Panda3D libraries are located.
Cel shading is a technique to make 3D objects look 2D or flat. In 2D, you can make an object look 3D by applying a smooth gradient. However, with cel shading, you're breaking up the gradients into abrupt transitions. Typically there is only one transition where the shading goes from fully lit to fully shadowed. When combined with outlining, cel shading can really sell the 2D cartoon look.
Another approach is to put your step values into a texture with the transitions going from darker to lighter. Using the diffuseIntensity as a U coordinate, it will automatically transform itself.
Using the step function again, set the specularIntensity to be either zero or one. You can also use one of the other approaches described up above for the specular highlight as well. After you've altered the specularIntensity, the rest of the lighting calculations are the same.
Chromatic aberration is a screen space technique that simulates lens distortion. Use it to give your scene a cinematic, lo-fi analog feel or to emphasize a chaotic event.
\n
Texture
\n
uniform sampler2D colorTexture;\n\n// ...
\n
The input texture needed is the scene's colors captured into a framebuffer texture.
The adjustable parameters for this technique are the red, green, and blue offsets. Feel free to play around with these to get the particular color fringe you're looking for. These particular offsets produce a yellowish orange and blue fringe.
The offsets can occur horizontally, vertically, or radially. One approach is to radiate out from the depth of field focal point. As the scene gets more out of focus, the chromatic aberration increases.
With the direction and offsets, make three samples of the scene's colors—one for the red, green, and blue channels. These will be the final fragment color.
Deferred rendering (deferred shading) is a screen space lighting technique. Instead of calculating the lighting for a scene while you traverse its geometry—you defer or wait to perform the lighting calculations until after the scene's geometry fragments have been culled or discarded. This can give you a performance boost depending on the complexity of your scene.
\n
Phases
\n
\n\n
\n\n
Deferred rendering is performed in two phases. The first phase involves going through the scene's geometry and rendering its positions or depths, normals, and materials into a framebuffer known as the geometry buffer or G-buffer. With the exception of some transformations, this is mostly a read-only phase so its performance cost is minimal. After this phase, you're only dealing with 2D textures in the shape of the screen.
\n
\n\n
\n\n
The second and last phase is where you perform your lighting calculations using the output of the first phase. This is when you calculate the ambient, diffuse, and specular colors. Shadow and normal mapping are performed in this phase as well.
\n
Advantages
\n
The reason for using deferred rendering is to reduce the number of lighting calculations made. With forward rendering, the number of lighting calculations scales with the number of fragments and lights. However, with deferred shading, the number of lighting calculations scales with the number of pixels and lights. Recall that for a single pixel, there can be multiple fragments produced. As you add geometry, the number of lighting calculations per pixel increases when using forward but not when using deferred.
\n
For simple scenes, deferred rendering doesn't provide much of a performance boost and may even hurt performance. However, for complex scenes with lots of lighting, it becomes the better option. Deferred becomes faster than forward because you're only calculating the lighting per light, per pixel. In forward rendering, you're calculating the lighting per light per fragment which can be multiple times per pixel.
\n
Disadvantages
\n
Deferred rendering allows you render complex scenes using many lights but it does come with its own set of tradeoffs. Transparency becomes an issue because the geometry data you could see through a semitransparent object is discarded in the first phase. Other tradesoffs include increased memory consumption due to the G-buffer and the extra workarounds needed to deal with aliasing.
Like SSAO, depth of field is an effect you can't live without once you've used it. Artistically, you can use it to draw your viewer's eye to a certain subject. But in general, depth of field adds a lot of realism for a little bit of effort.
\n
In Focus
\n
The first step is to render your scene completely in focus. Be sure to render this into a framebuffer texture. This will be one of the inputs to the depth of field shader.
\n
Out Of Focus
\n
The second step is to blur the scene as if it was completely out of focus. Like bloom and SSAO, you can use a box blur. Be sure to render this out-of-focus-scene into a framebuffer texture. This will be one of the inputs to the depth of field shader.
\n
Bokeh
\n
\n\n
\n\n
For a great bokeh effect, dilate the out of focus texture and use that as the out of focus input. See dilation for more details.
Feel free to tweak these two parameters. All positions at or below minDistance will be completely in focus. All positions at or beyond maxDistance will be completely out of focus.
The focus point is a position somewhere in your scene. All of the points in your scene are measured from the focus point.
\n
Choosing the focus point is up to you. The demo uses the scene position directly under the mouse when clicking the middle mouse button. However, it could be a constant distance from the camera or a static position.
smoothstep returns values from zero to one. The minDistance is the left-most edge. Any position less than the minimum distance, from the focus point, will be in focus or have a blur of zero. The maxDistance is the right-most edge. Any position greater than the maximum distance, from the focus point, will be out of focus or have a blur or one. For distances between the edges, blur will be between zero and one. These values are interpolated along a s-shaped curve.
The fragColor is a mixture of the in focus and out of focus color. The closer blur is to one, the more it will use the outOfFocusColor. Zero blur means this fragment is entirely in focus. At blur >= 1, this fragment is completely out of focus.
Dilation dilates or enlarges the brighter areas of an image while at the same time, contracts or shrinks the darker areas of an image. This tends to create a pillowy look. You can use dilation for a glow/bloom effect or to add bokeh to your depth of field.
The size and separation parameters control how dilated the image becomes. A larger size will increase the dilation at the cost of performance. A larger separation will increase the dilation at the cost of quality. The minThreshold and maxThreshold parameters control which parts of the image become dilated.
Loop through a size by size window, centered at the current fragment position. As you loop, find the brightest color based on the surrounding greyscale values.
\n
\n\n
\n\n
// ...\n\n// For a rectangular shape.\n//if (false);\n\n// For a diamond shape;\n//if (!(abs(i) <= size - abs(j))) { continue; }\n\n// For a circular shape.\nif (!(distance(vec2(i, j), vec2(0, 0)) <= size)) { continue; }\n\n// ...
\n
The window shape will determine the shape of the dilated parts of the image. For a rectangular shape, you can use every fragment covered by the window. For any other shape, skip the fragments that fall outside the desired shape.
The new fragment color is a mixture between the existing fragment color and the brightest color found. If the maximum greyscale value found is less than minThreshold, the fragment color is unchanged. If the maximum greyscale value is greater than maxThreshold, the fragment color is replaced with the brightest color found. For any other case, the fragment color is a mix between the current fragment color and the brightest color.
Film grain (when applied in subtle doses, unlike here) can add a bit of realism you don't notice until it's removed. Typically, it's the imperfections that make a digitally generated image more believable. In terms of the shader graph, film grain is usually the last effect applied before the game is put on screen.
\n
Amount
\n
// ...\n\nfloat amount = 0.1;\n\n// ...
\n
The amount controls how noticeable the film grain is. Crank it up for a snowy picture.
This snippet calculates the random intensity needed to adjust the amount.
\n
Time Since F1 = 0001020304050607080910\nFrame Number = F1 F3 F4 F5 F6\nosg_FrameTime = 0002040708
\n
osg_FrameTime is provided by Panda3D. The frame time is a timestamp of how many seconds have passed since the first frame. The example code uses this to animate the film grain as osg_FrameTime will always be different each frame.
\n
\n// ...\n\n ( gl_FragCoord.x\n + gl_FragCoord.y\n * 8009// Large number here.\n\n// ...
\n
For static film grain, replace osg_FrameTime with a large number. You may have to try different numbers to avoid seeing any patterns.
Both the x and y coordinate are used to create points or specs of film grain. If only x was used, there would only be vertical lines. Similarly, if only y was used, there would be only horizontal lines.
\n
The reason the snippet multiplies one coordinate by some number is to break up the diagonal symmetry.
\n
\n\n
\n\n
You can of course remove the coordinate multiplier for a somewhat decent looking rain effect. To animate the rain effect, multiply the output of sin by osg_FrameTime.
sin is used as a hashing function. The fragment's coordinates are hashed to some output of sin. This has the nice property that no matter the input (big or small), the output range is negative one to one.
sin is also used as a pseudo random number generator when combined with fract.
\n
>>> [floor(fract(4* sin(x * toRadians)) *10) for x inrange(0, 10)]\n[0, 0, 1, 2, 2, 3, 4, 4, 5, 6]\n\n>>> [floor(fract(10000* sin(x * toRadians)) *10) for x inrange(0, 10)]\n[0, 4, 8, 0, 2, 1, 7, 0, 0, 5]
\n
Take a look at the first sequence of numbers and then the second. Each sequence is deterministic but the second sequence has less of a pattern than the first. So while the output of fract(10000 * sin(...)) is deterministic, it doesn't have much of a discernible pattern.
\n
\n\n
\n\n
Here you see the sin multiplier going from 1, to 10, to 100, and then to 1000.
\n
As you increase the sin output multiplier, you get less and less of a pattern. This is the reason the snippet multiplies sin by 10,000.
Flow mapping is useful when you need to animate some fluid material. Much like diffuse maps map UV coordinates to diffuse colors and normal maps map UV coordinates to normals, flow maps map UV coordinates to 2D translations or flows.
\n
\n\n
\n\n
Here you see a flow map that maps UV coordinates to translations in the positive y-axis direction. Flow maps use the red and green channels to store translations in the x and y direction. The red channel is for the x-axis and the green channel is the y-axis. Both range from zero to one which translates to flows that range from (-1, -1) to (1, 1). This flow map is all one color consisting of 0.5 red and 0.6 green.
\n
[r, g, b] =\n [r * 2 - 1, g * 2 - 1, b * 2 - 1] =\n [ x, y, z]
\n
Recall how the colors in a normal map are converted to actual normals. There is a similar process for flow maps.
The flow map above maps each UV coordinate to the flow (0, 0.2). This indicates zero movement in the x direction and a movement of 0.2 in the y direction.
\n
The flows can be used to translate all sorts of things but they're typically used to offset the UV coordinates of a another texture.
For example, the demo program uses a flow map to animate the water. Here you see the flow map being used to animate the foam mask. This continuously moves the diffuse UV coordinates directly up, giving the foam mask the appearance of moving down stream.
You'll need how many seconds have passed since the first frame in order to animate the UV coordinates in the direction indicated by the flow. osg_FrameTime is provided by Panda3D. It is a timestamp of how many seconds have passed since the first frame.
Foam is typically used when simulating some body of water. Anywhere the water's flow is disrupted, you add some foam. The foam isn't much by itself but it can really connect the water with the rest of the scene.
\n
\n\n
\n\n
But don't stop at just water. You can use the same technique to make a river of lava for example.
\n
Vertex Positions
\n
Like screen space refraction, you'll need both the foreground and background vertex positions. The foreground being the scene with the foamy surface and the background being the scene without the foamy surface. Referrer back to SSAO for the details on how to acquire the vertex positions in view space.
\n
Mask
\n
\n\n
\n\n
You'll need to texture your scene with a foam mask. The demo masks everything off except the water. For the water, it textures it with a foam pattern.
Here you see the fragment shader that generates the foam mask. It takes a foam pattern texture and UV maps it to the scene's geometry using the diffuse UV coordinates. For every model, except the water, the shader is given a solid black texture as the foamPatternTexture.
The foam shader accepts a mask texture, the foreground vertex positions (positionFromTexture), and the background vertex positions (positionToTexture).
The adjustable parameters for the foam shader are the foam depth and color. The foam depth controls how much foam is shown. As the foam depth increases, the amount of foam shown increases.
Compute the distance from the foreground position to the background position. Since the positions are in view (camera) space, we only need the y value since it goes into the screen.
Fog (or mist as it's called in Blender) adds atmospheric haze to a scene, providing mystique and softening pop-ins (geometry suddenly entering into the camera's frustum).
In addition to the fog's attributes, you'll also need the fragment's vertex position.
\n
float fogMin = 0.00;\nfloat fogMax = 0.97;
\n
fogMax controls how much of the scene is still visible when the fog is most intense. fogMin controls how much of the fog is still visible when the fog is least intense.
The example code uses a linear model for calculating the fog's intensity. There's also an exponential model you could use.
\n
The fog's intensity is fogMin before or at the start of the fog's near distance. As the vertex position gets closer to the end of the fog's far distance, the intensity moves closer to fogMax. For any vertexes after the end of the fog, the intensity is clamped to fogMax.
Set the fragment's color to the fog color and the fragment's alpha channel to the intensity. As intensity gets closer to one, you'll have less of the scene's color and more of the fog color. When intensity reaches one, you'll have all fog color and nothing else.
Normally you calculate the fog in the same shader that does the lighting calculations. However, you can also break it out into its own framebuffer texture. Here you see the fog color being applied to the rest of the scene much like you would apply a layer in GIMP. This allows you to calculate the fog once instead calculating it in every shader that eventually needs it.
The fresnel factor alters the reflectiveness of a surface based on the camera or viewing angle. As a surface points away from the camera, its reflectiveness goes up. Similarly, as a surface points towards the camera, its reflectiveness goes down.
\n
\n\n
\n\n
In other words, as a surface becomes perpendicular with the camera, it becomes more mirror like. Utilizing this property, you can vary the opacity of reflections (such as specular and screen space reflections) and/or vary a surface's alpha values for a more plausible or realistic look.
In the lighting section, the specular component was a combination of the material's specular color, the light's specular color, and by how much the camera pointed into the light's reflection direction. Incorporating the fresnel factor, you'll now vary the material specular color based on the angle between the camera and the surface it's pointed at.
The first vector you'll need is the eye/view/camera vector. Recall that the eye vector points from the vertex position to the camera's position. If the vertex position is in view or camera space, the eye vector is the vertex position pointed in the opposite direction.
The fresnel factor is calculated using two vectors. The simplest two vectors to use are the eye and normal vector. However, if you're using the halfway vector (from the Blinn-Phong section), you can instead calculate the fresnel factor using the halfway and eye vector.
With the needed vectors in hand, you can now compute the fresnel factor. The fresnel factor ranges from zero to one. When the dot product is one, the fresnel factor is zero. When the dot product is less than or equal to zero, the fresnel factor is one. This equation comes from Schlick's approximation.
\n
\n\n
\n\n
In Schlick's approximation, the fresnelPower is five but you can alter this to your liking. The demo code varies it using the blue channel of the specular map with a maximum value of five.
Once the fresnel factor is determined, use it to modulate the material's specular color. As the fresnel factor approaches one, the material becomes more like a mirror or fully reflective.
As before, the specular component is a combination of the material's specular color, the light's specular color, and by how much the camera points into the direction of the light's reflection. However, using the fresnel factor, the material's specular color various depending on the orientation of the camera and the surface it's looking at.
Correcting for gamma will make your color calculations look correct. This isn't to say they'll look amazing but with gamma correction, you'll find that the colors meld together better, the shadows are more nuanced, and the highlights are more subtle. Without gamma correction, the shadowed areas tend to get crushed while the highlighted areas tend to get blown-out and over saturated making for a harsh contrast overall.
\n
If you're aiming for realism, gamma correction is especially important. As you perform more and more calculations, the tiny errors add up making it harder to achieve photorealism. The equations will be correct but the inputs and outputs will be wrong leaving you frustrated.
\n
It's easy to get twisted around when thinking about gamma correction but essentially it boils down to knowing what color space a color is in and how to convert that color to the color space you need. With those two pieces of the puzzle, gamma correction becomes a tedious yet simple chore you'll have to perform from time to time.
\n
Color Spaces
\n
\n\n
\n\n
The two color spaces you'll need to be aware of are sRGB (standard Red Green Blue) and RGB or linear color space.
Knowing what color space a color texture is in will determine how you handle it in your shaders. To determine the color space of a texture, use ImageMagick's identify. You'll find that most textures are in sRGB.
To convert a texture to a particular color space, use ImageMagick's convert program. Notice how a texture is darkened when transforming from sRGB to RGB.
\n
Decoding
\n
The red, green, and blue values in a sRGB color texture are encoded and cannot be modified directly. Modifying them directly would be like running spellcheck on an encrypted message. Before you can run spellcheck, you first have to decrypt the message. Similarly, to modify the values of an sRGB texture, you first have to decode or transform them to RGB or linear color space.
To decode a sRGB encoded color, raise the rgb values to the power of 2.2. Once you have decoded the color, you are now free to add, subtract, multiply, and divide it.
\n
By raising the color values to the power of 2.2, you're converting them from sRGB to RGB or linear color space. This conversion has the effect of darkening the colors.
\n
\n\n
\n\n
For example, vec3(0.9, 0.2, 0.3) becomes vec3(0.793, 0.028, 0.07).
\n
\n\n
\n\n
The 2.2 value is known as gamma. Loosely speaking, gamma can either be 1.0 / 2.2, 2.2, or 1.0. As you've seen, 2.2 is for decoding sRGB encoded color textures. As you will see, 1.0 / 2.2 is for encoding linear or RGB color textures. And 1.0 is RGB or linear color space since y = 1 * x + 0 and any base raised to the power of 1.0 is itself.
\n
Non-color Data
\n
\n\n
\n\n
One important exception to decoding is when the \"colors\" of a texture represent non-color data. Some examples of non-color data would be the normals in a normal map, the alpha channel, the heights in a height map, and the directions in a flow map. Only decode color related data or data that represents color. When dealing with non-color data, treat the sRGB color values as RGB or linear and skip the decoding process.
\n
Encoding
\n
\n\n
\n\n
The necessity for encoding and decoding stems from the fact that humans do not perceive lightness linearly and most displays (like a monitor) lack the precision or number of bits to accurately show both lighter and darker tonal values or shades. With only so many bits to go around, colors are encoded in such a way that more bits are devoted to the darker shades than the lighter shades since humans are more sensitive to darker tones than lighter tones. Encoding it this way uses the limited number of bits more effectively for human perception. Still, the only thing to remember is that your display is expecting sRGB encoded values. Therefore, if you decoded a sRGB value, you have to encode it before it makes its way to your display.
To encode a linear value or convert RGB to sRGB, raise the rgb values to the power of 1.0 / 2.2. Notice how 1.1 / 2.2 is the reciprocal of 2.2 or 2.2 / 1.0. Here you see the symmetry in decoding and encoding.
Instead of using the fixed-function pipeline, you'll be using the programmable GPU rendering pipeline. Since it is programmable, it is up to you to supply the programming in the form of shaders. A shader is a (typically small) program you write using a syntax reminiscent of C. The programmable GPU rendering pipeline has various different stages that you can program with shaders. The different types of shaders include vertex, tessellation, geometry, fragment, and compute. You'll only need to focus on the vertex and fragment stages for the techniques below.
\n
#version 150\n\nvoid main() {}
\n
Here is a bare-bones GLSL shader consisting of the GLSL version number and the main function.
Here is a stripped down GLSL vertex shader that transforms an incoming vertex to clip space and outputs this new position as the vertex's homogeneous position. The main procedure doesn't return anything since it is void and the gl_Position variable is a built-in output.
\n
Take note of the keywords uniform and in. The uniform keyword means this global variable is the same for all vertexes. Panda3D sets the p3d_ModelViewProjectionMatrix for you and it is the same matrix for each vertex. The in keyword means this global variable is being given to the shader. The vertex shader receives each vertex that makes up the geometry the vertex shader is attached to.
Here is a stripped down GLSL fragment shader that outputs the fragment color as solid green. Keep in mind that a fragment affects at most one screen pixel but a single pixel can be affected by many fragments.
\n
Take note of the out keyword. The out keyword means this global variable is being set by the shader. The name fragColor is arbitrary so feel free to choose a different one.
\n
\n\n
\n\n
This is the output of the two shaders shown above.
Interested in adding textures, lighting, shadows, normal maps, glowing objects, ambient occlusion, reflections, refractions, and more to your 3D game? Great! Below is a collection of shading techniques that will take your game visuals to new heights. I've explained each technique in such a way that you can take what you learn here and apply/port it to whatever stack you use—be it Godot, Unity, Unreal, or something else. For the glue in between the shaders, I've chosen the fabulous Panda3D game engine and the OpenGL Shading Language (GLSL). So if that is your stack, then you'll also get the benefit of learning how to use these shading techniques with Panda3D and OpenGL specifically.
The included license applies only to the software portion of 3D Game Shaders For Beginners— specifically the .cxx, .vert, and .frag source code files. No other portion of 3D Game Shaders For Beginners has been licensed for use.
Completing the lighting involves calculating and combining the ambient, diffuse, specular, and emission light aspects. The example code uses either Phong or Blinn-Phong lighting.
For every light, minus the ambient light, Panda3D gives you this convenient struct which is available to both the vertex and fragment shaders. The biggest convenience being the shadow map and shadow view matrix for transforming vertexes to shadow or light space.
Starting in the vertex shader, you'll need to transform and output the vertex from view space to shadow or light space for each light in your scene. You'll need this later in the fragment shader in order to render the shadows. Shadow or light space is where every coordinate is relative to the light position (the light is the origin).
\n
Fragment
\n
The fragment shader is where most of the lighting calculations take place.
Before you loop through the scene's lights, create an accumulator for both the diffuse and specular colors.
\n
// ...\n\nfor (int i = 0; i < p3d_LightSource.length(); ++i) {\n// ...\n }\n\n// ...
\n
Now you can loop through the lights, calculating the diffuse and specular colors for each one.
\n
Light Related Vectors
\n
\n\n
\n\n
Here you see the four major vectors you'll need to calculate the diffuse and specular colors contributed by each light. The light direction vector is the light blue arrow pointing to the light. The normal vector is the green arrow standing straight up. The reflection vector is the dark blue arrow mirroring the light direction vector. The eye or view vector is the orange arrow pointing towards the camera.
The light direction is from the vertex's position to the light's position.
\n
Panda3D sets p3d_LightSource[i].position.w to zero if this is a directional light. Directional lights do not have a position as they only have a direction. So if this is a directional light, the light direction will be the negative or opposite direction of the light as Panda3D sets p3d_LightSource[i].position.xyz to be -direction for directional lights.
\n
// ...\n\n normal = normalize(vertexNormal);\n\n// ...
\n
You'll need the vertex normal to be a unit vector. Unit vectors have a length of magnitude of one.
You'll need to take the dot product involving the light direction so its best to normalize it. This gives it a distance or magnitude of one (unit vector).
\n
The eye direction is the opposite of the vertex/fragment position since the vertex/fragment position is relative to the camera's position. Remember that the vertex/fragment position is in view space. So instead of going from the camera (eye) to the vertex/fragment, you go from the vertex/fragment to the eye (camera).
\n
The reflection vector is a reflection of the light direction at the surface normal. As the light \"ray\" hits the surface, it bounces off at the same angle it came in at. The angle between the light direction vector and the normal is known as the \"angle of incidence\". The angle between the reflection vector and the normal is known as the \"angle of reflection\".
\n
You'll have to negate the reflected light vector as it needs to point in the same direction as the eye vector. Remember the eye direction is from the vertex/fragment to the camera position. You'll use the reflection vector to calculate the intensity of the specular highlight.
The diffuse intensity is the dot product between the surface normal and the unit vector light direction. The dot product can range from negative one to one. If both vectors point in the same direction, the intensity is one. Any other case will be less than one.
\n
\n\n
\n\n
As the light vector approaches the same direction as the normal, the diffuse intensity approaches one.
You can now calculate the diffuse color contributed by this light. If the diffuse intensity is one, the diffuse color will be a mix between the diffuse texture color and the lights color. Any other intensity will cause the diffuse color to be darker.
\n
Notice how I clamp the diffuse color to be only as bright as the diffuse texture color is. This will protect the scene from being over exposed. When creating your diffuse textures, make sure to create them as if they were fully lit.
The specular intensity is the dot product between the eye vector and the reflection vector. As with the diffuse intensity, if the two vectors point in the same direction, the specular intensity is one. Any other intensity will diminish the amount of specular color contributed by this light.
\n
\n\n
\n\n
The material shininess determines how spread out the specular highlight is. This is typically set in a modeling program like Blender. In Blender it's known as the specular hardness.
This snippet keeps fragments outside of a spotlight's cone or frustum from being affected by the light. Fortunately, Panda3D sets upspotDirection and spotCosCutoff to also work for directional lights and points lights. Spotlights have both a position and direction. However, directional lights only have a direction and point lights only have a position. Still, this code works for all three lights avoiding the need for noisy if statements.
\n
// ...\n\n , -unitLightDirection\n\n// ...
\n
You must negate unitLightDirection. unitLightDirection goes from the fragment to the spotlight and you need it to go from the spotlight to the fragment since the spotDirection goes directly down the center of the spotlight's frustum some distance away from the spotlight's position.
For a spotlight, if the dot product between the fragment-to-light vector and the spotlight's direction vector is less than the cosine of half the spotlight's field of view angle, the shader disregards this light's influence.
\n
For directional lights and point lights, Panda3D sets spotCosCutoff to negative one. Recall that the dot product ranges from negative one to one. So it doesn't matter what the unitLightDirectionDelta is because it will always be greater than or equal to negative one.
Like the unitLightDirectionDelta snippet, this snippet also works for all three light types. For spotlights, this will make the fragments brighter as you move closer to the center of the spotlight's frustum. For directional lights and point lights, spotExponent is zero. Recall that anything to the power of zero is one so the diffuse color is one times itself meaning it is unchanged.
Panda3D makes applying shadows relatively easy by providing the shadow map and shadow transformation matrix for every scene light. To create the shadow transformation matrix yourself, you'll need to assemble a matrix that transforms view space coordinates to light space (coordinates are relative to the light's position). To create the shadow map yourself, you'll need to render the scene from the perspective of the light to a framebuffer texture. The framebuffer texture must hold the distances from the light to the fragments. This is known as a \"depth map\". Lastly, you'll need to manually give to your shader your DIY depth map as a uniform sampler2DShadow and your DIY shadow transformation matrix as a uniform mat4. At this point, you've recreated what Panda3D does for you automatically.
\n
The shadow snippet shown uses textureProj which is different from the texure function shown earlier. textureProj first divides vertexInShadowSpaces[i].xyz by vertexInShadowSpaces[i].w. After this, it uses vertexInShadowSpaces[i].xy to locate the depth stored in the shadow map. Next it uses vertexInShadowSpaces[i].z to compare this vertex's depth against the shadow map depth at vertexInShadowSpaces[i].xy. If the comparison passes, textureProj will return one. Otherwise, it will return zero. Zero meaning this vertex/fragment is in the shadow and one meaning this vertex/fragment is not in the shadow.
\n
textureProj can also return a value between zero and one depending on how the shadow map was set up. In this instance, textureProj performs multiple depth tests using neighboring depth values and returns a weighted average. This weighted average can give shadows a softer look.
The light's distance is just the magnitude or length of the light direction vector. Notice it's not using the normalized light direction as that distance would be one.
\n
You'll need the light distance to calculate the attenuation. Attenuation meaning the light's influence diminishes as you get further away from it.
\n
You can set constantAttenuation, linearAttenuation, and quadraticAttenuation to whatever values you would like. A good starting point is constantAttenuation = 1, linearAttenuation = 0, and quadraticAttenuation = 1. With these settings, the attenuation is one at the light's position and approaches zero as you move further away.
To calculate the final light color, add the diffuse and specular together. Be sure to add this to the accumulator as you loop through the scene's lights.
The ambient component to the lighting model is based on the material's ambient color, the ambient light's color, and the diffuse texture color.
\n
There should only ever be one ambient light. Because of this, the ambient color calculation only needs to occur once. Contrast this with the diffuse and specular color which must be accumulated for each spot/directional/point light. When you reach SSAO, you'll revisit the ambient color calculation.
The lookup table or LUT shader allows you to transform the colors of your game using an image editor like the GIMP. From color grading to turning day into night, the LUT shader is a handy tool for tweaking the look of your game.
\n
\n\n
\n\n
Before you can get started, you'll need to find a neutral LUT image. Neutral meaning that it leaves the fragment colors unchanged. The LUT needs to be 256 pixels wide by 16 pixels tall and contain 16 blocks with each block being 16 by 16 pixels.
\n
The LUT is mapped out into 16 blocks. Each block has a different level of blue. As you move across the blocks, from left to right, the amount of blue increases. You can see the amount of blue in each block's upper-left corner. Within each block, the amount of red increases as you move from left to right and the amount of green increases as you move from top to bottom. The upper-left corner of the first block is black since every RGB channel is zero. The lower-right corner of the last block is white since every RGB channel is one.
\n
\n\n
\n\n
With the neutral LUT in hand, take a screenshot of your game and open it in your image editor. Add the neutral LUT as a new layer and merge it with the screenshot. As you manipulate the colors of the screenshot, the LUT will be altered in the same way. When you're done editing, select only the LUT and save it as a new image. You now have your new lookup table and can begin writing your shader.
The LUT shader is a screen space technique. Therefore, sample the scene's color at the current fragment or screen position.
\n
// ...\n\nfloat u = floor(color.b * 15.0) / 15.0 * 240.0;\n u = (floor(color.r * 15.0) / 15.0 * 15.0) + u;\n u /= 255.0;\n\nfloat v = ceil(color.g * 15.0);\n v /= 15.0;\n v = 1.0 - v;\n\n// ...
\n
In order to transform the current fragment's color, using the LUT, you'll need to map the color to two UV coordinates on the lookup table texture. The first mapping (shown up above) is to the nearest left or lower bound block location and the second mapping (shown below) is to the nearest right or upper bound block mapping. At the end, you'll combine these two mappings to create the final color transformation.
\n
\n\n
\n\n
Each of the red, green, and blue channels maps to one of 16 possibilities in the LUT. The blue channel maps to one of the 16 upper-left block corners. After the blue channel maps to a block, the red channel maps to one of the 16 horizontal pixel positions within the block and the green channel maps to one of the 16 vertical pixel positions within the block. These three mappings will determine the UV coordinate you'll need to sample a color from the LUT.
\n
// ...\n\n u /= 255.0;\n\n v /= 15.0;\n v = 1.0 - v;\n\n// ...
\n
To calculate the final U coordinate, divide it by 255 since the LUT is 256 pixels wide and U ranges from zero to one. To calculate the final V coordinate, divide it by 15 since the LUT is 16 pixels tall and V ranges from zero to one. You'll also need to subtract the normalized V coordinate from one since V ranges from zero at the bottom to one at the top while the green channel ranges from zero at the top to 15 at the bottom.
\n
// ...\n\n vec3 left = texture(lookupTableTexture, vec2(u, v)).rgb;\n\n// ...
\n
Using the UV coordinates, sample a color from the lookup table. This is the nearest left block color.
\n
// ...\n\n u = ceil(color.b * 15.0) / 15.0 * 240.0;\n u = (ceil(color.r * 15.0) / 15.0 * 15.0) + u;\n u /= 255.0;\n\n v = 1.0 - (ceil(color.g * 15.0) / 15.0);\n\n vec3 right = texture(lookupTableTexture, vec2(u, v)).rgb;\n\n// ...
\n
Now you'll need to calculate the UV coordinates for the nearest right block color. Notice how ceil or ceiling is being used now instead of floor.
Not every channel will map perfectly to one of its 16 possibilities. For example, 0.5 doesn't map perfectly. At the lower bound (floor), it maps to 0.4666666666666667 and at the upper bound (ceil), it maps to 0.5333333333333333. Compare that with 0.4 which maps to 0.4 at the lower bound and 0.4 at the upper bound. For those channels which do not map perfectly, you'll need to mix the left and right sides based on where the channel falls between its lower and upper bound. For 0.5, it falls directly between them making the final color a mixture of half left and half right. However, for 0.132 the mixture will be 98% right and 2% left since the fractional part of 0.123 times 15.0 is 0.98.
\n
// ...\n\n fragColor = color;\n\n// ...
\n
Set the fragment color to the final mix and you're done.
To really sell the illusion of speed, you can do no better than motion blur. From high speed car chases to moving at warp speed, motion blur greatly improves the look and feel of fast moving objects.
\n
There are a few ways to implement motion blur as a screen space technique. The less involved implementation will only blur the scene in relation to the camera's movements while the more involved version will blur any moving objects even with the camera remaining still. The less involved technique is described below but the principle is the same.
The motion blur technique determines the blur direction by comparing the previous frame's vertex positions with the current frame's vertex positions. To do this, you'll need the previous frame's view-to-world matrix, the current frame's world-to-view matrix, and the camera lens' projection matrix.
The adjustable parameters are size and separation. size controls how many samples are taken along the blur direction. Increasing size increases the amount of blur at the cost of performance. separation controls how spread out the samples are along the blur direction. Increasing separation increases the amount of blur at the cost of accuracy.
To determine which way to blur this fragment, you'll need to know where things were last frame and where things are this frame. To figure out where things are now, sample the current vertex position. To figure out where things were last frame, transform the current position from view space to world space, using the previous frame's view-to-world matrix, and then transform it back to view space from world space using this frame's world-to-view matrix. This transformed position is this fragment's previous interpolated vertex position.
Now that you have the current and previous positions, transform them to screen space. With the positions in screen space, you can trace out the 2D direction you'll need to blur the onscreen image.
For a more seamless blur, sample in the direction of the blur and in the opposite direction of the blur. For now, set the two vectors to the fragment's UV coordinate.
\n
// ...\n\nfloat count = 1.0;\n\n// ...
\n
count is used to average all of the samples taken. It starts at one since you've already sampled the current fragment's color.
Normal mapping allows you to add surface details without adding any geometry. Typically, in a modeling program like Blender, you create a high poly and a low poly version of your mesh. You take the vertex normals from the high poly mesh and bake them into a texture. This texture is the normal map. Then inside the fragment shader, you replace the low poly mesh's vertex normals with the high poly mesh's normals you baked into the normal map. Now when you light your mesh, it will appear to have more polygons than it really has. This will keep your FPS high while at the same time retain most of the details from the high poly version.
\n
\n\n
\n\n
Here you see the progression from the high poly model to the low poly model to the low poly model with the normal map applied.
\n
\n\n
\n\n
Keep in mind though, normal mapping is only an illusion. After a certain angle, the surface will look flat again.
Starting in the vertex shader, you'll need to output to the fragment shader the normal vector, binormal vector, and the tangent vector. These vectors are used, in the fragment shader, to transform the normal map normal from tangent space to view space.
\n
p3d_NormalMatrix transforms the vertex normal, binormal, and tangent vectors to view space. Remember that in view space, all of the coordinates are relative to the camera's position.
\n
\n[p3d_NormalMatrix] is the upper 3x3 of the inverse transpose of the ModelViewMatrix.\nIt is used to transform the normal vector into view-space coordinates.\n \n \n\n
You'll also need to output, to the fragment shader, the UV coordinates for the normal map.
\n
Fragment
\n
Recall that the vertex normal was used to calculate the lighting. However, the normal map provides us with different normals to use when calculating the lighting. In the fragment shader, you need to swap out the vertex normals for the normals found in the normal map.
Earlier I showed how the normals are mapped to colors to create the normal map. Now this process needs to be reversed so you can get back the original normals that were baked into the map.
\n
(r, g, b) =\n ( r * 2 - 1\n , g * 2 - 1\n , b * 2 - 1\n ) =\n (x, y, z)
\n
Here's the process for unpacking the normals from the normal map.
The normals you get back from the normal map are typically in tangent space. They could be in another space, however. For example, Blender allows you to bake the normals in tangent, object, world, or camera space.
\n
\n\n
\n\n
To take the normal map normal from tangent space to view pace, construct a three by three matrix using the tangent, binormal, and vertex normal vectors. Multiply the normal by this matrix and be sure to normalize it.
\n
At this point, you're done. The rest of the lighting calculations are the same.
Outlining your scene's geometry can give your game a distinctive look, reminiscent of comic books and cartoons.
\n
Discontinuities
\n
The process of outlining is the process of finding and labeling discontinuities or differences. Every time you find what you consider a significant difference, you mark it with your line color. As you go about labeling or coloring in the differences, outlines or edges will start to form.
\n
Where you choose to search for the discontinuities is up to you. It could be the diffuse colors in your scene, the normals of your models, the depth buffer, or some other scene related data.
\n
The demo uses the interpolated vertex positions to render the outlines. However, a less straightforward but more typical way is to use both the scene's normals and depth buffer values to construct the outlines.
The demo darkens the colors of the scene where there's an outline. This tends to look nicer than a constant color since it provides some color variation to the edges.
The min and max separation parameters control the thickness of the outline depending on the fragment's distance from the camera or depth. The min and max distance control the significance of any changes found. The size parameter controls the constant thickness of the line no matter the fragment's position. The outline color is based on colorModifier and the current fragment's color.
Sample the position texture for the current fragment's position in the scene. Recall that the position texture is just a screen shaped quad making the UV coordinate the current fragment's screen coordinate divided by the dimensions of the screen.
The fragment's depth ranges from zero to one. When the fragment's view-space y coordinate matches the far clipping plane, the depth is one. When the fragment's view-space y coordinate matches the near clipping plane, the depth is zero. In other words, the depth ranges from zero at the near clipping plane all the way up to one at the far clipping plane.
Converting the position to a depth value isn't necessary but it allows you to vary the thickness of the outline based on how far away the fragment is from the camera. Far away fragments get a thinner line while nearer fragments get a thicker outline. This tends to look nicer than a constant thickness since it gives depth to the outline.
Calculate the significance of any difference discovered using the minDistance, maxDistance, and smoothstep. smoothstep returns values from zero to one. The minDistance is the left-most edge. Any difference less than the minimum distance will be zero. The maxDistance is the right-most edge. Any difference greater than the maximum distance will be one. For distances between the edges, the difference will be between zero and one. These values are interpolated along a s-shaped curve.
Start by creating a RGB noise texture. A good size is either 128 by 128 or 512 by 512. Be sure to blur it and make it tileable. This will produce a nice wavy, inky outline.
\n
// ...\n\nfloat noiseScale = 10.0;\n\n// ...
\n
The noiseScale parameter controls how distorted the outline is. The bigger the noiseScale, the sketchier the line.
Sample the noise texture using the current screen/fragment position and the size of the noise texture. Since you're distorting the UV coordinates used to sample the position vectors, you'll only need two of the three color channels. Map the two color channels from [0, 1] to [-1, 1]. Finally, scale the noise by the scale chosen earlier.
When sampling the surrounding positions inside the loop, add the noise vector to the current fragment's coordinates. The rest of the calculations are the same.
Pixelizing your 3D game can give it a interesting look and possibly save you time by not having to create all of the pixel art by hand. Combine it with the posterization for a true retro look.
\n
// ...\n\nint pixelSize = 5;\n\n// ...
\n
Feel free to adjust the pixel size. The bigger the pixel size, the blockier the image will be.
\n
\n\n
\n\n
// ...\n\nfloat x = int(gl_FragCoord.x) % pixelSize;\nfloat y = int(gl_FragCoord.y) % pixelSize;\n\n x = floor(pixelSize / 2.0) - x;\n y = floor(pixelSize / 2.0) - y;\n\n x = gl_FragCoord.x + x;\n y = gl_FragCoord.y + y;\n\n// ...
\n
The technique works by mapping each fragment to the center of its closest, non-overlapping pixel-sized window. These windows are laid out in a grid over the input texture. The center-of-the-window fragments determine the color for the other fragments in their window.
Posterization or color quantization is the process of reducing the number of unique colors in an image. You can use this shader to give your game a comic book or retro look. Combine it with outlining for a full-on cartoon art style.
\n
There are various different ways to implement posterization. This method works directly with the greyscale values and indirectly with the RGB values of the image. For each fragment, it maps the RGB color to a greyscale value. This greyscale value is then mapped to both its lower and upper level value. The closest level to the original greyscale value is then mapped back to an RGB value This new RGB value becomes the fragment color. I find this method produces nicer results than the more typical methods you'll find.
\n
// ...\n\nfloat levels = 10;\n\n// ...
\n
The levels parameter controls how many discrete bands or steps there are. This will break up the continuous values from zero to one into chunks. With four levels, 0.0 to 1.0 becomes 0.0, 0.25, 0.5, 0.75, and 1.0.
Map the greyscale value to its lower level and then calculate the difference between its lower level and itself. For example, if the greyscale value is 0.87 and there are four levels, its lower level is 0.75 and the difference is 0.12.
The closest level is used to calculate the adjustment. The adjustment is the ratio between the quantized and unquantized greyscale value. This adjustment is used to map the quantized greyscale value back to an RGB value.
\n
// ...\n\n fragColor.rgb * adjustment;\n\n// ...
\n
After multiplying rgb by the adjustment, max(r, max(g, b)) will now equal the quantized greyscale value. This maps the quantized greyscale value back to a red, green, and blue vector.
Before you write any shaders, you should be familiar with the following frames of reference or coordinate systems. All of them boil down to what origin (0, 0, 0) are these coordinates currently relative to? Once you know that, you can then transform them, via some matrix, to some other vector space if need be. Typically, when the output of some shader looks wrong, it's because of some coordinate system mix up.
\n
Model
\n
\n\n
\n\n
The model or object coordinate system is relative to the origin of the model. This is typically set to the center of the model's geometry in a modeling program like Blender.
\n
World
\n
\n\n
\n\n
The world space is relative to the origin of the scene/level/universe that you've created.
\n
View
\n
\n\n
\n\n
The view or eye coordinate space is relative to the position of the active camera.
\n
Clip
\n
\n\n
\n\n
The clip space is relative to the center of the camera's film. All coordinates are now homogeneous, ranging from negative one to one (-1, 1). X and y are parallel with the camera's film and the z coordinate is the depth.
\n
\n\n
\n\n
Any vertex not within the bounds of the camera's frustum or view volume is clipped or discarded. You can see this happening with the cube towards the back, clipped by the camera's far plane, and the cube off to the side.
\n
Screen
\n
\n\n
\n\n
The screen space is (typically) relative to the lower left corner of the screen. X goes from zero to the screen width. Y goes from zero to the screen height.
Instead of rendering/drawing/painting directly to the screen, the example code uses a technique called \"render to texture\". In order to render to a texture, you'll need to set up a framebuffer and bind a texture to it. Multiple textures can be bound to a single framebuffer.
\n
The textures bound to the framebuffer hold the vector(s) returned by the fragment shader. Typically these vectors are color vectors (r, g, b, a) but they could also be position or normal vectors (x, y, z, w). For each bound texture, the fragment shader can output a different vector. For example you could output a vertex's position and normal in a single pass.
\n
Most of the example code dealing with Panda3D involves setting up framebuffer textures. To keep things straightforward, nearly all of the fragment shaders in the example code have only one output. However, you'll want to output as much as you can each render pass to keep your frames per second (FPS) high.
\n
There are two framebuffer texture setups found in the example code.
\n
\n\n
\n\n
The first setup renders the mill scene into a framebuffer texture using a variety of vertex and fragment shaders. This setup will go through each of the mill scene's vertexes and corresponding fragments.
\n
In this setup, the example code performs the following.
\n
\n
Stores geometry data (like vertex position or normal) for later use.
\n
Stores material data (like the diffuse color) for later use.
\n
UV maps the various textures (diffuse, normal, shadow, etc.).
\n
Calculates the ambient, diffuse, specular, and emission lighting.
\n
\n
\n\n
\n\n
The second setup is an orthographic camera pointed at a screen-shaped rectangle. This setup will go through just the four vertexes and their corresponding fragments.
\n
In this second setup, the example code performs the following.
\n
\n
Manipulates the output of another framebuffer texture.
\n
Combines various framebuffer textures into one.
\n
\n
I like to think of this second setup as using layers in GIMP, Krita, or Inkscape.
\n
\n\n
\n\n
In the example code, you can see the output of a particular framebuffer texture by using the Tab key or the Shift+Tab keys.
Taking inspiration from the fresnel factor, rim lighting targets the rim or silhouette of an object. When combined with cel shading and outlining, it can really complete that cartoon look. You can also use it to highlight objects in the game, making it easier for players to navigate and accomplish tasks.
As it was for the fresnel factor, you'll need the eye vector. If your vertex positions are in view space, the eye vector is the negation of the vertex position.
The Intensity of the rim light ranges from zero to one. When the eye and normal vector point in the same direction, the rim light intensity is zero. As the two vectors start to point in different directions, the rim light intensity increases until it eventually reaches one when the eye and normal become orthogonal or perpendicular to one another.
step or smoothstep can also be used to control the falloff. This tends to look better when using cel shading. You'll learn more about these functions in later sections.
What color you use for the rim light is up to you. The demo code multiplies the diffuse light by the rimLightIntensity. This will highlight the silhouette without overexposing it and without lighting any shadowed fragments.
After you've built the example code, you can now run the executable or demo.
\n
./3d-game-shaders-for-beginners
\n
Here's how you run it on Linux or Mac.
\n
3d-game-shaders-for-beginners.exe
\n
Here's how you run it on Windows.
\n
Demo Controls
\n
The demo comes with both keyboard and mouse controls to move the camera around, toggle on and off the different effects, adjust the fog, and view the various different framebuffer textures.
\n
Mouse
\n
You can rotate the scene around by holding down the Left Mouse button and dragging. Hold down the Right Mouse button and drag to move up, down, left, and/or right. To zoom in, roll the Mouse Wheel forward. To zoom out, roll the Mouse Wheel backward.
\n
You can also change the focus point using the mouse. To change the focus point, click anywhere on the scene using the Middle Mouse button.
\n
Keyboard
\n
\n
w to rotate the scene down.
\n
a to rotate the scene clockwise.
\n
s to rotate the scene up.
\n
d to rotate the scene counterclockwise.
\n
z to zoom in to the scene.
\n
x to zoom out of the scene.
\n
⬅ to move left.
\n
➡ to move right.
\n
⬆ to move up.
\n
⬇ to move down.
\n
\n\n\n
\n
1 to show midday.
\n
2 to show midnight.
\n
\n\n\n
\n
Delete to toggle the sound.
\n
3 to toggle fresnel.
\n
4 to toggle rim lighting.
\n
5 to toggle particles.
\n
6 to toggle motion blur.
\n
7 to toggle Kuwahara filtering.
\n
8 to toggle cel shading.
\n
9 to toggle lookup table processing.
\n
0 to toggle between Phong and Blinn-Phong.
\n
y to toggle SSAO.
\n
u to toggle outlining.
\n
i to toggle bloom.
\n
o to toggle normal mapping.
\n
p to toggle fog.
\n
h to toggle depth of field.
\n
j to toggle posterization.
\n
k to toggle pixelization.
\n
l to toggle sharpen.
\n
n to toggle film grain.
\n
m to toggle screen space reflection.
\n
, to toggle screen space refraction.
\n
. to toggle flow mapping.
\n
/ to toggle the sun animation.
\n
\\ to toggle chromatic aberration.
\n
\n\n\n
\n
r to reset the scene.
\n
\n\n\n
\n
[ to decrease the fog near distance.
\n
Shift+[ to increase the fog near distance.
\n
] to increase the fog far distance.
\n
Shift+] to decrease the fog far distance.
\n
\n\n\n
\n
Shift+- to decrease the amount of foam.
\n
- to increase the amount of foam.
\n
\n\n\n
\n
Shift+= to decrease the relative index of refraction.
\n
= to increase the relative index of refraction.
\n
\n\n\n
\n
Tab to move forward through the framebuffer textures.
\n
Shift+Tab to move backward through the framebuffer textures.
Adding reflections can really ground your scene. Wet and shiny objects spring to life as nothing makes something look wet or shiny quite like reflections. With reflections, you can really sell the illusion of water and metallic objects.
\n
In the lighting section, you simulated the reflected, mirror-like image of the light source. This was the process of rendering the specular reflection. Recall that the specular light was computed using the reflected light direction. Similarly, using screen space reflection or SSR, you can simulate the reflection of other objects in the scene instead of just the light source. Instead of the light ray coming from the source and bouncing off into the camera, the light ray comes from some object in the scene and bounces off into the camera.
\n
SSR works by reflecting the screen image onto itself using only itself. Compare this to cube mapping which uses six screens or textures. In cube mapping, you reflect a ray from some point in your scene to some point on the inside of a cube surrounding your scene. In SSR, you reflect a ray from some point on your screen to some other point on your screen. By reflecting your screen onto itself, you can create the illusion of reflection. This illusion holds for the most part but SSR does fail in some cases as you'll see.
\n
Ray Marching
\n
Screen space reflection uses a technique known as ray marching to determine the reflection for each fragment. Ray marching is the process of iteratively extending or contracting the length or magnitude of some vector in order to probe or sample some space for information. The ray in screen space reflection is the position vector reflected about the normal.
\n
Intuitively, a light ray hits some point in the scene, bounces off, travels in the opposite direction of the reflected position vector, bounces off the current fragment, travels in the opposite direction of the position vector, and hits the camera lens allowing you to see the color of some point in the scene reflected in the current fragment. SSR is the process of tracing the light ray's path in reverse. It tries to find the reflected point the light ray bounced off of and hit the current fragment. With each iteration, the algorithm samples the scene's positions or depths, along the reflection ray, asking each time if the ray intersected with the scene's geometry. If there is an intersection, that position in the scene is a potential candidate for being reflected by the current fragment.
\n
Ideally there would be some analytical method for determining the first intersection point exactly. This first intersection point is the only valid point to reflect in the current fragment. Instead, this method is more like a game of battleship. You can't see the intersections (if there are any) so you start at the base of the reflection ray and call out coordinates as you travel in the direction of the reflection. With each call, you get back an answer of whether or not you hit something. If you do hit something, you try points around that area hoping to find the exact point of intersection.
\n
\n\n
\n\n
Here you see ray marching being used to calculate each fragment's reflected point. The vertex normal is the bright green arrow, the position vector is the bright blue arrow, and the bright red vector is the reflection ray marching through view space.
\n
Vertex Positions
\n
Like SSAO, you'll need the vertex positions in view space. Referrer back to SSAO for details.
\n
Vertex Normals
\n
To compute the reflections, you'll need the vertex normals in view space. Referrer back to SSAO for details.
\n
\n\n
\n\n
Here you see SSR using the normal mapped normals instead of the vertex normals. Notice how the reflection follows the ripples in the water versus the more mirror like reflection shown earlier.
\n
To use the normal maps instead, you'll need to transform the normal mapped normals from tangent space to view space just like you did in the lighting calculations. You can see this being done in normal.frag.
\n
Position Transformations
\n
\n\n
\n\n
Just like SSAO, SSR goes back and forth between the screen and view space. You'll need the camera lens' projection matrix to transform points in view space to clip space. From clip space, you'll have to transform the points again to UV space. Once in UV space, you can sample a vertex/fragment position from the scene which will be the closest position in the scene to your sample. This is the screen space part in screen space reflection since the \"screen\" is a texture UV mapped over a screen shaped rectangle.
\n
Reflected UV Coordinates
\n
There are a few ways you can implement SSR. The example code starts the reflection process by computing a reflected UV coordinate for each screen fragment. You could skip this part and go straight to computing the reflected color instead, using the final rendering of the scene.
\n
Recall that UV coordinates range from zero to one for both U and V. The screen is just a 2D texture UV mapped over a screen-sized rectangle. Knowing this, the example code doesn't actually need the final rendering of the scene to compute the reflections. It can instead calculate what UV coordinate each screen pixel will eventually use. These calculated UV coordinates can be saved to a framebuffer texture and used later when the scene has been rendered.
\n
\n\n
\n\n
Here you see the reflected UV coordinates. Without even rendering the scene yet, you can get a good feel for what the reflections will look like.
Like the other effects, SSR has a few parameters you can adjust. Depending on the complexity of the scene, it may take you awhile to find the right settings. Getting screen space reflections to look just right tends to be difficult when reflecting complex geometry.
\n
The maxDistance parameter controls how far a fragment can reflect. In other words, it controls the maximum length or magnitude of the reflection ray.
\n
The resolution parameter controls how many fragments are skipped while traveling or marching the reflection ray during the first pass. This first pass is to find a point along the ray's direction where the ray enters or goes behind some geometry in the scene. Think of this first pass as the rough pass. Note that the resolution ranges from zero to one. Zero will result in no reflections while one will travel fragment-by-fragment along the ray's direction. A resolution of one can slow down your FPS considerably especially with a large maxDistance.
\n
The steps parameter controls how many iterations occur during the second pass. This second pass is to find the exact point along the reflection ray's direction where the ray immediately hits or intersects with some geometry in the scene. Think of this second pass as the refinement pass.
\n
The thickness controls the cutoff between what counts as a possible reflection hit and what does not. Ideally, you'd like to have the ray immediately stop at some camera-captured position or depth in the scene. This would be the exact point where the light ray bounced off, hit your current fragment, and then bounced off into the camera. Unfortunately the calculations are not always that precise so thickness provides some wiggle room or tolerance. You'll want the thickness to be as small as possible—just a short distance beyond a sampled position or depth.
\n
\n\n
\n\n
You'll find that as the thickness gets larger, the reflections tend to smear in places.
\n
\n\n
\n\n
Going in the other direction, as the thickness gets smaller, the reflections become noisy with tiny little holes and narrow gaps.
Gather the current fragment's position, normal, and reflection about the normal. positionFrom is a vector from the camera position to the current fragment position. normal is a vector pointing in the direction of the interpolated vertex normal for the current fragment. pivot is the reflection ray or vector pointing in the reflected direction of the positionFrom vector. It currently has a length or magnitude of one.
Project or transform these start and end points from view space to screen space. These points are now fragment positions which correspond to pixel positions on the screen. Now that you know where the ray starts and ends on the screen, you can travel or march along its direction in screen space. Think of the ray as a line drawn on the screen. You'll travel along this line using it to sample the fragment positions stored in the position framebuffer texture.
\n
\n\n
\n\n
Note that you could march the ray through view space but this may under or over sample scene positions found in the position framebuffer texture. Recall that the position framebuffer texture is the size and shape of the screen. Every screen fragment or pixel corresponds to some position captured by the camera. A reflection ray may travel a long distance in view space, but in screen space, it may only travel through a few pixels. You can only sample the screen's pixels for positions so it is inefficient to potentially sample the same pixels over and over again while marching in view space. By marching in screen space, you'll more efficiently sample the fragments or pixels the ray actually occupies or covers.
The first pass will begin at the starting fragment position of the reflection ray. Convert the fragment position to a UV coordinate by dividing the fragment's coordinates by the position texture's dimensions.
Calculate the delta or difference between the X and Y coordinates of the end and start fragments. This will be how many pixels the ray line occupies in the X and Y dimension of the screen.
To handle all of the various different ways (vertical, horizontal, diagonal, etc.) the line can be oriented, you'll need to keep track of and use the larger difference. The larger difference will help you determine how much to travel in the X and Y direction each iteration, how many iterations are needed to travel the entire line, and what percentage of the line does the current position represent.
\n
useX is either one or zero. It is used to pick the X or Y dimension depending on which delta is bigger. delta is the larger delta of the two X and Y deltas. It is used to determine how much to march in either dimension each iteration and how many iterations to take during the first pass.
Calculate how much to increment the X and Y position by using the larger of the two deltas. If the two deltas are the same, each will increment by one each iteration. If one delta is larger than the other, the larger delta will increment by one while the smaller one will increment by less than one. This assumes the resolution is one. If the resolution is less than one, the algorithm will skip over fragments.
To move from the start fragment to the end fragment, the algorithm uses linear interpolation.
\n
current position x = (start x) * (1 - search1) + (end x) * search1;\ncurrent position y = (start y) * (1 - search1) + (end y) * search1;
\n
search1 ranges from zero to one. When search1 is zero, the current position is the start fragment. When search1 is one, the current position is the end fragment. For any other value, the current position is somewhere between the start and end fragment.
\n
search0 is used to remember the last position on the line where the ray missed or didn't intersect with any geometry. The algorithm will later use search0 in the second pass to help refine the point at which the ray touches the scene's geometry.
\n
// ...\n\nint hit0 = 0;\nint hit1 = 0;\n\n// ...
\n
hit0 indicates there was an intersection during the first pass. hit1 indicates there was an intersection during the second pass.
The viewDistance value is how far away from the camera the current point on the ray is. Recall that for Panda3D, the Y dimension goes in and out of the screen in view space. For other systems, the Z dimension goes in and out of the screen in view space. In any case, viewDistance is how far away from the camera the ray currently is. Note that if you use the depth buffer, instead of the vertex positions in view space, the viewDistance would be the Z depth.
\n
Make sure not to confuse the viewDistance value with the Y dimension of the line being traveled across the screen. The viewDistance goes from the camera into scene while the Y dimension of the line travels up or down the screen.
\n
The depth is the view distance difference between the current ray point and scene position. It tells you how far behind or in front of the scene the ray currently is. Remember that the scene positions are the interpolated vertex positions stored in the position framebuffer texture.
\n
// ...\n\nfor (i = 0; i < int(delta); ++i) {\n\n// ...
\n
You can now begin the first pass. The first pass runs while i is less than the delta value. When i reaches delta, the algorithm has traveled the entire length of the line. Remember that delta is the larger of the two X and Y deltas.
Advance the current fragment position closer to the end fragment. Use this new fragment position to look up a scene position stored in the position framebuffer texture.
Calculate the percentage or portion of the line the current fragment represents. If useX is zero, use the Y dimension of the line. If useX is one, use the X dimension of the line.
\n
When frag equals startFrag, search1 equals zero since frag - startFrag is zero. When frag equals endFrag, search1 is one since frag - startFrag equals delta.
\n
search1 is the percentage or portion of the line the current position represents. You'll need this to interpolate between the ray's view-space start and end distances from the camera.
You may be tempted to just interpolate between the view distances of the start and end view-space positions but this will give you the wrong view distance for the current position on the reflection ray. Instead, you'll need to perform perspective-correct interpolation which you see here.
If the difference is between zero and the thickness, this is a hit. Set hit0 to one and exit the first pass. If the difference is not between zero and the thickness, this is a miss. Set search0 to equal search1 to remember this position as the last known miss. Continue marching the ray towards the end fragment.
At this point you have finished the first pass. Set the search1 position to be halfway between the position of the last miss and the position of the last hit.
\n
// ...\n\n steps *= hit0;\n\nfor (i = 0; i < steps; ++i) {\n\n// ...
\n
You can now begin the second pass. If the reflection ray didn't hit anything in the first pass, skip the second pass.
Interpolate the view distance for the current ray line position and calculate the camera distance difference between the ray at this point and the scene.
If the depth is within bounds, this is a hit. Set hit1 to one and set search1 to be halfway between the last known miss position and this current hit position. If the depth is not within bounds, this is a miss. Set search1 to be halfway between this current miss position and the last known hit position. Move search0 to this current miss position. Continue this back and forth search while i is less than steps.
\n
// ...\n\nfloat visibility =\n hit1\n\n// ...
\n
You're now done with the second and final pass but before you can output the reflected UV coordinates, you'll need to calculate the visibility of the reflection. The visibility ranges from zero to one. If there wasn't a hit in the second pass, the visibility is zero.
\n
// ...\n\n * positionTo.w\n\n// ...
\n
If the reflected scene position's alpha or w component is zero, the visibility is zero. Note that if w is zero, there was no scene position at that point.
One of the ways in which screen space reflection can fail is when the reflection ray points in the general direction of the camera. If the reflection ray points towards the camera and hits something, it's most likely hitting the back side of something facing away from the camera.
\n
To handle this failure case, you'll need to gradually fade out the reflection based on how much the reflection vector points to the camera's position. If the reflection vector points directly in the opposite direction of the position vector, the visibility is zero. Any other direction results in the visibility being greater than zero.
\n
Remember to normalize both vectors when taking the dot product. unitPositionFrom is the normalized position vector. It has a length or magnitude of one.
As you sample scene positions along the reflection ray, you're hoping to find the exact point where the reflection ray first intersects with the scene's geometry. Unfortunately, you may not find this particular point. Fade out the reflection the further it is from the intersection point you did find.
Fade out the reflection based on how far way the reflected point is from the initial starting point. This will fade out the reflection instead of it ending abruptly as it reaches maxDistance.
If the reflected UV coordinates are out of bounds, set the visibility to zero. This occurs when the reflection ray travels outside the camera's frustum.
Set the blue and alpha component to the visibility as the UV coordinates only need the RG or XY components of the final vector.
\n
// ...\n\n fragColor = uv;\n\n// ...
\n
The final fragment color is the reflected UV coordinates and the visibility.
\n
Specular Map
\n
\n\n
\n\n
In addition to the reflected UV coordinates, you'll also need a specular map. The example code creates one using the fragment's material specular properties.
The specular fragment shader is quite simple. Using the fragment's material, the shader outputs the specular color and uses the alpha channel for the shininess. The shininess is mapped to a range of zero to one. In Blender, the maximum specular hardness or shininess is 511. When exporting from Blender to Panda3D, 511 is exported as 127.75. Feel free to adjust the shininess to range of zero to one however you see fit for your particular stack.
\n
The example code generates a specular map from the material specular properties but you could create one in GIMP, for example, and attach that as a texture to your 3D model. For instance, say your 3D treasure chest has shiny brackets on it but nothing else should reflect the environment. You can paint the brackets some shade of gray and the rest of the treasure chest black. This will mask off the brackets, allowing your shader to render the reflections on only the brackets and nothing else.
\n
Scene Colors
\n
\n\n
\n\n
You'll need to render the parts of the scene you wish to reflect and store this in a framebuffer texture. This is typically just the scene without any reflections.
\n
Reflected Scene Colors
\n
\n\n
\n\n
Here you see the reflected colors saved to a framebuffer texture.
Once you have the reflected UV coordinates, looking up the reflected colors is fairly easy. You'll need the reflected UV coordinates texture and the color texture containing the colors you wish to reflect.
Recall that the reflected UV texture stored the visibility in the B or blue component. This is the alpha channel for the reflected colors framebuffer texture.
The fragment color is a mix between no reflection and the reflected color based on the visibility. The visibility was computed during the reflected UV coordinates step.
\n
Blurred Reflected Scene Colors
\n
\n\n
\n\n
Now blur the reflected scene colors and store this in a framebuffer texture. The blurring is done using a box blur. Refer to the SSAO blurring step for details.
\n
The blurred reflected colors are used for surfaces that have a less than mirror like finish. These surfaces have tiny little hills and valleys that tend to diffuse or blur the reflection. I'll cover this more during the roughness calculation.
To generate the final reflections, you'll need the three framebuffer textures computed earlier. You'll need the reflected colors, the blurred reflected colors, and the specular map.
Map the specular color to a greyscale value. If the specular amount is none, set the frag color to nothing and return.
\n
Later on, you'll multiply the final reflection color by the specular amount. Multiplying by the specular amount allows you to control how much a material reflects its environment simply by brightening or darkening the greyscale value in the specular map.
Calculate the roughness based on the shininess value set during the specular map step. Recall that the shininess value was saved in the alpha channel of the specular map. The shininess determined how spread out or blurred the specular reflection was. Similarly, the roughness determines how blurred the reflection is. A roughness of one will produce the blurred reflection color. A roughness of zero will produce the non-blurred reflection color. Doing it this way allows you to control how blurred the reflection is just by changing the material's shininess value.
\n
The example code generates a roughness map from the material specular properties but you could create one in GIMP, for example, and attach that as a texture to your 3D model. For instance, say you have a tiled floor that has polished tiles and scratched up tiles. The polished tiles could be painted a more translucent white while the scratched up tiles could be painted a more opaque white. The more translucent/transparent the greyscale value, the more the shader will use the blurred reflected color. The scratched tiles will have a blurry reflection while the polished tiles will have a mirror like reflection.
Mix the reflected color and blurred reflected color based on the roughness. Multiply that vector by the specular amount and then set that value as the fragment color.
\n
The reflection color is a mix between the reflected scene color and the blurred reflected scene color based on the roughness. A high roughness will produce a blurry reflection meaning the surface is rough. A low roughness will produce a clear reflection meaning the surface is smooth.
Screen space refraction, much like screen space reflection, adds a touch of realism you can't find anywhere else. Glass, plastic, water, and other transparent/translucent materials spring to life.
\n
Screen space reflection and screen space refraction work almost identically expect for one major difference. Instead of using the reflected vector, screen space refraction uses the refracted vector. It's a slight change in code but a big difference visually.
\n
Vertex Positions
\n
Like SSAO, you'll need the vertex positions in view space. Referrer back to SSAO for details.
\n
However, unlike SSAO, you'll need the scene's vertex positions with and without the refractive objects. Refractive surfaces are translucent, meaning you can see through them. Since you can see through them, you'll need the vertex positions behind the refractive surface. Having both the foreground and background vertex positions will allow you to calculate UV coordinates and depth.
\n
Vertex Normals
\n
To compute the refractions, you'll need the scene's foreground vertex normals in view space. The background vertex normals aren't needed unless you need to incorporate the background surface detail while calculating the refracted UV coordinates and distances. Referrer back to SSAO for details.
\n
\n\n
\n\n
Here you see the water refracting the light with and without normal maps. If available, be sure to use the normal mapped normals instead of the vertex normals. The smoother and flatter the surface, the harder it is to tell the light is being refracted. There will be some distortion but not enough to make it worthwhile.
\n
To use the normal maps instead, you'll need to transform the normal mapped normals from tangent space to view space just like you did in the lighting calculations. You can see this being done in normal.frag.
\n
Position Transformations
\n
\n\n
\n\n
Just like SSAO and screen space reflection, screen space refraction goes back and forth between the screen and view space. You'll need the camera lens' projection matrix to transform points in view space to clip space. From clip space, you'll have to transform the points again to UV space. Once in UV space, you can sample a vertex/fragment position from the scene which will be the closest position in the scene to your sample. This is the screen space part in screen space refraction since the \"screen\" is a texture UV mapped over a screen shaped rectangle.
\n
Refracted UV Coordinates
\n
Recall that UV coordinates range from zero to one for both U and V. The screen is just a 2D texture UV mapped over a screen-sized rectangle. Knowing this, the example code doesn't actually need the final rendering of the scene to compute the refraction. It can instead calculate what UV coordinate each screen pixel will eventually use. These calculated UV coordinates can be saved to a framebuffer texture and used later when the scene has been rendered.
\n
The process of refracting the UV coordinates is very similar to the process of reflecting the UV coordinates. Below are the adjustments you'll need to turn reflection into refraction.
Reflection only deals with what is in front of the reflective surface. Refraction, however, deals with what is behind the refractive surface. To accommodate this, you'll need both the vertex positions of the scene with the refracting surfaces taken out and the vertex positions of the scene with the refracting surfaces left in.
\n
\n\n
\n\n
positionFromTexture are the scene's vertex positions with the refracting surfaces left in. positionToTexture are the scene's vertex positions with the refracting surfaces taken out. normalFromTexture are the scene's vertex normals with the refraction surfaces left in. There's no need for the vertex normals behind the refractive surfaces unless you want to incorporate the surface detail for the background geometry.
\n
// ...\n\nuniform vec2 rior;\n\n// ...
\n
Refraction has one more adjustable parameter than reflection. rior is the relative index of refraction or relative refractive index. It is the ratio of the refraction indexes of two mediums. So for example, going from water to air is 1 / 1.33 ≈ 0.75. The numerator is the refractive index of the medium the light is leaving and the denominator is the refractive index of the medium the light is entering. An rior of one means the light passes right through without being refracted or bent. As rior grows, the refraction will become a reflection.
\n
There's no requirement that rior must adhere to the real world. The demo uses 1.05. This is completely unrealistic (light does not travel faster through water than air) but the realistic setting produced too many artifacts. In the end, the distortion only has to be believable—not realistic.
\n
\n\n
\n\n
rior values above one tend to elongate the refraction while numbers below one tend to shrink the refraction.
\n
As it was with screen space reflection, the screen doesn't have the entire geometry of the scene. A refracted ray may march through the screen space and never hit a captured surface. Or it may hit a surface but it's the backside not captured by the camera. When this happened during reflection, the fragment was left blank. This indicated no reflection or not enough information to determine a reflection. Leaving the fragment blank was fine for reflection since the reflective surface would fill in the gaps.
\n
\n\n
\n\n
For refraction, however, we must set the fragment to some UV. If the fragment is left blank, the refractive surface will contain holes that let the detail behind it come through. This would be okay for a completely transparent surface but usually the refractive surface will have some tint to it, reflection, etc.
The best choice is to select the UV as if the rior was one. This will leave the UV coordinate unchanged, allowing the background to show through instead of there being a hole in the refractive surface.
\n
\n\n
\n\n
Here you see the refracted UV texture for the mill scene. The wheel and waterway disturb what is otherwise a smooth gradient. The disruptions shift the UV coordinates from their screen position to their refracted screen position.
The most important difference is the calculation of the refracted vector versus the reflected vector. Both use the unit position and normal but refract takes an additional parameter specifying the relative refractive index.
The positionTo, sampled by the uv coordinates, uses the positionToTexture. For reflection, you only need one framebuffer texture containing the scene's interpolated vertex positions in view space. However, for refraction, positionToTexture contains the vertex positions of the scene minus the refractive surfaces since the refraction ray typically goes behind the surface. If positionFromTexture and positionToTexture were the same for refraction, the refracted ray would hit the refractive surface instead of what is behind it.
\n
Refraction Mask
\n
\n\n
\n\n
You'll need a mask to filter out the non-refractive parts of the scene. This mask will determine which fragment does and does not receive a refracted color. You could use this mask during the refracted UV calculation step or later when you actually sample the colors at the refracted UV coordinates.
\n
The mill scene uses the models' material specular as a mask. For the demo's purposes, the specular map is sufficient but you may want to use a more specialized map. Refer back to screen space reflection for how to render the specular map.
\n
Background Colors
\n
\n\n
\n\n
You'll need to render the parts of the scene behind the refractive objects. This can be done by hiding the refractive objects and then rendering the scene to a framebuffer texture.
To render the actual refractions or foreground colors, you'll need the refracted UV coordinates, refraction mask, the foreground and background vertex positions, and the background colors.
tintColor and depthMax are adjustable parameters. tintColor colorizes the background color. depthMax ranges from zero to infinity. When the distance between the foreground and background position reaches depthMax, the foreground color will be the fully tinted background color. At distance zero, the foreground will be the background color.
Calculate the depth or distance between the foreground position and the background position. At zero depth, the foreground color will be the shallow color. At depthMax, the foreground color will be the deep color. The deep color is the background color tinted with tintColor.
Recall that the blue channel, in the refracted UV texture, is set to the visibility. The visibility declines as the refracted ray points back at the camera. While the visibility should always be one, it is put here for completeness. As the visibility lessens, the fragment color will receive less and less of the foreground color.
Below is the setup used to develop and test the example code.
\n
Environment
\n
The example code was developed and tested using the following environment.
\n
\n
Linux manjaro 5.10.42-1-MANJARO
\n
OpenGL renderer string: GeForce GTX 970/PCIe/SSE2
\n
OpenGL version string: 4.6.0 NVIDIA 465.31
\n
g++ (GCC) 11.1.0
\n
Panda3D 1.10.9
\n
\n
Materials
\n
Each Blender material used to build mill-scene.egg has five textures in the following order.
\n
\n
Diffuse
\n
Normal
\n
Specular
\n
Reflection
\n
Refraction
\n
\n
By having the same maps in the same positions for all models, the shaders can be generalized, reducing the need to duplicate code.
\n
If an object uses its vertex normals, a \"flat blue\" normal map is used.
\n
\n\n
\n\n
Here is an example of a flat normal map. The only color it contains is flat blue (red = 128, green = 128, blue = 255). This color represents a unit (length one) normal pointing in the positive z-axis (0, 0, 1).
Here you see the unit normal (0, 0, 1) converted to flat blue (128, 128, 255) and flat blue converted to the unit normal. You'll learn more about this in the normal mapping technique.
\n
\n\n
\n\n
Up above is one of the specular maps used. The red and blue channel work to control the amount of specular reflection seen based on the camera angle. The green channel controls the shininess factor. You'll learn more about this in the lighting and fresnel factor sections.
\n
The reflection and refraction textures mask off the objects that are either reflective, refractive, or both. For the reflection texture, the red channel controls the amount of reflection and the green channel controls how clear or blurry the reflection is.
\n
Panda3D
\n
The example code uses Panda3D as the glue between the shaders. This has no real influence over the techniques described, meaning you'll be able to take what you learn here and apply it to your stack or game engine of choice. Panda3D does provide some conveniences. I have pointed these out so you can either find an equivalent convenience provided by your stack or replicate it yourself, if your stack doesn't provide something equivalent.
\n
Three Panda3D configurations were changed for the purposes of the demo program. You can find these in config.prc. The configurations changed were gl-coordinate-system default, textures-power-2 down, and textures-auto-power-2 1. Refer to the Panda3D configuration page in the manual for more details.
\n
Panda3D defaults to a z-up, right-handed coordinate system while OpenGL uses a y-up, right-handed system. gl-coordinate-system default keeps you from having to translate between the two inside your shaders. textures-auto-power-2 1 allows us to use texture sizes that are not a power of two if the system supports it. This comes in handy when doing SSAO and other screen/window sized related techniques since the screen/window size is usually not a power of two. textures-power-2 down downsizes our textures to a power of two if the system only supports texture sizes being a power of two.
The neighboring fragments are up, down, left, and right. After multiplying both the neighbors and the current fragment by their particular values, sum the result.
SSAO is one of those effects you never knew you needed and can't live without once you have it. It can take a scene from mediocre to wow! For fairly static scenes, you can bake ambient occlusion into a texture but for more dynamic scenes, you'll need a shader. SSAO is one of the more fairly involved shading techniques, but once you pull it off, you'll feel like a shader master.
\n
By using only a handful of textures, SSAO can approximate the ambient occlusion of a scene. This is faster than trying to compute the ambient occlusion by going through all of the scene's geometry. These handful of textures all originate in screen space giving screen space ambient occlusion its name.
\n
Inputs
\n
The SSAO shader will need the following inputs.
\n
\n
Vertex position vectors in view space.
\n
Vertex normal vectors in view space.
\n
Sample vectors in tangent space.
\n
Noise vectors in tangent space.
\n
The camera lens' projection matrix.
\n
\n
Vertex Positions
\n
\n\n
\n\n
Storing the vertex positions into a framebuffer texture is not a necessity. You can recreate them from the camera's depth buffer. This being a beginners guide, I'll avoid this optimization and keep it straight forward. Feel free to use the depth buffer, however, for your implementation.
If you do decide to use the depth buffer, here's how you can set it up using Panda3D.
\n
in vec4 vertexPosition;\n\nout vec4 fragColor;\n\nvoid main() {\n fragColor = vertexPosition;\n}
\n
Here's the simple shader used to render out the view space vertex positions into a framebuffer texture. The more involved work is setting up the framebuffer texture such that the fragment vector components it receives are not clamped to [0, 1] and that each one has a high enough precision (a high enough number of bits). For example, if a particular interpolated vertex position is <-139.444444566, 0.00000034343, 2.5>, you don't want it stored into the texture as <0.0, 0.0, 1.0>.
Here's how the example code sets up the framebuffer texture to store the vertex positions. It wants 32 bits per red, green, blue, and alpha components and disables clamping the values to [0, 1] The set_rgba_bits(32, 32, 32, 32) call sets the bits and also disables the clamping.
Here's the equivalent OpenGL call. GL_RGB32F sets the bits and also disables the clamping.
\n
\nIf the color buffer is fixed-point, the components of the source and destination\nvalues and blend factors are each clamped to [0, 1] or [−1, 1] respectively for\nan unsigned normalized or signed normalized color buffer prior to evaluating the blend\nequation.\nIf the color buffer is floating-point, no clamping occurs.\n \n \n\n
\n\n
\n\n
\n\n
Here you see the vertex positions with y being the up vector.
\n
Recall that Panda3D sets z as the up vector but OpenGL uses y as the up vector. The position shader outputs the vertex positions with z being up since Panda3D was configured with gl-coordinate-system default.
\n
Vertex Normals
\n
\n\n
\n\n
You'll need the vertex normals to correctly orient the samples you'll take in the SSAO shader. The example code generates multiple sample vectors distributed in a hemisphere but you could use a sphere and do away with the need for normals all together.
\n
in vec3 vertexNormal;\n\nout vec4 fragColor;\n\nvoid main() {\n vec3 normal = normalize(vertexNormal);\n\n fragColor = vec4(normal, 1);\n}
\n
Like the position shader, the normal shader is simple as well. Be sure to normalize the vertex normal and remember that they are in view space.
\n
\n\n
\n\n
Here you see the vertex normals with y being the up vector.
\n
Recall that Panda3D sets z as the up vector but OpenGL uses y as the up vector. The normal shader outputs the vertex positions with z being up since Panda3D was configured with gl-coordinate-system default.
\n
\n\n
\n\n
Here you see SSAO being used with the normal maps instead of the vertex normals. This adds an extra level of detail and will pair nicely with the normal mapped lighting.
To use the normal maps instead, you'll need to transform the normal mapped normals from tangent space to view space just like you did in the lighting calculations.
\n
Samples
\n
To determine the amount of ambient occlusion for any particular fragment, you'll need to sample the surrounding area. The more samples you use, the better the approximation at the cost of performance.
To get a good sweep of the sampled area, you'll need to generate some noise vectors. These noise vectors will randomly tilt the hemisphere around the current fragment.
\n
Ambient Occlusion
\n
\n\n
\n\n
SSAO works by sampling the view space around a fragment. The more samples that are below a surface, the darker the fragment color. These samples are positioned at the fragment and pointed in the general direction of the vertex normal. Each sample is used to look up a position in the position framebuffer texture. The position returned is compared to the sample. If the sample is farther away from the camera than the position, the sample counts towards the fragment being occluded.
\n
\n\n
\n\n
Here you see the space above the surface being sampled for occlusion.
Like some of the other techniques, the SSAO shader has a few control knobs you can tweak to get the exact look you're going for. The bias adds to the sample's distance from the camera. You can use the bias to combat \"acne\". The radius increases or decreases the coverage area of the sample space. The magnitude either lightens or darkens the occlusion map. The contrast either washes out or increases the starkness of the occlusion map.
Retrieve the position, normal, and random vector for later use. Recall that the example code created a set number of random vectors. The random vector is chosen based on the current fragment's screen position.
Using the random and normal vectors, assemble the tangent, binormal, and normal matrix. You'll need this matrix to transform the sample vectors from tangent space to view space.
\n
// ...\n\nfloat occlusion = NUM_SAMPLES;\n\nfor (int i = 0; i < NUM_SAMPLES; ++i) {\n// ...\n }\n\n// ...
\n
With the matrix in hand, the shader can now loop through the samples, subtracting how many are not occluded.
Using the sample's position in view space, transform it from view space to clip space to UV space.
\n
-1 * 0.5 + 0.5 = 0\n1 * 0.5 + 0.5 = 1
\n
Recall that clip space components range from negative one to one and that UV coordinates range from zero to one. To transform clip space coordinates to UV coordinates, multiply by one half and add one half.
Using the offset UV coordinates, created by projecting the 3D sample onto the 2D position texture, find the corresponding position vector. This takes you from view space to clip space to UV space back to view space. The shader takes this round trip to find out if some geometry is behind, at, or in front of this sample. If the sample is in front of or at some geometry, this sample doesn't count towards the fragment being occluded. If the sample is behind some geometry, this sample counts towards the fragment being occluded.
Now weight this sampled position by how far it is inside or outside the radius. Finally, subtract this sample from the occlusion factor since it assumes all of the samples are occluded before the loop.
Divide the occluded count by the number of samples to scale the occlusion factor from [0, NUM_SAMPLES] to [0, 1]. Zero means full occlusion and one means no occlusion. Now assign the occlusion factor to the fragment's color and you're done.
For the demo's purposes, the example code sets the alpha channel to alpha channel of the position framebuffer texture to avoid covering up the background.
\n
Blurring
\n
\n\n
\n\n
The SSAO framebuffer texture is noisy as is. You'll want to blur it to remove the noise. Refer back to the section on blurring. For the best results, use a median or Kuwahara filter to preserve the sharp edges.
The final stop for SSAO is back in the lighting calculation. Here you see the occlusion factor being looked up in the SSAO framebuffer texture and then included in the ambient light calculation.
Texturing involves mapping some color or some other kind of vector to a fragment using UV coordinates. Both U and V range from zero to one. Each vertex gets a UV coordinate and this is outputted in the vertex shader.
\n
\n\n
\n\n
The fragment shader receives the UV coordinate interpolated. Interpolated meaning the UV coordinate for the fragment is somewhere between the UV coordinates for the vertexes that make up the triangle face.
Here you see the vertex shader outputting the texture coordinate to the fragment shader. Notice how it's a two dimensional vector. One dimension for U and one for V.
When performing render to texture, the mesh is a flat rectangle with the same aspect ratio as the screen. Because of this, you can calculate the UV coordinates knowing only A) the width and height of the screen sized texture being UV mapped to the rectangle and B) the fragment's x and y coordinate. To map x to U, divide x by the width of the input texture. Similarly, to map y to V, divide y by the height of the input texture. You'll see this technique used in the example code.
\n \n\n"
},
{
"path": "sections/blinn-phong.md",
"content": "[:arrow_backward:](lighting.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](fresnel-factor.md)\n\n# 3D Game Shaders For Beginners\n\n## Blinn-Phong\n\n
\n\n
\n\nBlinn-Phong is a slight adjustment of the Phong model you saw in the [lighting](lighting.md) section.\nIt provides more plausible or realistic specular reflections.\nYou'll notice that Blinn-Phong produces elliptical or elongated specular reflections\nversus the spherical specular reflections produced by the Phong model.\nIn certain cases, Blinn-Phong can be more efficient to calculate than Phong.\n\n```c\n // ...\n\n vec3 light = normal(lightPosition.xyz - vertexPosition.xyz);\n vec3 eye = normalize(-vertexPosition.xyz);\n vec3 halfway = normalize(light + eye);\n\n // ...\n```\n\nInstead of computing the reflection vector,\ncompute the halfway or half angle vector.\nThis vector is between the view/camera/eye and light direction vector.\n\n
\n\n
\n\n```c\n // ...\n\n float specularIntensity = dot(normal, halfway);\n\n // ...\n```\n\nThe specular intensity is now the dot product of the normal and halfway vector.\nIn the Phong model, it is the dot product of the reflection and view vector.\n\n
\n\n
\n\nThe half angle vector (magenta arrow) will point in the same direction as the normal (green arrow) when the\nview vector (orange arrow) points in the same direction as the reflection vector (magenta arrow).\nIn this case, both the Blinn-Phong and Phong specular intensity will be one.\n\n
\n\n
\n\nIn other cases, the specular intensity for Blinn-Phong will be greater than zero\nwhile the specular intensity for Phong will be zero.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [base.vert](../demonstration/shaders/vertex/base.vert)\n- [base.frag](../demonstration/shaders/fragment/base.frag)\n\n## Copyright\n\n(C) 2020 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](lighting.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](fresnel-factor.md)\n"
},
{
"path": "sections/bloom.md",
"content": "[:arrow_backward:](blur.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](ssao.md)\n\n# 3D Game Shaders For Beginners\n\n## Bloom\n\n
\n\n
\n\nAdding bloom to a scene can really sell the illusion of the lighting model.\nLight emitting objects are more believable and specular highlights get an extra dose of shimmer.\n\n```c\n //...\n\n int size = 5;\n float separation = 3;\n float threshold = 0.4;\n float amount = 1;\n\n // ...\n```\n\nThese parameters control the look and feel.\n`size` determines how blurred the effect is.\n`separation` spreads out the blur.\n`threshold` controls which fragments are illuminated.\nAnd the last parameter, `amount`, controls how much bloom is outputted.\n\n```c\n // ...\n\n vec2 texSize = textureSize(colorTexture, 0).xy;\n\n float value = 0.0;\n float count = 0.0;\n\n vec4 result = vec4(0);\n vec4 color = vec4(0);\n\n for (int i = -size; i <= size; ++i) {\n for (int j = -size; j <= size; ++j) {\n // ...\n }\n }\n\n // ...\n```\n\nThe technique starts by looping through a kernel/matrix/window centered over the current fragment.\nThis is similar to the window used for [outlining](outlining.md).\nThe size of the window is `size * 2 + 1` by `size * 2 + 1`.\nSo for example, with a `size` setting of two, the window uses `(2 * 2 + 1)^2 = 25` samples per fragment.\n\n```c\n // ...\n\n color =\n texture\n ( colorTexture\n , ( gl_FragCoord.xy\n + (vec2(i, j) * separation)\n )\n / texSize\n );\n\n value = max(color.r, max(color.g, color.b));\n if (value < threshold) { color = vec4(0); }\n\n result += color;\n count += 1.0;\n\n // ...\n```\n\nFor each iteration,\nit retrieves the color from the input texture and turns the red, green, and blue values into a greyscale value.\nIf this greyscale value is less than the threshold, it discards this color by making it solid black.\nAfter evaluating the sample's greyscale value, it adds its RGB values to `result`.\n\n```c\n // ...\n\n result /= count;\n\n fragColor = mix(vec4(0), result, amount);\n\n // ...\n```\n\nAfter it's done summing up the samples, it divides the sum of the color samples by the number of samples taken.\nThe result is the average color of itself and its neighbors.\nBy doing this for every fragment, you end up with a blurred image.\nThis form of blurring is known as a [box blur](blur.md#box-blur).\n\n
\n\n
\n\nHere you see the progression of the bloom algorithm.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [bloom.frag](../demonstration/shaders/fragment/outline.frag)\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](blur.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](ssao.md)\n"
},
{
"path": "sections/blur.md",
"content": "[:arrow_backward:](fog.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](bloom.md)\n\n# 3D Game Shaders For Beginners\n\n## Blur\n\n
\n\n
\n\nThe need to blur this or that can come up quite often as you try to obtain\na particular look or perform some technique like motion blur.\nBelow are just some of ways you can blur your game's imagery.\n\n### Box Blur\n\n
\n\n
\n\nThe box blur or mean filter algorithm is a simple to implement blurring effect.\nIt's fast and gets the job done.\nIf you need more finesse, you can upgrade to a Gaussian blur.\n\n```c\n // ...\n\n vec2 texSize = textureSize(colorTexture, 0).xy;\n vec2 texCoord = gl_FragCoord.xy / texSize;\n\n int size = int(parameters.x);\n if (size <= 0) { fragColor = texture(colorTexture, texCoord); return; }\n\n // ...\n```\n\nThe `size` parameter controls how blurry the result is.\nIf the `size` is zero or less, return the fragment untouched.\n\n```c\n // ...\n\n float separation = parameters.y;\n separation = max(separation, 1);\n\n // ...\n```\n\nThe `separation` parameter spreads out the blur without having to sample additional fragments.\n`separation` ranges from one to infinity.\n\n
\n\n
\n\n```c\n // ...\n\n for (int i = -size; i <= size; ++i) {\n for (int j = -size; j <= size; ++j) {\n // ...\n }\n }\n\n // ...\n```\n\nLike the [outlining](outlining.md) technique,\nthe box blur technique uses a kernel/matrix/window centered around the current fragment.\nThe size of the window is `size * 2 + 1` by `size * 2 + 1`.\nSo for example, with a `size` setting of two, the window uses `(2 * 2 + 1)^2 = 25` samples per fragment.\n\n```c\n // ...\n\n fragColor +=\n texture\n ( colorTexture\n , ( gl_FragCoord.xy\n + (vec2(i, j) * separation)\n )\n / texSize\n );\n\n // ...\n```\n\nTo compute the mean or average of the samples in the window,\nstart by loop through the window, adding up each color vector.\n\n\n```c\n // ...\n\n fragColor /= pow(size * 2 + 1, 2);\n\n // ...\n```\n\nTo finish computing the mean, divide the sum of the colors sampled by the number of samples taken.\nThe final fragment color is the mean or average of the fragments sampled inside the window.\n\n### Median Filter\n\n
\n\n
\n\nThe box blur uses the mean color of the samples taken.\nThe median filter uses the median color of the samples taken.\nBy using the median instead of the mean,\nthe edges in the image are preserved—meaning the edges stay nice and crisp.\nFor example, look at the windows in the box blurred image versus the median filtered image.\n\nUnfortunately, finding the median can be slower than finding the mean.\nYou could sort the values and choose the middle one but that would take at least quasilinear time.\nThere is a technique to find the median in linear time but it can be quite awkward inside a shader.\nThe numerical approach below approximates the median in linear time.\nHow well it approximates the median can be controlled.\n\n
\n\n
\n\nAt lower quality approximations,\nyou end up with a nice [painterly](https://en.wikipedia.org/wiki/Painterliness) look.\n\n```c\n// ...\n\n#define MAX_SIZE 4\n#define MAX_KERNEL_SIZE ((MAX_SIZE * 2 + 1) * (MAX_SIZE * 2 + 1))\n#define MAX_BINS_SIZE 100\n\n// ...\n```\n\nThese are the hard limits for the `size` parameter, window size, and `bins` array.\n\n\n```c\n // ...\n\n vec2 texSize = textureSize(colorTexture, 0).xy;\n vec2 texCoord = gl_FragCoord.xy / texSize;\n\n int size = int(parameters.x);\n if (size <= 0) { fragColor = texture(colorTexture, texCoord); return; }\n if (size > MAX_SIZE) { size = MAX_SIZE; }\n int kernelSize = int(pow(size * 2 + 1, 2));\n\n // ...\n```\n\nThe `size` parameter controls how blurry or smeared the effect is.\nIf the size is at or below zero, return the current fragment untouched.\nFrom the `size` parameter, calculate the total size of the kernel or window.\nThis is how many samples you'll be taking per fragment.\n\n```c\n // ...\n\n int binsSize = int(parameters.y);\n binsSize = clamp(binsSize, 1, MAX_BINS_SIZE);\n\n // ...\n```\n\nSet up the `binsSize`, making sure to limit it by the `MAX_BINS_SIZE`.\n\n```c\n // ...\n\n int i = 0;\n int j = 0;\n int count = 0;\n int binIndex = 0;\n\n // ...\n```\n\n`i` and `j` are used to sample the given texture around the current fragment.\n`i` is also used as a general for loop count.\n`count` is used in the initialization of the `colors` array which you'll see later.\n`binIndex` is used to approximate the median color.\n\n```c\n // ...\n\n vec4 colors[MAX_KERNEL_SIZE];\n float bins[MAX_BINS_SIZE];\n int binIndexes[colors.length()];\n\n // ...\n```\n\nThe `colors` array holds the sampled colors taken from the input texture.\n`bins` is used to approximate the median of the sampled colors.\nEach bin holds a count of how many colors fall into its range when converting each color into a greyscale value (between zero and one).\nAs `binsSize` approaches 100, the algorithm finds the true median almost always.\n`binIndexes` stores the `bins` index or which bin each sample falls into.\n\n```c\n // ...\n\n float total = 0;\n float limit = floor(float(kernelSize) / 2) + 1;\n\n // ...\n```\n\n`total` keeps track of how many colors you've come across as you loop through `bins`.\nWhen `total` reaches `limit`, you return whatever `bins` index you're at.\nThe `limit` is the median index.\nFor example, if the window size is 81, `limit` is 41 which is directly in the middle (40 samples below and 40 samples above).\n\n```c\n // ...\n\n float value = 0;\n vec3 valueRatios = vec3(0.3, 0.59, 0.11);\n\n // ...\n```\n\nThese are used to covert and hold each color sample's greyscale value.\nInstead of dividing red, green, and blue by one third,\nit uses 30% of red, 59% of green, and 11% of blue for a total of 100%.\n\n```c\n // ...\n\n for (i = -size; i <= size; ++i) {\n for (j = -size; j <= size; ++j) {\n colors[count] =\n texture\n ( colorTexture\n , ( gl_FragCoord.xy\n + vec2(i, j)\n )\n / texSize\n );\n count += 1;\n }\n }\n\n // ...\n```\n\nLoop through the window and collect the color samples into `colors`.\n\n```c\n // ...\n\n for (i = 0; i < binsSize; ++i) {\n bins[i] = 0;\n }\n\n // ...\n```\n\nInitialize the `bins` array with zeros.\n\n```c\n // ...\n\n for (i = 0; i < kernelSize; ++i) {\n value = dot(colors[i].rgb, valueRatios);\n binIndex = int(floor(value * binsSize));\n binIndex = clamp(binIndex, 0, binsSize - 1);\n bins[binIndex] += 1;\n binIndexes[i] = binIndex;\n }\n\n // ...\n```\n\nLoop through the colors and convert each one to a greyscale value.\n`dot(colors[i].rgb, valueRatios)` is the weighted sum `colors.r * 0.3 + colors.g * 0.59 + colors.b * 0.11`.\n\nEach value will fall into some bin.\nEach bin covers some range of values.\nFor example, if the number of bins is 10, the first bin covers everything from zero up to but not including 0.1.\nIncrement the number of colors that fall into this bin and remember the color sample's bin index so you can look it up later.\n\n```c\n // ...\n\n binIndex = 0;\n\n for (i = 0; i < binsSize; ++i) {\n total += bins[i];\n if (total >= limit) {\n binIndex = i;\n break;\n }\n }\n\n // ...\n```\n\nLoop through the bins, tallying up the number of colors seen so far.\nWhen you reach the median index, exit the loop and remember the last `bins` index reached.\n\n```c\n // ...\n\n fragColor = colors[0];\n\n for (i = 0; i < kernelSize; ++i) {\n if (binIndexes[i] == binIndex) {\n fragColor = colors[i];\n break;\n }\n }\n\n // ...\n```\n\nNow loop through the `binIndexes` and find the first color with the last `bins` indexed reached.\nIts greyscale value is the approximated median which in many cases will be the true median value.\nSet this color as the fragColor and exit the loop and shader.\n\n### Kuwahara Filter\n\n
\n\n
\n\nLike the median filter, the kuwahara filter preserves the major edges found in the image.\nYou'll notice that it has a more block like or chunky pattern to it.\nIn practice,\nthe Kuwahara filter runs faster than the median filter, allowing for larger `size` values without a noticeable slowdown.\n\n```c\n// ...\n\n#define MAX_SIZE 5\n#define MAX_KERNEL_SIZE ((MAX_SIZE * 2 + 1) * (MAX_SIZE * 2 + 1))\n\n// ...\n```\n\nSet a hard limit for the `size` parameter and the number of samples taken.\n\n```c\n// ...\n\nint i = 0;\nint j = 0;\nint count = 0;\n\n// ...\n```\n\nThese are used to sample the input texture and set up the `values` array.\n\n```c\n// ...\n\nvec3 valueRatios = vec3(0.3, 0.59, 0.11);\n\n// ...\n```\n\nLike the median filter, you'll be converting the color samples into greyscale values.\n\n```c\n// ...\n\nfloat values[MAX_KERNEL_SIZE];\n\n// ...\n```\n\nInitialize the `values` array.\nThis will hold the greyscale values for the color samples.\n\n```c\n// ...\n\nvec4 color = vec4(0);\nvec4 meanTemp = vec4(0);\nvec4 mean = vec4(0);\nfloat valueMean = 0;\nfloat variance = 0;\nfloat minVariance = -1;\n\n// ...\n```\n\nThe Kuwahara filter works by computing the variance of four subwindows and then using the mean of the subwindow with the smallest variance.\n\n```c\n// ...\n\nvoid findMean(int i0, int i1, int j0, int j1) {\n\n// ...\n```\n\n`findMean` is a function defined outside of `main`.\nEach run of `findMean` will remember the mean of the given subwindow that has the lowest variance seen so far.\n\n```c\n // ...\n\n meanTemp = vec4(0);\n count = 0;\n\n // ...\n```\n\nMake sure to reset `count` and `meanTemp` before computing the mean of the given subwindow.\n\n```c\n // ...\n\n for (i = i0; i <= i1; ++i) {\n for (j = j0; j <= j1; ++j) {\n color =\n texture\n ( colorTexture\n , (gl_FragCoord.xy + vec2(i, j))\n / texSize\n );\n\n meanTemp += color;\n\n values[count] = dot(color.rgb, valueRatios);\n\n count += 1;\n }\n }\n\n // ...\n```\n\nSimilar to the box blur, loop through the given subwindow and add up each color.\nAt the same time, make sure to store the greyscale value for this sample in `values`.\n\n```c\n // ...\n\n meanTemp.rgb /= count;\n valueMean = dot(meanTemp.rgb, valueRatios);\n\n // ...\n```\n\nTo compute the mean, divide the samples sum by the number of samples taken.\nCalculate the greyscale value for the mean.\n\n```c\n // ...\n\n for (i = 0; i < count; ++i) {\n variance += pow(values[i] - valueMean, 2);\n }\n\n variance /= count;\n\n // ...\n```\n\nNow calculate the variance for this given subwindow.\nThe variance is the average squared difference between each sample's greyscale value the mean greyscale value.\n\n```c\n // ...\n\n if (variance < minVariance || minVariance <= -1) {\n mean = meanTemp;\n minVariance = variance;\n }\n}\n\n// ...\n```\n\nIf the variance is smaller than what you've seen before or this is the first variance you've seen,\nset the mean of this subwindow as the final mean and update the minimum variance seen so far.\n\n```c\n// ...\n\nvoid main() {\n int size = int(parameters.x);\n if (size <= 0) { fragColor = texture(colorTexture, texCoord); return; }\n\n // ...\n```\n\nBack in `main`, set the `size` parameter.\nIf the size is at or below zero, return the fragment unchanged.\n\n
\n\n
\n\n```c\n // ...\n\n // Lower Left\n\n findMean(-size, 0, -size, 0);\n\n // Upper Right\n\n findMean(0, size, 0, size);\n\n // Upper Left\n\n findMean(-size, 0, 0, size);\n\n // Lower Right\n\n findMean(0, size, -size, 0);\n\n // ...\n```\n\nAs stated above,\nthe Kuwahara filter works by computing the variance of four subwindows\nand then using the mean of the subwindow with the lowest variance as the final fragment color.\nNote that the four subwindows overlap each other.\n\n```c\n // ...\n\n mean.a = 1;\n fragColor = mean;\n\n // ...\n```\n\nAfter computing the variance and mean for each subwindow,\nset the fragment color to the mean of the subwindow with the lowest variance.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [position.frag](../demonstration/shaders/fragment/position.frag)\n- [box-blur.frag](../demonstration/shaders/fragment/box-blur.frag)\n- [median-filter.frag](../demonstration/shaders/fragment/median-filter.frag)\n- [kuwahara-filter.frag](../demonstration/shaders/fragment/kuwahara-filter.frag)\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](fog.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](bloom.md)\n"
},
{
"path": "sections/building-the-demo.md",
"content": "[:arrow_backward:](setup.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](running-the-demo.md)\n\n# 3D Game Shaders For Beginners\n\n## Building The Demo\n\n
\n\n
\n\nBefore you can try out the demo program, you'll have to build the example code first.\n\n### Dependencies\n\nBefore you can compile the example code, you'll need to install\n[Panda3D](https://www.panda3d.org/)\nfor your platform.\nPanda3D is available for Linux, Mac, and Windows.\n\n### Linux\n\nStart by [installing](https://www.panda3d.org/manual/?title=Installing_Panda3D_in_Linux) the\n[Panda3D SDK](https://www.panda3d.org/download/sdk-1-10-9/) for your distribution.\n\nMake sure to locate where the Panda3D headers and libraries are.\nThe headers and libraries are most likely in `/usr/include/panda3d/` and `/usr/lib/panda3d/` respectively.\n\nNext clone this repository and change directory into it.\n\n```bash\ngit clone https://github.com/lettier/3d-game-shaders-for-beginners.git\ncd 3d-game-shaders-for-beginners/demonstration\n```\n\nNow compile the source code into an object file.\n\n```bash\ng++ \\\n -c src/main.cxx \\\n -o 3d-game-shaders-for-beginners.o \\\n -std=gnu++11 \\\n -O2 \\\n -I/path/to/python/include/ \\\n -I/path/to/panda3d/include/\n```\n\nWith the object file created, create the executable by linking the object file to its dependencies.\n\n```bash\ng++ \\\n 3d-game-shaders-for-beginners.o \\\n -o 3d-game-shaders-for-beginners \\\n -L/path/to/panda3d/lib \\\n -lp3framework \\\n -lpanda \\\n -lpandafx \\\n -lpandaexpress \\\n -lpandaphysics \\\n -lp3dtoolconfig \\\n -lp3dtool \\\n -lpthread\n```\n\nFor more help, see the [Panda3D manual](https://www.panda3d.org/manual/?title=How_to_compile_a_C++_Panda3D_program_on_Linux).\n\n### Mac\n\nStart by installing the [Panda3D SDK](https://www.panda3d.org/download/sdk-1-10-9/) for Mac.\n\nMake sure to locate where the Panda3D headers and libraries are.\n\nNext clone this repository and change directory into it.\n\n```bash\ngit clone https://github.com/lettier/3d-game-shaders-for-beginners.git\ncd 3d-game-shaders-for-beginners\n```\n\nNow compile the source code into an object file.\nYou'll have to find where the Python 2.7 and Panda3D include directories are.\n\n```bash\nclang++ \\\n -c main.cxx \\\n -o 3d-game-shaders-for-beginners.o \\\n -std=gnu++11 \\\n -g \\\n -O2 \\\n -I/path/to/python/include/ \\\n -I/path/to/panda3d/include/\n```\n\nWith the object file created, create the executable by linking the object file to its dependencies.\nYou'll need to track down where the Panda3D libraries are located.\n\n```bash\nclang++ \\\n 3d-game-shaders-for-beginners.o \\\n -o 3d-game-shaders-for-beginners \\\n -L/path/to/panda3d/lib \\\n -lp3framework \\\n -lpanda \\\n -lpandafx \\\n -lpandaexpress \\\n -lpandaphysics \\\n -lp3dtoolconfig \\\n -lp3dtool \\\n -lpthread\n```\n\nFor more help, see the [Panda3D manual](https://www.panda3d.org/manual/?title=How_to_compile_a_C++_Panda3D_program_on_macOS).\n\n### Windows\n\nStart by [installing](https://www.panda3d.org/manual/?title=Installing_Panda3D_in_Windows) the\n[Panda3D SDK](https://www.panda3d.org/download/sdk-1-10-9/) for Windows.\n\nMake sure to locate where the Panda3D headers and libraries are.\n\nNext clone this repository and change directory into it.\n\n```bash\ngit clone https://github.com/lettier/3d-game-shaders-for-beginners.git\ncd 3d-game-shaders-for-beginners\n```\n\nFor more help, see the [Panda3D manual](https://www.panda3d.org/manual/?title=Running_your_Program&language=cxx).\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](setup.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](running-the-demo.md)\n"
},
{
"path": "sections/cel-shading.md",
"content": "[:arrow_backward:](rim-lighting.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](normal-mapping.md)\n\n# 3D Game Shaders For Beginners\n\n## Cel Shading\n\n
\n\n
\n\nCel shading is a technique to make 3D objects look 2D or flat.\nIn 2D,\nyou can make an object look 3D by applying a smooth gradient.\nHowever, with cel shading, you're breaking up the gradients into abrupt transitions.\nTypically there is only one transition where the shading goes from fully lit to fully shadowed.\nWhen combined with [outlining](outlining.md), cel shading can really sell the 2D cartoon look.\n\n## Diffuse\n\n```c\n // ...\n\n float diffuseIntensity = max(dot(normal, unitLightDirection), 0.0);\n diffuseIntensity = step(0.1, diffuseIntensity);\n\n // ...\n```\n\nRevisiting the [lighting](lighting.md#diffuse) model,\nmodify the `diffuseIntensity` such that it is either zero or one.\n\n
\n\n
\n\nThe `step` function returns zero if the input is less than the edge and one otherwise.\n\n
\n\n
\n\n```c\n // ...\n\n if (diffuseIntensity >= 0.8) { diffuseIntensity = 1.0; }\n else if (diffuseIntensity >= 0.6) { diffuseIntensity = 0.6; }\n else if (diffuseIntensity >= 0.3) { diffuseIntensity = 0.3; }\n else { diffuseIntensity = 0.0; }\n\n // ...\n```\n\nIf you would like to have a few steps or transitions,\nyou can perform something like the above.\n\n
\n\n
\n\n```c\n // ...\n\n diffuseIntensity = texture(steps, vec2(diffuseIntensity, 0.0)).r;\n\n // ...\n```\n\nAnother approach is to put your step values into a texture with the transitions going from darker to lighter.\nUsing the `diffuseIntensity` as a U coordinate, it will automatically transform itself.\n\n## Specular\n\n```c\n\n // ...\n\n float specularIntensity = clamp(dot(normal, halfwayDirection), 0.0, 1.0);\n specularIntensity = step(0.98, specularIntensity);\n\n // ...\n```\n\nUsing the `step` function again, set the `specularIntensity` to be either zero or one.\nYou can also use one of the other approaches described up above for the specular highlight as well.\nAfter you've altered the `specularIntensity`, the rest of the lighting calculations are the same.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [base.vert](../demonstration/shaders/vertex/base.vert)\n- [base.frag](../demonstration/shaders/fragment/base.frag)\n\n## Copyright\n\n(C) 2020 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](rim-lighting.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](normal-mapping.md)\n"
},
{
"path": "sections/chromatic-aberration.md",
"content": "[:arrow_backward:](motion-blur.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](screen-space-reflection.md)\n\n# 3D Game Shaders For Beginners\n\n## Chromatic Aberration\n\n
\n\n
\n\nChromatic aberration is a screen space technique that simulates lens distortion.\nUse it to give your scene a cinematic, lo-fi analog feel or to emphasize a chaotic event.\n\n### Texture\n\n```c\nuniform sampler2D colorTexture;\n\n// ...\n```\n\nThe input texture needed is the scene's colors captured into a framebuffer texture.\n\n### Parameters\n\n
\n\n
\n\n```c\n // ...\n\n float redOffset = 0.009;\n float greenOffset = 0.006;\n float blueOffset = -0.006;\n\n // ...\n```\n\nThe adjustable parameters for this technique are the red, green, and blue offsets.\nFeel free to play around with these to get the particular color fringe you're looking for.\nThese particular offsets produce a yellowish orange and blue fringe.\n\n### Direction\n\n```c\n// ...\n\nuniform vec2 mouseFocusPoint;\n\n // ...\n\n vec2 texSize = textureSize(colorTexture, 0).xy;\n vec2 texCoord = gl_FragCoord.xy / texSize;\n\n vec2 direction = texCoord - mouseFocusPoint;\n\n // ...\n```\n\nThe offsets can occur horizontally, vertically, or radially.\nOne approach is to radiate out from the [depth of field](depth-of-field.md) focal point.\nAs the scene gets more out of focus, the chromatic aberration increases.\n\n### Samples\n\n```c\n// ...\n\nout vec4 fragColor;\n\n // ...\n\n fragColor.r = texture(colorTexture, texCoord + (direction * vec2(redOffset ))).r;\n fragColor.g = texture(colorTexture, texCoord + (direction * vec2(greenOffset))).g;\n fragColor.ba = texture(colorTexture, texCoord + (direction * vec2(blueOffset ))).ba;\n}\n```\n\nWith the direction and offsets,\nmake three samples of the scene's colors—one for the red, green, and blue channels.\nThese will be the final fragment color.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [chromatic-aberration.frag](../demonstration/shaders/fragment/chromatic-aberration.frag)\n\n## Copyright\n\n(C) 2021 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](motion-blur.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](screen-space-reflection.md)\n"
},
{
"path": "sections/deferred-rendering.md",
"content": "[:arrow_backward:](normal-mapping.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](fog.md)\n\n# 3D Game Shaders For Beginners\n\n## Deferred Rendering\n\n
\n\n
\n\nDeferred rendering (deferred shading) is a screen space lighting technique.\nInstead of calculating the lighting for a scene while you traverse its\ngeometry—you defer or wait to perform the lighting calculations until\nafter the scene's geometry fragments have been culled or discarded.\nThis can give you a performance boost depending on the complexity of your scene.\n\n### Phases\n\n
\n\n
\n\nDeferred rendering is performed in two phases.\nThe first phase involves going through the scene's geometry and rendering its\npositions or depths,\nnormals,\nand materials\ninto a framebuffer known as the geometry buffer or G-buffer.\nWith the exception of some transformations,\nthis is mostly a read-only phase so its performance cost is minimal.\nAfter this phase, you're only dealing with 2D textures in the shape of the screen.\n\n
\n\n
\n\nThe second and last phase is where you perform your lighting calculations using the output of the first phase.\nThis is when you calculate the ambient, diffuse, and specular colors.\nShadow and normal mapping are performed in this phase as well.\n\n### Advantages\n\nThe reason for using deferred rendering is to reduce the number of lighting calculations made.\nWith forward rendering, the number of lighting calculations scales with the number of fragments and lights.\nHowever, with deferred shading, the number of lighting calculations scales with the number of pixels and lights.\nRecall that for a single pixel, there can be multiple fragments produced.\nAs you add geometry,\nthe number of lighting calculations per pixel increases when using forward but not when using deferred.\n\nFor simple scenes,\ndeferred rendering doesn't provide much of a performance boost and may even hurt performance.\nHowever, for complex scenes with lots of lighting, it becomes the better option.\nDeferred becomes faster than forward because you're only calculating the lighting per light, per pixel.\nIn forward rendering,\nyou're calculating the lighting per light per fragment which can be multiple times per pixel.\n\n### Disadvantages\n\nDeferred rendering allows you render complex scenes using many lights but it does come with its own set of tradeoffs.\nTransparency becomes an issue because the geometry data you could see through a semitransparent object is discarded in the first phase.\nOther tradesoffs include increased memory consumption due to the G-buffer and the extra workarounds needed to deal with aliasing.\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](normal-mapping.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](fog.md)\n"
},
{
"path": "sections/depth-of-field.md",
"content": "[:arrow_backward:](outlining.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](posterization.md)\n\n# 3D Game Shaders For Beginners\n\n## Depth Of Field\n\n
\n\n
\n\nLike [SSAO](ssao.md), depth of field is an effect you can't live without once you've used it.\nArtistically, you can use it to draw your viewer's eye to a certain subject.\nBut in general, depth of field adds a lot of realism for a little bit of effort.\n\n### In Focus\n\nThe first step is to render your scene completely in focus.\nBe sure to render this into a framebuffer texture.\nThis will be one of the inputs to the depth of field shader.\n\n### Out Of Focus\n\nThe second step is to blur the scene as if it was completely out of focus.\nLike bloom and SSAO, you can use a [box blur](blur.md#box-blur).\nBe sure to render this out-of-focus-scene into a framebuffer texture.\nThis will be one of the inputs to the depth of field shader.\n\n#### Bokeh\n\n
\n\n
\n\nFor a great bokeh effect, dilate the out of focus texture and use that as the out of focus input.\nSee [dilation](dilation.md) for more details.\n\n### Mixing\n\n```c\n // ...\n\n float minDistance = 1.0;\n float maxDistance = 3.0;\n\n // ...\n```\n\nFeel free to tweak these two parameters.\nAll positions at or below `minDistance` will be completely in focus.\nAll positions at or beyond `maxDistance` will be completely out of focus.\n\n```c\n // ...\n\n vec4 focusColor = texture(focusTexture, texCoord);\n vec4 outOfFocusColor = texture(outOfFocusTexture, texCoord);\n\n // ...\n```\n\nYou'll need the in focus and out of focus colors.\n\n```c\n // ...\n\n vec4 position = texture(positionTexture, texCoord);\n\n // ...\n```\n\nYou'll also need the vertex position in view space.\nYou can reuse the position framebuffer texture you used for [SSAO](ssao.md#vertex-positions).\n\n```c\n // ...\n\n vec4 focusPoint = texture(positionTexture, mouseFocusPoint);\n\n // ...\n```\n\nThe focus point is a position somewhere in your scene.\nAll of the points in your scene are measured from the focus point.\n\nChoosing the focus point is up to you.\nThe demo uses the scene position directly under the mouse when clicking the middle mouse button.\nHowever, it could be a constant distance from the camera or a static position.\n\n
\n\n
\n\n```c\n // ...\n\n float blur =\n smoothstep\n ( minDistance\n , maxDistance\n , abs(position.y - focusPoint.y)\n );\n\n // ...\n```\n\n`smoothstep` returns values from zero to one.\nThe `minDistance` is the left-most edge.\nAny position less than the minimum distance, from the focus point, will be in focus or have a blur of zero.\nThe `maxDistance` is the right-most edge.\nAny position greater than the maximum distance, from the focus point, will be out of focus or have a blur or one.\nFor distances between the edges,\nblur will be between zero and one.\nThese values are interpolated along a s-shaped curve.\n\n```c\n\n // ...\n\n fragColor = mix(focusColor, outOfFocusColor, blur);\n\n // ...\n```\n\nThe `fragColor` is a mixture of the in focus and out of focus color.\nThe closer `blur` is to one, the more it will use the `outOfFocusColor`.\nZero `blur` means this fragment is entirely in focus.\nAt `blur >= 1`, this fragment is completely out of focus.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [box-blur.frag](../demonstration/shaders/fragment/box-blur.frag)\n- [depth-of-field.frag](../demonstration/shaders/fragment/depth-of-field.frag)\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](outlining.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](posterization.md)\n"
},
{
"path": "sections/dilation.md",
"content": "[:arrow_backward:](sharpen.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](film-grain.md)\n\n# 3D Game Shaders For Beginners\n\n## Dilation\n\n
\n\n
\n\nDilation dilates or enlarges the brighter areas of an image while at the same time,\ncontracts or shrinks the darker areas of an image.\nThis tends to create a pillowy look.\nYou can use dilation for a glow/bloom effect or to add bokeh to your [depth of field](depth-of-field.md).\n\n```c\n // ...\n\n int size = int(parameters.x);\n float separation = parameters.y;\n float minThreshold = 0.1;\n float maxThreshold = 0.3;\n\n // ...\n```\n\nThe `size` and `separation` parameters control how dilated the image becomes.\nA larger `size` will increase the dilation at the cost of performance.\nA larger `separation` will increase the dilation at the cost of quality.\nThe `minThreshold` and `maxThreshold` parameters control which parts of the image become dilated.\n\n```c\n // ...\n\n vec2 texSize = textureSize(colorTexture, 0).xy;\n vec2 fragCoord = gl_FragCoord.xy;\n\n fragColor = texture(colorTexture, fragCoord / texSize);\n\n // ...\n```\n\nSample the color at the current fragment's position.\n\n```c\n // ...\n\n float mx = 0.0;\n vec4 cmx = fragColor;\n\n for (int i = -size; i <= size; ++i) {\n for (int j = -size; j <= size; ++j) {\n // ...\n }\n }\n\n // ...\n```\n\nLoop through a `size` by `size` window, centered at the current fragment position.\nAs you loop, find the brightest color based on the surrounding greyscale values.\n\n
\n\n
\n\n```c\n // ...\n\n // For a rectangular shape.\n //if (false);\n\n // For a diamond shape;\n //if (!(abs(i) <= size - abs(j))) { continue; }\n\n // For a circular shape.\n if (!(distance(vec2(i, j), vec2(0, 0)) <= size)) { continue; }\n\n // ...\n```\n\nThe window shape will determine the shape of the dilated parts of the image.\nFor a rectangular shape, you can use every fragment covered by the window.\nFor any other shape, skip the fragments that fall outside the desired shape.\n\n```c\n // ...\n\n vec4 c =\n texture\n ( colorTexture\n , ( gl_FragCoord.xy\n + (vec2(i, j) * separation)\n )\n / texSize\n );\n\n // ...\n```\n\nSample a fragment color from the surrounding window.\n\n```c\n // ...\n\n float mxt = dot(c.rgb, vec3(0.21, 0.72, 0.07));\n\n // ...\n```\n\nConvert the sampled color to a greyscale value.\n\n```c\n // ...\n\n if (mxt > mx) {\n mx = mxt;\n cmx = c;\n }\n\n // ...\n```\n\nIf the sampled greyscale value is larger than the current maximum greyscale value,\nupdate the maximum greyscale value and its corresponding color.\n\n```c\n // ...\n\n fragColor.rgb =\n mix\n ( fragColor.rgb\n , cmx.rgb\n , smoothstep(minThreshold, maxThreshold, mx)\n );\n\n // ...\n```\n\nThe new fragment color is a mixture between the existing fragment color and\nthe brightest color found.\nIf the maximum greyscale value found is less than `minThreshold`,\nthe fragment color is unchanged.\nIf the maximum greyscale value is greater than `maxThreshold`,\nthe fragment color is replaced with the brightest color found.\nFor any other case,\nthe fragment color is a mix between the current fragment color and the brightest color.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [dilation.frag](../demonstration/shaders/fragment/dilation.frag)\n\n## Copyright\n\n(C) 2020 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](sharpen.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](film-grain.md)\n"
},
{
"path": "sections/film-grain.md",
"content": "[:arrow_backward:](dilation.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](lookup-table.md)\n\n# 3D Game Shaders For Beginners\n\n## Film Grain\n\n
\n\n
\n\nFilm grain (when applied in subtle doses, unlike here)\ncan add a bit of realism you don't notice until it's removed.\nTypically, it's the imperfections that make a digitally generated image more believable.\nIn terms of the shader graph, film grain is usually the last effect applied before the game is put on screen.\n\n### Amount\n\n```c\n // ...\n\n float amount = 0.1;\n\n // ...\n```\n\nThe `amount` controls how noticeable the film grain is.\nCrank it up for a snowy picture.\n\n### Random Intensity\n\n```c\n// ...\n\nuniform float osg_FrameTime;\n\n //...\n\n float toRadians = 3.14 / 180;\n\n //...\n\n float randomIntensity =\n fract\n ( 10000\n * sin\n (\n ( gl_FragCoord.x\n + gl_FragCoord.y\n * osg_FrameTime\n )\n * toRadians\n )\n );\n\n // ...\n```\n\nThis snippet calculates the random intensity needed to adjust the amount.\n\n```c\nTime Since F1 = 00 01 02 03 04 05 06 07 08 09 10\nFrame Number = F1 F3 F4 F5 F6\nosg_FrameTime = 00 02 04 07 08\n```\n\n`osg_FrameTime` is\n[provided](https://github.com/panda3d/panda3d/blob/daa57733cb9b4ccdb23e28153585e8e20b5ccdb5/panda/src/display/graphicsStateGuardian.cxx#L930)\nby Panda3D.\nThe frame time is a timestamp of how many seconds have passed since the first frame.\nThe example code uses this to animate the film grain as `osg_FrameTime` will always be different each frame.\n\n```c\n\n // ...\n\n ( gl_FragCoord.x\n + gl_FragCoord.y\n * 8009 // Large number here.\n\n // ...\n```\n\nFor static film grain, replace `osg_FrameTime` with a large number.\nYou may have to try different numbers to avoid seeing any patterns.\n\n
\n\n
\n\n```c\n // ...\n\n * sin\n (\n ( gl_FragCoord.x\n + gl_FragCoord.y\n * someNumber\n\n // ...\n```\n\nBoth the x and y coordinate are used to create points or specs of film grain.\nIf only x was used, there would only be vertical lines.\nSimilarly, if only y was used, there would be only horizontal lines.\n\nThe reason the snippet multiplies one coordinate by some number is to break up the diagonal symmetry.\n\n
\n\n
\n\nYou can of course remove the coordinate multiplier for a somewhat decent looking rain effect.\nTo animate the rain effect, multiply the output of `sin` by `osg_FrameTime`.\n\n```c\n // ...\n\n ( gl_FragCoord.x\n + gl_FragCoord.y\n\n // ...\n```\n\nPlay around with the x and y coordinate to try and get the rain to change directions.\nKeep only the x coordinate for a straight downpour.\n\n```c\ninput = (gl_FragCoord.x + gl_FragCoord.y * osg_FrameTime) * toRadians\n frame(10000 * sin(input)) =\n fract(10000 * sin(6.977777777777778)) =\n fract(10000 * 0.6400723818964882) =\n```\n\n`sin` is used as a hashing function.\nThe fragment's coordinates are hashed to some output of `sin`.\nThis has the nice property that no matter the input (big or small), the output range is negative one to one.\n\n```c\nfract(10000 * sin(6.977777777777778)) =\n fract(10000 * 0.6400723818964882) =\n fract(6400.723818964882) =\n 0.723818964882\n```\n\n`sin` is also used as a pseudo random number generator when combined with `fract`.\n\n```python\n>>> [floor(fract(4 * sin(x * toRadians)) * 10) for x in range(0, 10)]\n[0, 0, 1, 2, 2, 3, 4, 4, 5, 6]\n\n>>> [floor(fract(10000 * sin(x * toRadians)) * 10) for x in range(0, 10)]\n[0, 4, 8, 0, 2, 1, 7, 0, 0, 5]\n```\n\nTake a look at the first sequence of numbers and then the second.\nEach sequence is deterministic but the second sequence has less of a pattern than the first.\nSo while the output of `fract(10000 * sin(...))` is deterministic, it doesn't have much of a discernible pattern.\n\n
\n\n
\n\nHere you see the `sin` multiplier going from 1, to 10, to 100, and then to 1000.\n\nAs you increase the `sin` output multiplier, you get less and less of a pattern.\nThis is the reason the snippet multiplies `sin` by 10,000.\n\n### Fragment Color\n\n```c\n // ...\n\n vec2 texSize = textureSize(colorTexture, 0).xy;\n vec2 texCoord = gl_FragCoord.xy / texSize;\n\n vec4 color = texture(colorTexture, texCoord);\n\n // ...\n```\n\nConvert the fragment's coordinates to UV coordinates.\nUsing these UV coordinates, look up the texture color for this fragment.\n\n```c\n // ...\n\n amount *= randomIntensity;\n\n color.rgb += amount;\n\n // ...\n```\n\nAdjust the amount by the random intensity and add this to the color.\n\n```c\n // ...\n\n fragColor = color;\n\n // ...\n```\n\nSet the fragment color and you're done.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [film-grain.frag](../demonstration/shaders/fragment/film-grain.frag)\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](dilation.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](lookup-table.md)\n"
},
{
"path": "sections/flow-mapping.md",
"content": "[:arrow_backward:](foam.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](outlining.md)\n\n# 3D Game Shaders For Beginners\n\n## Flow Mapping\n\n
\n\n
\n\nFlow mapping is useful when you need to animate some fluid material.\nMuch like diffuse maps map UV coordinates to diffuse colors and normal maps map UV coordinates to normals,\nflow maps map UV coordinates to 2D translations or flows.\n\n
\n\n
\n\nHere you see a flow map that maps UV coordinates to translations in the positive y-axis direction.\nFlow maps use the red and green channels to store translations in the x and y direction.\nThe red channel is for the x-axis and the green channel is the y-axis.\nBoth range from zero to one which translates to flows that range from `(-1, -1)` to `(1, 1)`.\nThis flow map is all one color consisting of 0.5 red and 0.6 green.\n\n```c\n[r, g, b] =\n [r * 2 - 1, g * 2 - 1, b * 2 - 1] =\n [ x, y, z]\n```\n\nRecall how the colors in a normal map are converted to actual normals.\nThere is a similar process for flow maps.\n\n```c\n// ...\n\nuniform sampler2D flowTexture;\n\n vec2 flow = texture(flowTexture, uv).xy;\n flow = (flow - 0.5) * 2;\n\n // ...\n```\n\nTo convert a flow map color to a flow,\nyou minus 0.5 from the channel (red and green) and multiply by two.\n\n```c\n(r, g) =\n ( (r - 0.5) * 2\n , (g - 0.5) * 2\n ) =\n ( (0.5 - 0.5) * 2\n , (0.6 - 0.5) * 2\n ) =\n (x, y) =\n (0, 0.2)\n```\n\nThe flow map above maps each UV coordinate to the flow `(0, 0.2)`.\nThis indicates zero movement in the x direction and a movement of 0.2 in the y direction.\n\nThe flows can be used to translate all sorts of things but they're typically used to\noffset the UV coordinates of a another texture.\n\n
\n\n
\n\n```c\n // ...\n\n vec2 flow = texture(flowTexture, diffuseCoord).xy;\n flow = (flow - 0.5) * 2;\n\n vec4 foamPattern =\n texture\n ( foamPatternTexture\n , vec2\n ( diffuseCoord.x + flow.x * osg_FrameTime\n , diffuseCoord.y + flow.y * osg_FrameTime\n )\n );\n\n // ...\n```\n\nFor example, the demo program uses a flow map to animate the water.\nHere you see the flow map being used to animate the\n[foam mask](foam.md#mask).\nThis continuously moves the diffuse UV coordinates directly up,\ngiving the foam mask the appearance of moving down stream.\n\n```c\n // ...\n\n ( diffuseCoord.x + flow.x * osg_FrameTime\n , diffuseCoord.y + flow.y * osg_FrameTime\n\n // ...\n```\n\nYou'll need how many seconds have passed since the first frame\nin order to animate the UV coordinates in the direction indicated by the flow.\n`osg_FrameTime` is\n[provided](https://github.com/panda3d/panda3d/blob/daa57733cb9b4ccdb23e28153585e8e20b5ccdb5/panda/src/display/graphicsStateGuardian.cxx#L930)\nby Panda3D.\nIt is a timestamp of how many seconds have passed since the first frame.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [base.vert](../demonstration/shaders/vertex/base.vert)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [normal.frag](../demonstration/shaders/fragment/normal.frag)\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](foam.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](outlining.md)\n"
},
{
"path": "sections/foam.md",
"content": "[:arrow_backward:](screen-space-refraction.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](flow-mapping.md)\n\n# 3D Game Shaders For Beginners\n\n## Foam\n\n
\n\n
\n\nFoam is typically used when simulating some body of water.\nAnywhere the water's flow is disrupted, you add some foam.\nThe foam isn't much by itself but it can really connect the water with the rest of the scene.\n\n
\n\n
\n\nBut don't stop at just water.\nYou can use the same technique to make a river of lava for example.\n\n### Vertex Positions\n\nLike\n[screen space refraction](screen-space-refraction.md),\nyou'll need both the foreground and background vertex positions.\nThe foreground being the scene with the foamy surface\nand the background being the scene without the foamy surface.\nReferrer back to [SSAO](ssao.md#vertex-positions) for the details\non how to acquire the vertex positions in view space.\n\n### Mask\n\n
\n\n
\n\nYou'll need to texture your scene with a foam mask.\nThe demo masks everything off except the water.\nFor the water, it textures it with a foam pattern.\n\n```c\n// ...\n\nuniform sampler2D foamPatternTexture;\n\nin vec2 diffuseCoord;\n\nout vec4 fragColor;\n\nvoid main() {\n vec4 foamPattern = texture(foamPatternTexture, diffuseCoord);\n\n fragColor = vec4(vec3(dot(foamPattern.rgb, vec3(1)) / 3), 1);\n}\n```\n\nHere you see the fragment shader that generates the foam mask.\nIt takes a foam pattern texture and UV maps it to the scene's geometry using the diffuse UV coordinates.\nFor every model, except the water, the shader is given a solid black texture as the `foamPatternTexture`.\n\n```c\n // ...\n\n fragColor = vec4(vec3(dot(foamPattern.rgb, vec3(1)) / 3), 1);\n\n // ...\n```\n\nThe fragment color is converted to greyscale,\nas a precaution,\nsince the foam shader expects the foam mask to be greyscale.\n\n### Uniforms\n\n```c\n// ...\n\nuniform sampler2D maskTexture;\nuniform sampler2D positionFromTexture;\nuniform sampler2D positionToTexture;\n\n// ...\n```\n\nThe foam shader accepts a mask texture,\nthe foreground vertex positions (`positionFromTexture`),\nand the background vertex positions (`positionToTexture`).\n\n### Parameters\n\n```c\n // ...\n\n float foamDepth = 4;\n vec4 foamColor = vec4(0.8, 0.85, 0.92, 1);\n\n // ...\n```\n\nThe adjustable parameters for the foam shader are the foam depth and color.\nThe foam depth controls how much foam is shown.\nAs the foam depth increases, the amount of foam shown increases.\n\n### Distance\n\n```c\n // ...\n\n vec4 positionFrom = texture(positionFromTexture, texCoord);\n vec4 positionTo = texture(positionToTexture, texCoord);\n\n float depth = (positionTo.xyz - positionFrom.xyz).y;\n\n // ...\n```\n\nCompute the distance from the foreground position to the background position.\nSince the positions are in view (camera) space, we only need the y value since it goes into the screen.\n\n### Amount\n\n```c\n // ...\n\n float amount = clamp(depth / foamDepth.x, 0, 1);\n amount = 1 - amount;\n amount *= mask.r;\n amount = amount * amount / (2 * (amount * amount - amount) + 1);\n\n // ...\n```\n\nThe amount of foam is based on the depth, the foam depth parameter, and the mask value.\n\n
\n\n
\n\n```c\n // ...\n\n amount = amount * amount / (2 * (amount * amount - amount) + 1);\n\n // ...\n```\n\nReshape the amount using the ease in and out easing function.\nThis will give a lot of foam near depth zero and little to no foam near `foamDepth`.\n\n### Fragment Color\n\n```c\n // ...\n\n fragColor = mix(vec4(0), foamColor, amount);\n\n // ...\n```\n\nThe fragment color is a mix between transparent black and the foam color based on the amount.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [base.vert](../demonstration/shaders/vertex/base.vert)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [position.frag](../demonstration/shaders/fragment/position.frag)\n- [foam-mask.frag](../demonstration/shaders/fragment/foam-mask.frag)\n- [foam.frag](../demonstration/shaders/fragment/foam.frag)\n- [base-combine.frag](../demonstration/shaders/fragment/base-combine.frag)\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](screen-space-refraction.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](flow-mapping.md)\n"
},
{
"path": "sections/fog.md",
"content": "[:arrow_backward:](deferred-rendering.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](blur.md)\n\n# 3D Game Shaders For Beginners\n\n## Fog\n\n
\n\n
\n\nFog (or mist as it's called in Blender) adds atmospheric haze to a scene,\nproviding mystique and softening pop-ins (geometry suddenly entering into the camera's frustum).\n\n```c\n// ...\n\nuniform vec4 color;\n\nuniform vec2 nearFar;\n\n// ...\n```\n\nTo calculate the fog, you'll need its color, near distance, and far distance.\n\n```c\n// ...\n\nuniform sampler2D positionTexture;\n\n // ...\n\n vec2 texSize = textureSize(positionTexture, 0).xy;\n vec2 texCoord = gl_FragCoord.xy / texSize;\n\n vec4 position = texture(positionTexture, texCoord);\n\n // ...\n```\n\nIn addition to the fog's attributes, you'll also need the fragment's vertex `position`.\n\n```c\n float fogMin = 0.00;\n float fogMax = 0.97;\n```\n\n`fogMax` controls how much of the scene is still visible when the fog is most intense.\n`fogMin` controls how much of the fog is still visible when the fog is least intense.\n\n```c\n // ...\n\n float near = nearFar.x;\n float far = nearFar.y;\n\n float intensity =\n clamp\n ( (position.y - near)\n / (far - near)\n , fogMin\n , fogMax\n );\n\n // ...\n```\n\nThe example code uses a linear model for calculating the fog's intensity.\nThere's also an exponential model you could use.\n\nThe fog's intensity is `fogMin` before or at the start of the fog's `near` distance.\nAs the vertex `position` gets closer to the end of the fog's `far` distance, the `intensity` moves closer to `fogMax`.\nFor any vertexes after the end of the fog, the `intensity` is clamped to `fogMax`.\n\n```c\n // ...\n\n fragColor = vec4(color.rgb, intensity);\n\n // ...\n```\n\nSet the fragment's color to the fog `color` and the fragment's alpha channel to the `intensity`.\nAs `intensity` gets closer to one, you'll have less of the scene's color and more of the fog color.\nWhen `intensity` reaches one, you'll have all fog color and nothing else.\n\n```c\n// ...\n\nuniform sampler2D baseTexture;\nuniform sampler2D fogTexture;\n\n // ...\n\n vec2 texSize = textureSize(baseTexture, 0).xy;\n vec2 texCoord = gl_FragCoord.xy / texSize;\n\n vec4 baseColor = texture(baseTexture, texCoord);\n vec4 fogColor = texture(fogTexture, texCoord);\n\n fragColor = baseColor;\n\n // ...\n\n fragColor = mix(fragColor, fogColor, min(fogColor.a, 1));\n\n // ...\n```\n\nNormally you calculate the fog in the same shader that does the lighting calculations.\nHowever, you can also break it out into its own framebuffer texture.\nHere you see the fog color being applied to the rest of the scene much like you would apply a layer in GIMP.\nThis allows you to calculate the fog once instead calculating it in every shader that eventually needs it.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [position.frag](../demonstration/shaders/fragment/position.frag)\n- [normal.frag](../demonstration/shaders/fragment/normal.frag)\n- [fog.frag](../demonstration/shaders/fragment/fog.frag)\n- [outline.frag](../demonstration/shaders/fragment/outline.frag)\n- [scene-combine.frag](../demonstration/shaders/fragment/scene-combine.frag)\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](deferred-rendering.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](blur.md)\n"
},
{
"path": "sections/fresnel-factor.md",
"content": "[:arrow_backward:](blinn-phong.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](rim-lighting.md)\n\n# 3D Game Shaders For Beginners\n\n## Fresnel Factor\n\n
\n\n
\n\nThe fresnel factor alters the reflectiveness of a surface based on the camera or viewing angle.\nAs a surface points away from the camera, its reflectiveness goes up.\nSimilarly, as a surface points towards the camera, its reflectiveness goes down.\n\n
\n\n
\n\nIn other words, as a surface becomes perpendicular with the camera, it becomes more mirror like.\nUtilizing this property, you can vary the opacity of reflections\n(such as [specular](lighting.md#specular) and [screen space reflections](screen-space-reflection.md))\nand/or vary a surface's alpha values for a more plausible or realistic look.\n\n### Specular Reflection\n\n
\n\n
\n\n```c\n vec4 specular = materialSpecularColor\n * lightSpecularColor\n * pow(max(dot(eye, reflection), 0.0), shininess);\n```\n\nIn the [lighting](lighting.md#specular) section,\nthe specular component was a combination of the\nmaterial's specular color,\nthe light's specular color,\nand by how much the camera pointed into the light's reflection direction.\nIncorporating the fresnel factor,\nyou'll now vary the material specular color based on the angle between the camera and the surface it's pointed at.\n\n```c\n // ...\n\n vec3 eye = normalize(-vertexPosition.xyz);\n\n // ...\n```\n\nThe first vector you'll need is the eye/view/camera vector.\nRecall that the eye vector points from the vertex position to the camera's position.\nIf the vertex position is in view or camera space,\nthe eye vector is the vertex position pointed in the opposite direction.\n\n```c\n // ...\n\n vec3 light = normal(lightPosition.xyz - vertexPosition.xyz);\n vec3 halfway = normalize(light + eye);\n\n // ...\n```\n\nThe fresnel factor is calculated using two vectors.\nThe simplest two vectors to use are the eye and normal vector.\nHowever, if you're using the halfway vector (from the [Blinn-Phong](blinn-phong.md) section),\nyou can instead calculate the fresnel factor using the halfway and eye vector.\n\n```c\n\n // ...\n\n float fresnelFactor = dot(halfway, eye); // Or dot(normal, eye).\n fresnelFactor = max(fresnelFactor, 0.0);\n fresnelFactor = 1.0 - fresnelFactor;\n fresnelFactor = pow(fresnelFactor, fresnelPower);\n\n // ...\n```\n\nWith the needed vectors in hand,\nyou can now compute the fresnel factor.\nThe fresnel factor ranges from zero to one.\nWhen the dot product is one,\nthe fresnel factor is zero.\nWhen the dot product is less than or equal to zero,\nthe fresnel factor is one.\nThis equation comes from\n[Schlick's approximation](https://en.wikipedia.org/wiki/Schlick%27s_approximation).\n\n
\n\n
\n\nIn Schlick's approximation,\nthe `fresnelPower` is five but you can alter this to your liking.\nThe demo code varies it using the blue channel of the specular map with a maximum value of five.\n\n```c\n // ...\n\n materialSpecularColor.rgb = mix(materialSpecularColor.rgb, vec3(1.0), fresnelFactor);\n\n // ...\n```\n\nOnce the fresnel factor is determined,\nuse it to modulate the material's specular color.\nAs the fresnel factor approaches one,\nthe material becomes more like a mirror or fully reflective.\n\n```c\n // ...\n\n vec4 specular = vec4(vec3(0.0), 1.0);\n specular.rgb = materialSpecularColor.rgb\n * lightSpecularColor.rgb\n * pow\n ( max(dot(normal, halfway), 0.0) // Or max(dot(reflection, eye), 0.0).\n , shininess\n );\n\n // ...\n```\n\nAs before,\nthe specular component is a combination of the\nmaterial's specular color,\nthe light's specular color,\nand by how much the camera points into the direction of the light's reflection.\nHowever,\nusing the fresnel factor,\nthe material's specular color various depending on the orientation of the camera and the surface it's looking at.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [base.vert](../demonstration/shaders/vertex/base.vert)\n- [base.frag](../demonstration/shaders/fragment/base.frag)\n\n## Copyright\n\n(C) 2020 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](blinn-phong.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](rim-lighting.md)\n"
},
{
"path": "sections/gamma-correction.md",
"content": "[:arrow_backward:](lookup-table.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](setup.md)\n\n# 3D Game Shaders For Beginners\n\n## Gamma Correction\n\n
\n\n
\n\nCorrecting for gamma will make your color calculations look correct.\nThis isn't to say they'll look amazing but with gamma correction,\nyou'll find that the colors meld together better,\nthe shadows are more nuanced,\nand the highlights are more subtle.\nWithout gamma correction,\nthe shadowed areas tend to get crushed while the highlighted areas tend to get blown-out and\nover saturated making for a harsh contrast overall.\n\nIf you're aiming for realism,\ngamma correction is especially important.\nAs you perform more and more calculations,\nthe tiny errors add up making it harder to achieve photorealism.\nThe equations will be correct but the inputs and outputs will be wrong leaving you frustrated.\n\nIt's easy to get twisted around when thinking about gamma correction\nbut essentially it boils down to knowing what color space a color is in and how to convert that color to the color space you need.\nWith those two pieces of the puzzle,\ngamma correction becomes a tedious yet simple chore you'll have to perform from time to time.\n\n### Color Spaces\n\n
\n\n
\n\nThe two color spaces you'll need to be aware of are sRGB (standard Red Green Blue) and RGB or linear color space.\n\n```bash\nidentify -format \"%[colorspace]\\n\" house-diffuse-srgb.png\nsRGB\n\nidentify -format \"%[colorspace]\\n\" house-diffuse-rgb.png\nRGB\n```\n\nKnowing what color space a color texture is in will determine how you handle it in your shaders.\nTo determine the color space of a texture, use ImageMagick's `identify`.\nYou'll find that most textures are in sRGB.\n\n```bash\nconvert house-diffuse-srgb -colorspace rgb house-diffuse-rgb.png\n```\n\nTo convert a texture to a particular color space, use ImageMagick's `convert` program.\nNotice how a texture is darkened when transforming from sRGB to RGB.\n\n### Decoding\n\n\nThe red, green, and blue values in a sRGB color texture are encoded and cannot be modified directly.\nModifying them directly would be like running spellcheck on an encrypted message.\nBefore you can run spellcheck,\nyou first have to decrypt the message.\nSimilarly,\nto modify the values of an sRGB texture,\nyou first have to decode or transform them to RGB or linear color space.\n\n```c\n // ...\n\n color = texture(color_texture, uv);\n color.rgb = pow(color.rgb, 2.2);\n\n // ...\n```\n\nTo decode a sRGB encoded color,\nraise the `rgb` values to the power of `2.2`.\nOnce you have decoded the color,\nyou are now free to add, subtract, multiply, and divide it.\n\nBy raising the color values to the power of `2.2`,\nyou're converting them from sRGB to RGB or linear color space.\nThis conversion has the effect of darkening the colors.\n\n
\n\nThe `2.2` value is known as gamma.\nLoosely speaking, gamma can either be `1.0 / 2.2`, `2.2`, or `1.0`.\nAs you've seen, `2.2` is for decoding sRGB encoded color textures.\nAs you will see, `1.0 / 2.2` is for encoding linear or RGB color textures.\nAnd `1.0` is RGB or linear color space since `y = 1 * x + 0` and\nany base raised to the power of `1.0` is itself.\n\n#### Non-color Data\n\n
\n\n
\n\nOne important exception to decoding is when the \"colors\" of a texture represent non-color data.\nSome examples of non-color data would be the normals in a normal map,\nthe alpha channel,\nthe heights in a height map,\nand the directions in a flow map.\nOnly decode color related data or data that represents color.\nWhen dealing with non-color data,\ntreat the sRGB color values as RGB or linear and skip the decoding process.\n\n### Encoding\n\n
\n\n
\n\nThe necessity for encoding and decoding stems from the fact that humans do not perceive lightness linearly and\nmost displays (like a monitor) lack the precision or number of bits to accurately show both lighter and darker tonal values or shades.\nWith only so many bits to go around,\ncolors are encoded in such a way that more bits are devoted to the darker shades than the lighter shades\nsince humans are more sensitive to darker tones than lighter tones.\nEncoding it this way uses the limited number of bits more effectively for human perception.\nStill, the only thing to remember is that your display is expecting sRGB encoded values.\nTherefore, if you decoded a sRGB value, you have to encode it before it makes its way to your display.\n\n```c\n // ...\n\n color = texture(color_texture, uv);\n color.rgb = pow(color.rgb, 1.0 / 2.2);\n\n // ...\n```\n\nTo encode a linear value or convert RGB to sRGB,\nraise the `rgb` values to the power of `1.0 / 2.2`.\nNotice how `1.1 / 2.2` is the reciprocal of `2.2` or `2.2 / 1.0`.\nHere you see the symmetry in decoding and encoding.\n\n
\n\n
\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [gamma-correction.frag](../demonstration/shaders/fragment/gamma-correction.frag)\n\n## Copyright\n\n(C) 2020 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](lookup-table.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](setup.md)\n"
},
{
"path": "sections/glsl.md",
"content": "[:arrow_backward:](reference-frames.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](render-to-texture.md)\n\n# 3D Game Shaders For Beginners\n\n## GLSL\n\n
\n\n
\n\nInstead of using the\n[fixed-function](https://en.wikipedia.org/wiki/Fixed-function)\npipeline,\nyou'll be using the programmable GPU rendering pipeline.\nSince it is programmable, it is up to you to supply the programming in the form of shaders.\nA shader is a (typically small) program you write using a syntax reminiscent of C.\nThe programmable GPU rendering pipeline has various different stages that you can program with shaders.\nThe different types of shaders include vertex, tessellation, geometry, fragment, and compute.\nYou'll only need to focus on the vertex and fragment stages for the techniques below.\n\n```c\n#version 150\n\nvoid main() {}\n```\n\nHere is a bare-bones GLSL shader consisting of the GLSL version number and the main function.\n\n```c\n#version 150\n\nuniform mat4 p3d_ModelViewProjectionMatrix;\n\nin vec4 p3d_Vertex;\n\nvoid main()\n{\n gl_Position = p3d_ModelViewProjectionMatrix * p3d_Vertex;\n}\n```\n\nHere is a stripped down GLSL vertex shader that transforms an incoming vertex to clip space\nand outputs this new position as the vertex's homogeneous position.\nThe `main` procedure doesn't return anything since it is `void` and the `gl_Position` variable is a built-in output.\n\nTake note of the keywords `uniform` and `in`.\nThe `uniform` keyword means this global variable is the same for all vertexes.\nPanda3D sets the `p3d_ModelViewProjectionMatrix` for you and it is the same matrix for each vertex.\nThe `in` keyword means this global variable is being given to the shader.\nThe vertex shader receives each vertex that makes up the geometry the vertex shader is attached to.\n\n```c\n#version 150\n\nout vec4 fragColor;\n\nvoid main() {\n fragColor = vec4(0, 1, 0, 1);\n}\n```\n\nHere is a stripped down GLSL fragment shader that outputs the fragment color as solid green.\nKeep in mind that a fragment affects at most one screen pixel but a single pixel can be affected by many fragments.\n\nTake note of the `out` keyword.\nThe `out` keyword means this global variable is being set by the shader.\nThe name `fragColor` is arbitrary so feel free to choose a different one.\n\n
\n\n
\n\nThis is the output of the two shaders shown above.\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](reference-frames.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](render-to-texture.md)\n"
},
{
"path": "sections/lighting.md",
"content": "[:arrow_backward:](texturing.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](blinn-phong.md)\n\n# 3D Game Shaders For Beginners\n\n## Lighting\n\n
\n\n
\n\nCompleting the lighting involves calculating and combining the ambient, diffuse, specular, and emission light aspects.\nThe example code uses either Phong or Blinn-Phong lighting.\n\n### Vertex\n\n```c\n// ...\n\nuniform struct p3d_LightSourceParameters\n { vec4 color\n\n ; vec4 ambient\n ; vec4 diffuse\n ; vec4 specular\n\n ; vec4 position\n\n ; vec3 spotDirection\n ; float spotExponent\n ; float spotCutoff\n ; float spotCosCutoff\n\n ; float constantAttenuation\n ; float linearAttenuation\n ; float quadraticAttenuation\n\n ; vec3 attenuation\n\n ; sampler2DShadow shadowMap\n\n ; mat4 shadowViewMatrix\n ;\n } p3d_LightSource[NUMBER_OF_LIGHTS];\n\n// ...\n```\n\nFor every light, minus the ambient light, Panda3D gives you this convenient\nstruct which is available to both the vertex and fragment shaders.\nThe biggest convenience being the shadow map and shadow view matrix for transforming vertexes to shadow or light space.\n\n```c\n // ...\n\n vertexPosition = p3d_ModelViewMatrix * p3d_Vertex;\n\n // ...\n\n for (int i = 0; i < p3d_LightSource.length(); ++i) {\n vertexInShadowSpaces[i] = p3d_LightSource[i].shadowViewMatrix * vertexPosition;\n }\n\n // ...\n```\n\nStarting in the vertex shader, you'll need to transform and\noutput the vertex from view space to shadow or light space for each light in your scene.\nYou'll need this later in the fragment shader in order to render the shadows.\nShadow or light space is where every coordinate is relative to the light position (the light is the origin).\n\n### Fragment\n\nThe fragment shader is where most of the lighting calculations take place.\n\n#### Material\n\n```c\n// ...\n\nuniform struct\n { vec4 ambient\n ; vec4 diffuse\n ; vec4 emission\n ; vec3 specular\n ; float shininess\n ;\n } p3d_Material;\n\n// ...\n```\n\nPanda3D gives us the material (in the form of a struct) for the mesh or model you are currently rendering.\n\n#### Multiple Lights\n\n```c\n // ...\n\n vec4 diffuse = vec4(0.0, 0.0, 0.0, diffuseTex.a);\n vec4 specular = vec4(0.0, 0.0, 0.0, diffuseTex.a);\n\n // ...\n```\n\nBefore you loop through the scene's lights, create an accumulator for both the diffuse and specular colors.\n\n```c\n // ...\n\n for (int i = 0; i < p3d_LightSource.length(); ++i) {\n // ...\n }\n\n // ...\n```\n\nNow you can loop through the lights, calculating the diffuse and specular colors for each one.\n\n#### Light Related Vectors\n\n
\n\n
\n\nHere you see the four major vectors you'll need to calculate the diffuse and specular colors contributed by each light.\nThe light direction vector is the light blue arrow pointing to the light.\nThe normal vector is the green arrow standing straight up.\nThe reflection vector is the dark blue arrow mirroring the light direction vector.\nThe eye or view vector is the orange arrow pointing towards the camera.\n\n```c\n // ...\n\n vec3 lightDirection =\n p3d_LightSource[i].position.xyz\n - vertexPosition.xyz\n * p3d_LightSource[i].position.w;\n\n // ...\n```\n\nThe light direction is from the vertex's position to the light's position.\n\nPanda3D sets `p3d_LightSource[i].position.w` to zero if this is a directional light.\nDirectional lights do not have a position as they only have a direction.\nSo if this is a directional light,\nthe light direction will be the negative or opposite direction of the light as Panda3D sets\n`p3d_LightSource[i].position.xyz` to be `-direction` for directional lights.\n\n```c\n // ...\n\n normal = normalize(vertexNormal);\n\n // ...\n```\n\nYou'll need the vertex normal to be a unit vector.\nUnit vectors have a length of magnitude of one.\n\n```c\n // ...\n\n vec3 unitLightDirection = normalize(lightDirection);\n vec3 eyeDirection = normalize(-vertexPosition.xyz);\n vec3 reflectedDirection = normalize(-reflect(unitLightDirection, normal));\n\n // ...\n```\n\nNext you'll need three more vectors.\n\nYou'll need to take the dot product involving the light direction so its best to normalize it.\nThis gives it a distance or magnitude of one (unit vector).\n\nThe eye direction is the opposite of the vertex/fragment position since the vertex/fragment position is relative to the camera's position.\nRemember that the vertex/fragment position is in view space.\nSo instead of going from the camera (eye) to the vertex/fragment, you go from the vertex/fragment to the eye (camera).\n\nThe\n[reflection vector](http://asawicki.info/news_1301_reflect_and_refract_functions.html)\nis a reflection of the light direction at the surface normal.\nAs the light \"ray\" hits the surface, it bounces off at the same angle it came in at.\nThe angle between the light direction vector and the normal is known as the \"angle of incidence\".\nThe angle between the reflection vector and the normal is known as the \"angle of reflection\".\n\nYou'll have to negate the reflected light vector as it needs to point in the same direction as the eye vector.\nRemember the eye direction is from the vertex/fragment to the camera position.\nYou'll use the reflection vector to calculate the intensity of the specular highlight.\n\n#### Diffuse\n\n```c\n // ...\n\n float diffuseIntensity = dot(normal, unitLightDirection);\n\n if (diffuseIntensity < 0.0) { continue; }\n\n // ...\n```\n\nThe diffuse intensity is the dot product between the surface normal and the unit vector light direction.\nThe dot product can range from negative one to one.\nIf both vectors point in the same direction, the intensity is one.\nAny other case will be less than one.\n\n
\n\n
\n\nAs the light vector approaches the same direction as the normal, the diffuse intensity approaches one.\n\n```c\n // ...\n\n if (diffuseIntensity < 0.0) { continue; }\n\n // ...\n```\n\nIf the diffuse intensity is zero or less, move on to the next light.\n\n```c\n // ...\n\n vec4 diffuseTemp =\n vec4\n ( clamp\n ( diffuseTex.rgb\n * p3d_LightSource[i].diffuse.rgb\n * diffuseIntensity\n , 0\n , 1\n )\n , diffuseTex.a\n );\n\n diffuseTemp = clamp(diffuseTemp, vec4(0), diffuseTex);\n\n // ...\n```\n\nYou can now calculate the diffuse color contributed by this light.\nIf the diffuse intensity is one, the diffuse color will be a mix between the diffuse texture color and the lights color.\nAny other intensity will cause the diffuse color to be darker.\n\nNotice how I clamp the diffuse color to be only as bright as the diffuse texture color is.\nThis will protect the scene from being over exposed.\nWhen creating your diffuse textures, make sure to create them as if they were fully lit.\n\n#### Specular\n\nAfter diffuse, comes specular.\n\n
\n\n
\n\n```c\n // ...\n\n float specularIntensity = max(dot(reflectedDirection, eyeDirection), 0);\n\n vec4 specularTemp =\n clamp\n ( vec4(p3d_Material.specular, 1)\n * p3d_LightSource[i].specular\n * pow\n ( specularIntensity\n , p3d_Material.shininess\n )\n , 0\n , 1\n );\n\n // ...\n```\n\nThe specular intensity is the dot product between the eye vector and the reflection vector.\nAs with the diffuse intensity, if the two vectors point in the same direction, the specular intensity is one.\nAny other intensity will diminish the amount of specular color contributed by this light.\n\n
\n\n
\n\nThe material shininess determines how spread out the specular highlight is.\nThis is typically set in a modeling program like Blender.\nIn Blender it's known as the specular hardness.\n\n#### Spotlights\n\n```c\n // ...\n\n float unitLightDirectionDelta =\n dot\n ( normalize(p3d_LightSource[i].spotDirection)\n , -unitLightDirection\n );\n\n if (unitLightDirectionDelta < p3d_LightSource[i].spotCosCutoff) { continue; }\n\n // ...\n}\n```\n\nThis snippet keeps fragments outside of a spotlight's cone or frustum from being affected by the light.\nFortunately, Panda3D\n[sets up](https://github.com/panda3d/panda3d/blob/daa57733cb9b4ccdb23e28153585e8e20b5ccdb5/panda/src/display/graphicsStateGuardian.cxx#L1705)\n`spotDirection` and `spotCosCutoff` to also work for directional lights and points lights.\nSpotlights have both a position and direction.\nHowever, directional lights only have a direction and point lights only have a position.\nStill, this code works for all three lights avoiding the need for noisy if statements.\n\n```c\n // ...\n\n , -unitLightDirection\n\n // ...\n```\n\nYou must negate `unitLightDirection`.\n`unitLightDirection` goes from the fragment to the spotlight and you need it to go from the spotlight to the fragment\nsince the `spotDirection` goes directly down the center of the spotlight's frustum some distance away from the spotlight's position.\n\n```c\nspotCosCutoff = cosine(0.5 * spotlightLensFovAngle);\n```\n\nFor a spotlight, if the dot product between the fragment-to-light vector and the spotlight's direction vector is less than the cosine\nof half the spotlight's field of view angle, the shader disregards this light's influence.\n\nFor directional lights and point lights, Panda3D sets `spotCosCutoff` to negative one.\nRecall that the dot product ranges from negative one to one.\nSo it doesn't matter what the `unitLightDirectionDelta` is because it will always be greater than or equal to negative one.\n\n```c\n // ...\n\n diffuseTemp *= pow(unitLightDirectionDelta, p3d_LightSource[i].spotExponent);\n\n // ...\n```\n\nLike the `unitLightDirectionDelta` snippet, this snippet also works for all three light types.\nFor spotlights, this will make the fragments brighter as you move closer to the center of the spotlight's frustum.\nFor directional lights and point lights, `spotExponent` is zero.\nRecall that anything to the power of zero is one so the diffuse color is one times itself meaning it is unchanged.\n\n#### Shadows\n\n```c\n // ...\n\n float shadow =\n textureProj\n ( p3d_LightSource[i].shadowMap\n , vertexInShadowSpaces[i]\n );\n\n diffuseTemp.rgb *= shadow;\n specularTemp.rgb *= shadow;\n\n // ...\n```\n\nPanda3D makes applying shadows relatively easy by providing the shadow map and shadow transformation matrix for every scene light.\nTo create the shadow transformation matrix yourself,\nyou'll need to assemble a matrix that transforms view space coordinates to light space (coordinates are relative to the light's position).\nTo create the shadow map yourself, you'll need to render the scene from the perspective of the light to a framebuffer texture.\nThe framebuffer texture must hold the distances from the light to the fragments.\nThis is known as a \"depth map\".\nLastly, you'll need to manually give to your shader your DIY depth map as a `uniform sampler2DShadow`\nand your DIY shadow transformation matrix as a `uniform mat4`.\nAt this point, you've recreated what Panda3D does for you automatically.\n\nThe shadow snippet shown uses `textureProj` which is different from the `texure` function shown earlier.\n`textureProj` first divides `vertexInShadowSpaces[i].xyz` by `vertexInShadowSpaces[i].w`.\nAfter this, it uses `vertexInShadowSpaces[i].xy` to locate the depth stored in the shadow map.\nNext it uses `vertexInShadowSpaces[i].z` to compare this vertex's depth against the shadow map depth at\n`vertexInShadowSpaces[i].xy`.\nIf the comparison passes, `textureProj` will return one.\nOtherwise, it will return zero.\nZero meaning this vertex/fragment is in the shadow and one meaning this vertex/fragment is not in the shadow.\n\n`textureProj` can also return a value between zero and one depending on how the shadow map was set up.\nIn this instance, `textureProj` performs multiple depth tests using neighboring depth values and returns a weighted average.\nThis weighted average can give shadows a softer look.\n\n#### Attenuation\n\n
\n\n
\n\n```c\n // ...\n\n float lightDistance = length(lightDirection);\n\n float attenuation =\n 1\n / ( p3d_LightSource[i].constantAttenuation\n + p3d_LightSource[i].linearAttenuation\n * lightDistance\n + p3d_LightSource[i].quadraticAttenuation\n * (lightDistance * lightDistance)\n );\n\n diffuseTemp.rgb *= attenuation;\n specularTemp.rgb *= attenuation;\n\n // ...\n```\n\nThe light's distance is just the magnitude or length of the light direction vector.\nNotice it's not using the normalized light direction as that distance would be one.\n\nYou'll need the light distance to calculate the attenuation.\nAttenuation meaning the light's influence diminishes as you get further away from it.\n\nYou can set `constantAttenuation`, `linearAttenuation`, and `quadraticAttenuation` to whatever values you would like.\nA good starting point is `constantAttenuation = 1`, `linearAttenuation = 0`, and `quadraticAttenuation = 1`.\nWith these settings, the attenuation is one at the light's position and approaches zero as you move further away.\n\n#### Final Light Color\n\n```c\n // ...\n\n diffuse += diffuseTemp;\n specular += specularTemp;\n\n // ...\n```\n\nTo calculate the final light color, add the diffuse and specular together.\nBe sure to add this to the accumulator as you loop through the scene's lights.\n\n#### Ambient\n\n```c\n// ...\n\nuniform sampler2D p3d_Texture1;\n\n// ...\n\nuniform struct\n { vec4 ambient\n ;\n } p3d_LightModel;\n\n// ...\n\nin vec2 diffuseCoord;\n\n // ...\n\n vec4 diffuseTex = texture(p3d_Texture1, diffuseCoord);\n\n // ...\n\n vec4 ambient = p3d_Material.ambient * p3d_LightModel.ambient * diffuseTex;\n\n// ...\n```\n\nThe ambient component to the lighting model is based on the material's ambient color,\nthe ambient light's color, and the diffuse texture color.\n\nThere should only ever be one ambient light.\nBecause of this, the ambient color calculation only needs to occur once.\nContrast this with the diffuse and specular color which must be accumulated for each spot/directional/point light.\nWhen you reach [SSAO](ssao.md), you'll revisit the ambient color calculation.\n\n#### Putting It All Together\n\n```c\n // ...\n\n vec4 outputColor = ambient + diffuse + specular + p3d_Material.emission;\n\n // ...\n```\n\nThe final color is the sum of the ambient color, diffuse color, specular color, and the emission color.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [base.vert](../demonstration/shaders/vertex/base.vert)\n- [base.frag](../demonstration/shaders/fragment/base.frag)\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](texturing.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](blinn-phong.md)\n"
},
{
"path": "sections/lookup-table.md",
"content": "[:arrow_backward:](film-grain.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](gamma-correction.md)\n\n# 3D Game Shaders For Beginners\n\n## Lookup Table (LUT)\n\n
\n\n
\n\nThe lookup table or LUT shader allows you to transform the colors of your game\nusing an image editor like the [GIMP](https://www.gimp.org/).\nFrom color grading to turning day into night,\nthe LUT shader is a handy tool for tweaking the look of your game.\n\n
\n\n
\n\nBefore you can get started,\nyou'll need to find a neutral LUT image.\nNeutral meaning that it leaves the fragment colors unchanged.\nThe LUT needs to be 256 pixels wide by 16 pixels tall and contain 16 blocks\nwith each block being 16 by 16 pixels.\n\nThe LUT is mapped out into 16 blocks.\nEach block has a different level of blue.\nAs you move across the blocks, from left to right, the amount of blue increases.\nYou can see the amount of blue in each block's upper-left corner.\nWithin each block,\nthe amount of red increases as you move from left to right and\nthe amount of green increases as you move from top to bottom.\nThe upper-left corner of the first block is black since every RGB channel is zero.\nThe lower-right corner of the last block is white since every RGB channel is one.\n\n
\n\n
\n\nWith the neutral LUT in hand, take a screenshot of your game and open it in your image editor.\nAdd the neutral LUT as a new layer and merge it with the screenshot.\nAs you manipulate the colors of the screenshot, the LUT will be altered in the same way.\nWhen you're done editing, select only the LUT and save it as a new image.\nYou now have your new lookup table and can begin writing your shader.\n\n```c\n // ...\n\n vec2 texSize = textureSize(colorTexture, 0).xy;\n\n vec4 color = texture(colorTexture, gl_FragCoord.xy / texSize);\n\n // ...\n```\n\nThe LUT shader is a screen space technique.\nTherefore, sample the scene's color at the current fragment or screen position.\n\n```c\n // ...\n\n float u = floor(color.b * 15.0) / 15.0 * 240.0;\n u = (floor(color.r * 15.0) / 15.0 * 15.0) + u;\n u /= 255.0;\n\n float v = ceil(color.g * 15.0);\n v /= 15.0;\n v = 1.0 - v;\n\n // ...\n```\n\nIn order to transform the current fragment's color,\nusing the LUT,\nyou'll need to map the color to two UV coordinates on the lookup table texture.\nThe first mapping (shown up above) is to the nearest left or lower bound block location and\nthe second mapping (shown below) is to the nearest right or upper bound block mapping.\nAt the end, you'll combine these two mappings to create the final color transformation.\n\n
\n\n
\n\nEach of the red, green, and blue channels maps to one of 16 possibilities in the LUT.\nThe blue channel maps to one of the 16 upper-left block corners.\nAfter the blue channel maps to a block,\nthe red channel maps to one of the 16 horizontal pixel positions within the block and\nthe green channel maps to one of the 16 vertical pixel positions within the block.\nThese three mappings will determine the UV coordinate you'll need to sample a color from the LUT.\n\n```c\n // ...\n\n u /= 255.0;\n\n v /= 15.0;\n v = 1.0 - v;\n\n // ...\n```\n\nTo calculate the final U coordinate, divide it by 255 since the LUT is 256 pixels wide and U ranges from zero to one.\nTo calculate the final V coordinate, divide it by 15 since the LUT is 16 pixels tall and V ranges from zero to one.\nYou'll also need to subtract the normalized V coordinate from one since V ranges from zero at the bottom to one at the top while\nthe green channel ranges from zero at the top to 15 at the bottom.\n\n```c\n // ...\n\n vec3 left = texture(lookupTableTexture, vec2(u, v)).rgb;\n\n // ...\n```\n\nUsing the UV coordinates, sample a color from the lookup table.\nThis is the nearest left block color.\n\n```c\n // ...\n\n u = ceil(color.b * 15.0) / 15.0 * 240.0;\n u = (ceil(color.r * 15.0) / 15.0 * 15.0) + u;\n u /= 255.0;\n\n v = 1.0 - (ceil(color.g * 15.0) / 15.0);\n\n vec3 right = texture(lookupTableTexture, vec2(u, v)).rgb;\n\n // ...\n```\n\nNow you'll need to calculate the UV coordinates for the nearest right block color.\nNotice how `ceil` or ceiling is being used now instead of `floor`.\n\n
\n\n
\n\n```c\n // ...\n\n color.r = mix(left.r, right.r, fract(color.r * 15.0));\n color.g = mix(left.g, right.g, fract(color.g * 15.0));\n color.b = mix(left.b, right.b, fract(color.b * 15.0));\n\n // ...\n```\n\nNot every channel will map perfectly to one of its 16 possibilities.\nFor example, `0.5` doesn't map perfectly.\nAt the lower bound (`floor`),\nit maps to `0.4666666666666667` and at the upper bound (`ceil`),\nit maps to `0.5333333333333333`.\nCompare that with `0.4` which maps to `0.4` at the lower bound and `0.4` at the upper bound.\nFor those channels which do not map perfectly,\nyou'll need to mix the left and right sides based on where the channel falls between its lower and upper bound.\nFor `0.5`, it falls directly between them making the final color a mixture of half left and half right.\nHowever,\nfor `0.132` the mixture will be 98% right and 2% left since the fractional part of `0.123` times `15.0` is `0.98`.\n\n```c\n // ...\n\n fragColor = color;\n\n // ...\n```\n\nSet the fragment color to the final mix and you're done.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [lookup-table.frag](../demonstration/shaders/fragment/lookup-table.frag)\n\n## Copyright\n\n(C) 2020 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](film-grain.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](gamma-correction.md)\n"
},
{
"path": "sections/motion-blur.md",
"content": "[:arrow_backward:](ssao.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](chromatic-aberration.md)\n\n# 3D Game Shaders For Beginners\n\n## Motion Blur\n\n
\n\n
\n\nTo really sell the illusion of speed, you can do no better than motion blur.\nFrom high speed car chases to moving at warp speed,\nmotion blur greatly improves the look and feel of fast moving objects.\n\nThere are a few ways to implement motion blur as a screen space technique.\nThe less involved implementation will only blur the scene in relation to the camera's movements\nwhile the more involved version will blur any moving objects even with the camera remaining still.\nThe less involved technique is described below but the principle is the same.\n\n### Textures\n\n```c\nuniform sampler2D positionTexture;\nuniform sampler2D colorTexture;\n\n// ...\n```\n\nThe input textures needed are the vertex positions in view space and the scene's colors.\nRefer back to [SSAO](ssao.md#vertex-positions) for acquiring the vertex positions.\n\n### Matrices\n\n```c\n// ...\n\nuniform mat4 previousViewWorldMat;\nuniform mat4 worldViewMat;\nuniform mat4 lensProjection;\n\n// ...\n```\n\nThe motion blur technique determines the blur direction by comparing\nthe previous frame's vertex positions with the current frame's vertex positions.\nTo do this, you'll need the previous frame's view-to-world matrix,\nthe current frame's world-to-view matrix,\nand the camera lens' projection matrix.\n\n### Parameters\n\n```c\n// ...\n\nuniform vec2 parameters;\n\n// ...\n\nvoid main() {\n int size = int(parameters.x);\n float separation = parameters.y;\n\n// ...\n```\n\nThe adjustable parameters are `size` and `separation`.\n`size` controls how many samples are taken along the blur direction.\nIncreasing `size` increases the amount of blur at the cost of performance.\n`separation` controls how spread out the samples are along the blur direction.\nIncreasing `separation` increases the amount of blur at the cost of accuracy.\n\n### Blur Direction\n\n```c\n // ...\n\n vec2 texSize = textureSize(colorTexture, 0).xy;\n vec2 texCoord = gl_FragCoord.xy / texSize;\n\n vec4 position1 = texture(positionTexture, texCoord);\n vec4 position0 = worldViewMat * previousViewWorldMat * position1;\n\n // ...\n```\n\nTo determine which way to blur this fragment,\nyou'll need to know where things were last frame and where things are this frame.\nTo figure out where things are now,\nsample the current vertex position.\nTo figure out where things were last frame,\ntransform the current position from view space to world space,\nusing the previous frame's view-to-world matrix,\nand then transform it back to view space from world space using this frame's world-to-view matrix.\nThis transformed position is this fragment's previous interpolated vertex position.\n\n
\n\n
\n\n```c\n // ...\n\n position0 = lensProjection * position0;\n position0.xyz /= position0.w;\n position0.xy = position0.xy * 0.5 + 0.5;\n\n position1 = lensProjection * position1;\n position1.xyz /= position1.w;\n position1.xy = position1.xy * 0.5 + 0.5;\n\n // ...\n```\n\nNow that you have the current and previous positions,\ntransform them to screen space.\nWith the positions in screen space,\nyou can trace out the 2D direction you'll need to blur the onscreen image.\n\n```c\n // ...\n\n // position1.xy = position0.xy + direction;\n vec2 direction = position1.xy - position0.xy;\n\n if (length(direction) <= 0.0) { return; }\n\n // ...\n```\n\nThe blur direction goes from the previous position to the current position.\n\n### Blurring\n\n```c\n // ...\n\n fragColor = texture(colorTexture, texCoord);\n\n // ...\n```\n\nSample the current fragment's color.\nThis will be the first of the colors blurred together.\n\n```c\n // ...\n\n direction.xy *= separation;\n\n // ...\n```\n\nMultiply the direction vector by the separation.\n\n```c\n // ...\n\n vec2 forward = texCoord;\n vec2 backward = texCoord;\n\n // ...\n```\n\nFor a more seamless blur,\nsample in the direction of the blur and in the opposite direction of the blur.\nFor now, set the two vectors to the fragment's UV coordinate.\n\n```c\n // ...\n\n float count = 1.0;\n\n // ...\n```\n\n`count` is used to average all of the samples taken.\nIt starts at one since you've already sampled the current fragment's color.\n\n```c\n // ...\n\n for (int i = 0; i < size; ++i) {\n forward += direction;\n backward -= direction;\n\n fragColor +=\n texture\n ( colorTexture\n , forward\n );\n fragColor +=\n texture\n ( colorTexture\n , backward\n );\n\n count += 2.0;\n }\n\n // ...\n```\n\nSample the screen's colors both in the forward and backward direction of the blur.\nBe sure to add these samples together as you travel along.\n\n```c\n // ...\n\n fragColor /= count;\n}\n```\n\nThe final fragment color is the average color of the samples taken.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [position.frag](../demonstration/shaders/fragment/position.frag)\n- [motion-blur.frag](../demonstration/shaders/fragment/motion-blur.frag)\n\n## Copyright\n\n(C) 2020 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](ssao.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](chromatic-aberration.md)\n"
},
{
"path": "sections/normal-mapping.md",
"content": "[:arrow_backward:](cel-shading.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](deferred-rendering.md)\n\n# 3D Game Shaders For Beginners\n\n## Normal Mapping\n\n
\n\n
\n\nNormal mapping allows you to add surface details without adding any geometry.\nTypically, in a modeling program like Blender, you create a high poly and a low poly version of your mesh.\nYou take the vertex normals from the high poly mesh and bake them into a texture.\nThis texture is the normal map.\nThen inside the fragment shader, you replace the low poly mesh's vertex normals with the\nhigh poly mesh's normals you baked into the normal map.\nNow when you light your mesh, it will appear to have more polygons than it really has.\nThis will keep your FPS high while at the same time retain most of the details from the high poly version.\n\n
\n\n
\n\nHere you see the progression from the high poly model to the low poly model to the low poly model with the normal map applied.\n\n
\n\n
\n\nKeep in mind though, normal mapping is only an illusion.\nAfter a certain angle, the surface will look flat again.\n\n### Vertex\n\n```c\n// ...\n\nuniform mat3 p3d_NormalMatrix;\n\n// ...\n\nin vec3 p3d_Normal;\n\n// ...\n\nin vec3 p3d_Binormal;\nin vec3 p3d_Tangent;\n\n // ...\n\n vertexNormal = normalize(p3d_NormalMatrix * p3d_Normal);\n binormal = normalize(p3d_NormalMatrix * p3d_Binormal);\n tangent = normalize(p3d_NormalMatrix * p3d_Tangent);\n\n // ...\n```\n\nStarting in the vertex shader,\nyou'll need to output to the fragment shader the normal vector, binormal vector, and the tangent vector.\nThese vectors are used, in the fragment shader, to transform the normal map normal from tangent space to view space.\n\n`p3d_NormalMatrix` transforms the vertex normal, binormal, and tangent vectors to view space.\nRemember that in view space, all of the coordinates are relative to the camera's position.\n\n
\n[p3d_NormalMatrix] is the upper 3x3 of the inverse transpose of the ModelViewMatrix.\nIt is used to transform the normal vector into view-space coordinates.\n \n \n\n
\n\nYou'll also need to output, to the fragment shader, the UV coordinates for the normal map.\n\n### Fragment\n\nRecall that the vertex normal was used to calculate the lighting.\nHowever, the normal map provides us with different normals to use when calculating the lighting.\nIn the fragment shader, you need to swap out the vertex normals for the normals found in the normal map.\n\n```c\n// ...\n\nuniform sampler2D p3d_Texture1;\n\n// ...\n\nin vec2 normalCoord;\n\n // ...\n\n /* Find */\n vec4 normalTex = texture(p3d_Texture1, normalCoord);\n\n // ...\n```\n\nUsing the normal map coordinates the vertex shader sent, pull out the normal from the normal map.\n\n```c\n // ...\n\n vec3 normal;\n\n // ...\n\n /* Unpack */\n normal =\n normalize\n ( normalTex.rgb\n * 2.0\n - 1.0\n );\n\n // ...\n```\n\nEarlier I showed how the normals are mapped to colors to create the normal map.\nNow this process needs to be reversed so you can get back the original normals that were baked into the map.\n\n```c\n(r, g, b) =\n ( r * 2 - 1\n , g * 2 - 1\n , b * 2 - 1\n ) =\n (x, y, z)\n```\n\nHere's the process for unpacking the normals from the normal map.\n\n```c\n // ...\n\n /* Transform */\n normal =\n normalize\n ( mat3\n ( tangent\n , binormal\n , vertexNormal\n )\n * normal\n );\n\n // ...\n```\n\nThe normals you get back from the normal map are typically in tangent space.\nThey could be in another space, however.\nFor example, Blender allows you to bake the normals in tangent, object, world, or camera space.\n\n
\n\n
\n\nTo take the normal map normal from tangent space to view pace,\nconstruct a three by three matrix using the tangent, binormal, and vertex normal vectors.\nMultiply the normal by this matrix and be sure to normalize it.\n\nAt this point, you're done.\nThe rest of the lighting calculations are the same.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [base.vert](../demonstration/shaders/vertex/base.vert)\n- [base.frag](../demonstration/shaders/fragment/base.frag)\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](cel-shading.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](deferred-rendering.md)\n"
},
{
"path": "sections/outlining.md",
"content": "[:arrow_backward:](flow-mapping.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](depth-of-field.md)\n\n# 3D Game Shaders For Beginners\n\n## Outlining\n\n
\n\n
\n\nOutlining your scene's geometry can give your game a distinctive look,\nreminiscent of comic books and cartoons.\n\n### Discontinuities\n\nThe process of outlining is the process of finding and labeling discontinuities or differences.\nEvery time you find what you consider a significant difference,\nyou mark it with your line color.\nAs you go about labeling or coloring in the differences, outlines or edges will start to form.\n\nWhere you choose to search for the discontinuities is up to you.\nIt could be the diffuse colors in your scene,\nthe normals of your models,\nthe depth buffer,\nor some other scene related data.\n\nThe demo uses the interpolated vertex positions to render the outlines.\nHowever,\na less straightforward but more typical way is to use both the\nscene's normals and depth buffer values to construct the outlines.\n\n### Vertex Positions\n\n```c\n// ...\n\nuniform sampler2D positionTexture;\n\n// ...\n```\n\nLike SSAO, you'll need the vertex positions in view space.\nReferrer back to [SSAO](ssao.md#vertex-positions) for details.\n\n### Scene Colors\n\n```c\n// ...\n\nuniform sampler2D colorTexture;\n\n// ...\n```\n\nThe demo darkens the colors of the scene where there's an outline.\nThis tends to look nicer than a constant color since it provides\nsome color variation to the edges.\n\n### Parameters\n\n```c\n // ...\n\n float minSeparation = 1.0;\n float maxSeparation = 3.0;\n float minDistance = 0.5;\n float maxDistance = 2.0;\n int size = 1;\n vec3 colorModifier = vec3(0.324, 0.063, 0.099);\n\n // ...\n```\n\nThe min and max separation parameters control the thickness of the outline\ndepending on the fragment's distance from the camera or depth.\nThe min and max distance control the significance of any changes found.\nThe `size` parameter controls the constant thickness of the line no matter the fragment's position.\nThe outline color is based on `colorModifier` and the current fragment's color.\n\n### Fragment Position\n\n```c\n // ...\n\n vec2 texSize = textureSize(colorTexture, 0).xy;\n vec2 fragCoord = gl_FragCoord.xy;\n vec2 texCoord = fragCoord / texSize;\n\n vec4 position = texture(positionTexture, texCoord);\n\n // ...\n```\n\nSample the position texture for the current fragment's position in the scene.\nRecall that the position texture is just a screen shaped quad making the UV coordinate\nthe current fragment's screen coordinate divided by the dimensions of the screen.\n\n### Fragment Depth\n\n```c\n // ...\n\n float depth =\n clamp\n ( 1.0\n - ( (far - position.y)\n / (far - near)\n )\n , 0.0\n , 1.0\n );\n\n float separation = mix(maxSeparation, minSeparation, depth);\n\n // ...\n```\n\nThe fragment's depth ranges from zero to one.\nWhen the fragment's view-space y coordinate matches the far clipping plane,\nthe depth is one.\nWhen the fragment's view-space y coordinate matches the near clipping plane,\nthe depth is zero.\nIn other words,\nthe depth ranges from zero at the near clipping plane all the way up to one at the far clipping plane.\n\n```c\n // ...\n\n float separation = mix(maxSeparation, minSeparation, depth);\n\n // ...\n```\n\nConverting the position to a depth value isn't necessary\nbut it allows you to vary the thickness of the outline based on how far\naway the fragment is from the camera.\nFar away fragments get a thinner line while nearer fragments get a thicker outline.\nThis tends to look nicer than a constant thickness since it gives depth to the outline.\n\n### Finding The Discontinuities\n\n
\n\n
\n\n\n```c\n // ...\n\n float mx = 0.0;\n\n for (int i = -size; i <= size; ++i) {\n for (int j = -size; j <= size; ++j) {\n // ...\n }\n }\n\n // ...\n```\n\nNow that you have the current fragment's position,\nloop through an i by j grid or window surrounding the current fragment.\n\n\n```c\n // ...\n\n texCoord =\n ( fragCoord\n + (vec2(i, j) * separation)\n )\n / texSize;\n\n vec4 positionTemp =\n texture\n ( positionTexture\n , texCoord\n );\n\n mx = max(mx, abs(position.y - positionTemp.y));\n\n // ...\n```\n\nWith each iteration,\nfind the biggest distance between this fragment's\nand the surrounding fragments' positions.\n\n
\n\n
\n\n```c\n // ...\n\n float diff = smoothstep(minDistance, maxDistance, mx);\n\n // ...\n```\n\nCalculate the significance of any difference discovered using the `minDistance`, `maxDistance`, and `smoothstep`.\n`smoothstep` returns values from zero to one.\nThe `minDistance` is the left-most edge.\nAny difference less than the minimum distance will be zero.\nThe `maxDistance` is the right-most edge.\nAny difference greater than the maximum distance will be one.\nFor distances between the edges,\nthe difference will be between zero and one.\nThese values are interpolated along a s-shaped curve.\n\n### Line Color\n\n```c\n // ...\n\n vec3 lineColor = texture(colorTexture, texCoord).rgb * colorModifier;\n\n // ...\n```\n\nThe line color is the current fragment color either darkened or lightened.\n\n### Fragment Color\n\n
\n\n
\n\n```c\n // ...\n\n fragColor.rgb = vec4(lineColor, diff);\n\n // ...\n```\n\nThe fragment's RGB color is the `lineColor` and its alpha channel is `diff`.\n\n### Sketchy\n\n
\n\n
\n\nFor a sketchy outline, you can distort the UV coordinates used to sample the position vectors.\n\n
\n\n
\n\n```c\n// ...\n\nuniform sampler2D noiseTexture;\n\n// ...\n```\n\nStart by creating a RGB noise texture.\nA good size is either 128 by 128 or 512 by 512.\nBe sure to blur it and make it tileable.\nThis will produce a nice wavy, inky outline.\n\n```c\n // ...\n\n float noiseScale = 10.0;\n\n // ...\n```\n\nThe `noiseScale` parameter controls how distorted the outline is.\nThe bigger the `noiseScale`, the sketchier the line.\n\n```c\n // ...\n\n vec2 fragCoord = gl_FragCoord.xy;\n\n vec2 noise = texture(noiseTexture, fragCoord / textureSize(noiseTexture, 0).xy).rb;\n noise = noise * 2.0 - 1.0;\n noise *= noiseScale;\n\n // ...\n```\n\nSample the noise texture using the current screen/fragment position and the size of the noise texture.\nSince you're distorting the UV coordinates used to sample the position vectors,\nyou'll only need two of the three color channels.\nMap the two color channels from `[0, 1]` to `[-1, 1]`.\nFinally, scale the noise by the scale chosen earlier.\n\n```c\n // ...\n\n vec2 texSize = textureSize(colorTexture, 0).xy;\n vec2 texCoord = (fragCoord - noise) / texSize;\n\n vec4 position = texture(positionTexture, texCoord);\n\n // ...\n```\n\nWhen sampling the current position,\nsubtract the noise vector from the current fragment's coordinates.\n\n
\n\n
\n\nYou could instead add it to the current fragment's coordinates\nwhich will create more of a squiggly line that loosely follows the geometry.\n\n```c\n // ...\n\n texCoord =\n (vec2(i, j) * separation + fragCoord + noise)\n / texSize;\n\n // ...\n\n vec4 positionTemp =\n texture\n ( positionTexture\n , texCoord\n );\n\n // ...\n```\n\nWhen sampling the surrounding positions inside the loop,\nadd the noise vector to the current fragment's coordinates.\nThe rest of the calculations are the same.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [base.vert](../demonstration/shaders/vertex/base.vert)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [position.frag](../demonstration/shaders/fragment/position.frag)\n- [outline.frag](../demonstration/shaders/fragment/outline.frag)\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](flow-mapping.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](depth-of-field.md)\n"
},
{
"path": "sections/pixelization.md",
"content": "[:arrow_backward:](posterization.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](sharpen.md)\n\n# 3D Game Shaders For Beginners\n\n## Pixelization\n\n
\n\n
\n\nPixelizing your 3D game can give it a interesting look and\npossibly save you time by not having to create all of the pixel art by hand.\nCombine it with the posterization for a true retro look.\n\n```c\n // ...\n\n int pixelSize = 5;\n\n // ...\n```\n\nFeel free to adjust the pixel size.\nThe bigger the pixel size, the blockier the image will be.\n\n
\n\n
\n\n```c\n // ...\n\n float x = int(gl_FragCoord.x) % pixelSize;\n float y = int(gl_FragCoord.y) % pixelSize;\n\n x = floor(pixelSize / 2.0) - x;\n y = floor(pixelSize / 2.0) - y;\n\n x = gl_FragCoord.x + x;\n y = gl_FragCoord.y + y;\n\n // ...\n```\n\nThe technique works by mapping each fragment to the center of its closest, non-overlapping\npixel-sized window.\nThese windows are laid out in a grid over the input texture.\nThe center-of-the-window fragments determine the color for the other fragments in their window.\n\n```c\n // ...\n\n fragColor = texture(colorTexture, vec2(x, y) / texSize);\n\n // ...\n```\n\nOnce you have determined the correct fragment coordinate to use,\npull its color from the input texture and assign that to the fragment color.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [pixelize.frag](../demonstration/shaders/fragment/pixelize.frag)\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](posterization.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](sharpen.md)\n"
},
{
"path": "sections/posterization.md",
"content": "[:arrow_backward:](depth-of-field.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](pixelization.md)\n\n# 3D Game Shaders For Beginners\n\n## Posterization\n\n
\n\n
\n\nPosterization or color quantization is the process of reducing the number of unique colors in an image.\nYou can use this shader to give your game a comic book or retro look.\nCombine it with [outlining](outlining.md) for a full-on cartoon art style.\n\nThere are various different ways to implement posterization.\nThis method works directly with the greyscale values and indirectly with the RGB values of the image.\nFor each fragment, it maps the RGB color to a greyscale value.\nThis greyscale value is then mapped to both its lower and upper level value.\nThe closest level to the original greyscale value is then mapped back to an RGB value\nThis new RGB value becomes the fragment color.\nI find this method produces nicer results than the more typical methods you'll find.\n\n```c\n // ...\n\n float levels = 10;\n\n // ...\n```\n\nThe `levels` parameter controls how many discrete bands or steps there are.\nThis will break up the continuous values from zero to one into chunks.\nWith four levels, `0.0` to `1.0` becomes `0.0`, `0.25`, `0.5`, `0.75`, and `1.0`.\n\n\n```c\n // ...\n\n fragColor = texture(posterizeTexture, texCoord);\n\n // ...\n```\n\nSample the current fragment's color.\n\n```c\n // ...\n\n float greyscale = max(fragColor.r, max(fragColor.g, fragColor.b));\n\n // ...\n```\n\nMap the RGB values to a greyscale value.\nIn this instance, the greyscale value is the maximum value of the R, G, and B values.\n\n```c\n // ...\n\n float lower = floor(greyscale * levels) / levels;\n float lowerDiff = abs(greyscale - lower);\n\n // ...\n```\n\nMap the greyscale value to its lower level and\nthen calculate the difference between its lower level and itself.\nFor example,\nif the greyscale value is `0.87` and there are four levels, its lower level is `0.75` and the difference is `0.12`.\n\n\n```c\n // ...\n\n float upper = ceil(greyscale * levels) / levels;\n float upperDiff = abs(upper - greyscale);\n\n // ...\n```\n\nNow calculate the upper level and the difference.\nKeeping with the example up above, the upper level is `1.0` and the difference is `0.13`.\n\n```c\n // ...\n\n float level = lowerDiff <= upperDiff ? lower : upper;\n float adjustment = level / greyscale;\n\n // ...\n```\n\nThe closest level is used to calculate the adjustment.\nThe adjustment is the ratio between the quantized and unquantized greyscale value.\nThis adjustment is used to map the quantized greyscale value back to an RGB value.\n\n\n```c\n // ...\n\n fragColor.rgb * adjustment;\n\n // ...\n```\n\nAfter multiplying `rgb` by the adjustment, `max(r, max(g, b))` will now equal the quantized greyscale value.\nThis maps the quantized greyscale value back to a red, green, and blue vector.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [posterize.frag](../demonstration/shaders/fragment/posterize.frag)\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](depth-of-field.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](pixelization.md)\n"
},
{
"path": "sections/reference-frames.md",
"content": "[:arrow_backward:](running-the-demo.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](glsl.md)\n\n# 3D Game Shaders For Beginners\n\n## Reference Frames\n\nBefore you write any shaders, you should be familiar with the following frames of reference or coordinate systems.\nAll of them boil down to what origin `(0, 0, 0)` are these coordinates currently relative to?\nOnce you know that, you can then transform them, via some matrix, to some other vector space if need be.\nTypically, when the output of some shader looks wrong, it's because of some coordinate system mix up.\n\n### Model\n\n
\n\n
\n\nThe model or object coordinate system is relative to the origin of the model.\nThis is typically set to the center of the model's geometry in a modeling program like Blender.\n\n### World\n\n
\n\n
\n\nThe world space is relative to the origin of the scene/level/universe that you've created.\n\n### View\n\n
\n\n
\n\nThe view or eye coordinate space is relative to the position of the active camera.\n\n### Clip\n\n
\n\n
\n\nThe clip space is relative to the center of the camera's film.\nAll coordinates are now homogeneous, ranging from negative one to one `(-1, 1)`.\nX and y are parallel with the camera's film and the z coordinate is the depth.\n\n
\n\n
\n\nAny vertex not within the bounds of the camera's frustum or view volume is clipped or discarded.\nYou can see this happening with the cube towards the back, clipped by the camera's far plane, and the cube off to the side.\n\n### Screen\n\n
\n\n
\n\nThe screen space is (typically) relative to the lower left corner of the screen.\nX goes from zero to the screen width.\nY goes from zero to the screen height.\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](running-the-demo.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](glsl.md)\n"
},
{
"path": "sections/render-to-texture.md",
"content": "[:arrow_backward:](glsl.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](texturing.md)\n\n# 3D Game Shaders For Beginners\n\n## Render To Texture\n\nInstead of rendering/drawing/painting directly to the screen, the example code uses a technique called \"render to texture\".\nIn order to render to a texture, you'll need to set up a framebuffer and bind a texture to it.\nMultiple textures can be bound to a single framebuffer.\n\nThe textures bound to the framebuffer hold the vector(s) returned by the fragment shader.\nTypically these vectors are color vectors `(r, g, b, a)` but they could also be position or normal vectors `(x, y, z, w)`.\nFor each bound texture, the fragment shader can output a different vector.\nFor example you could output a vertex's position and normal in a single pass.\n\nMost of the example code dealing with Panda3D involves setting up\n[framebuffer textures](https://www.panda3d.org/manual/?title=Render-to-Texture_and_Image_Postprocessing).\nTo keep things straightforward, nearly all of the fragment shaders in the example code have only one output.\nHowever, you'll want to output as much as you can each render pass to keep your frames per second (FPS) high.\n\nThere are two framebuffer texture setups found in the example code.\n\n
\n\n
\n\nThe first setup renders the mill scene into a framebuffer texture using a variety of vertex and fragment shaders.\nThis setup will go through each of the mill scene's vertexes and corresponding fragments.\n\nIn this setup, the example code performs the following.\n\n- Stores geometry data (like vertex position or normal) for later use.\n- Stores material data (like the diffuse color) for later use.\n- UV maps the various textures (diffuse, normal, shadow, etc.).\n- Calculates the ambient, diffuse, specular, and emission lighting.\n\n
\n\n
\n\nThe second setup is an orthographic camera pointed at a screen-shaped rectangle.\nThis setup will go through just the four vertexes and their corresponding fragments.\n\nIn this second setup, the example code performs the following.\n\n- Manipulates the output of another framebuffer texture.\n- Combines various framebuffer textures into one.\n\nI like to think of this second setup as using layers in GIMP, Krita, or Inkscape.\n\n
\n\n
\n\nIn the example code, you can see the output of a particular framebuffer texture\nby using the Tab key or the Shift+Tab keys.\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](glsl.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](texturing.md)\n"
},
{
"path": "sections/rim-lighting.md",
"content": "[:arrow_backward:](fresnel-factor.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](cel-shading.md)\n\n# 3D Game Shaders For Beginners\n\n## Rim Lighting\n\n
\n\n
\n\nTaking inspiration from the [fresnel factor](fresnel-factor.md),\nrim lighting targets the rim or silhouette of an object.\nWhen combined with [cel shading](cel-shading.md) and [outlining](outlining.md),\nit can really complete that cartoon look.\nYou can also use it to highlight objects in the game,\nmaking it easier for players to navigate and accomplish tasks.\n\n```c\n // ...\n\n vec3 eye = normalize(-vertexPosition.xyz);\n\n // ...\n```\n\nAs it was for the fresnel factor,\nyou'll need the eye vector.\nIf your vertex positions are in view space,\nthe eye vector is the negation of the vertex position.\n\n
\n\n
\n\n```c\n // ...\n\n float rimLightIntensity = dot(eye, normal);\n rimLightIntensity = 1.0 - rimLightIntensity;\n rimLightIntensity = max(0.0, rimLightIntensity);\n\n // ...\n```\n\nThe Intensity of the rim light ranges from zero to one.\nWhen the eye and normal vector point in the same direction,\nthe rim light intensity is zero.\nAs the two vectors start to point in different directions,\nthe rim light intensity increases\nuntil it eventually reaches one when the eye and normal become orthogonal or perpendicular to one another.\n\n
\n\n
\n\n```c\n // ...\n\n rimLightIntensity = pow(rimLightIntensity, rimLightPower);\n\n // ...\n```\n\nYou can control the falloff of the rim light using the power function.\n\n```c\n // ...\n\n rimLightIntensity = smoothstep(0.3, 0.4, rimLightIntensity)\n\n // ...\n```\n\n`step` or `smoothstep` can also be used to control the falloff.\nThis tends to look better when using [cel shading](cel-shading.md).\nYou'll learn more about these functions in later sections.\n\n```c\n // ...\n\n vec4 rimLight = rimLightIntensity * diffuse;\n rimLight.a = diffuse.a;\n\n // ...\n```\n\nWhat color you use for the rim light is up to you.\nThe demo code multiplies the diffuse light by the `rimLightIntensity`.\nThis will highlight the silhouette without overexposing it\nand without lighting any shadowed fragments.\n\n```c\n // ...\n\n vec4 outputColor = vec4(0.0);\n outputColor.a = diffuseColor.a;\n outputColor.rgb =\n ambient.rgb\n + diffuse.rgb\n + specular.rgb\n + rimLight.rgb\n + emission.rgb;\n\n // ...\n```\n\nAfter you've calculated the rim light,\nadd it to the ambient, diffuse, specular, and emission lights.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [base.frag](../demonstration/shaders/fragment/base.frag)\n\n## Copyright\n\n(C) 2020 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](fresnel-factor.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](cel-shading.md)\n"
},
{
"path": "sections/running-the-demo.md",
"content": "[:arrow_backward:](building-the-demo.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](reference-frames.md)\n\n# 3D Game Shaders For Beginners\n\n## Running The Demo\n\n
\n\n
\n\nAfter you've built the example code, you can now run the executable or demo.\n\n```bash\n./3d-game-shaders-for-beginners\n```\n\nHere's how you run it on Linux or Mac.\n\n```bash\n3d-game-shaders-for-beginners.exe\n```\n\nHere's how you run it on Windows.\n\n### Demo Controls\n\nThe demo comes with both keyboard and mouse controls to move the camera around,\ntoggle on and off the different effects,\nadjust the fog,\nand view the various different framebuffer textures.\n\n#### Mouse\n\nYou can rotate the scene around by holding down the Left Mouse button and dragging.\nHold down the Right Mouse button and drag to move up, down, left, and/or right.\nTo zoom in, roll the Mouse Wheel forward.\nTo zoom out, roll the Mouse Wheel backward.\n\nYou can also change the focus point using the mouse.\nTo change the focus point,\nclick anywhere on the scene using the Middle Mouse button.\n\n#### Keyboard\n\n- w to rotate the scene down.\n- a to rotate the scene clockwise.\n- s to rotate the scene up.\n- d to rotate the scene counterclockwise.\n- z to zoom in to the scene.\n- x to zoom out of the scene.\n- ⬅ to move left.\n- ➡ to move right.\n- ⬆ to move up.\n- ⬇ to move down.\n\n\n\n- 1 to show midday.\n- 2 to show midnight.\n\n\n\n- Delete to toggle the sound.\n- 3 to toggle fresnel.\n- 4 to toggle rim lighting.\n- 5 to toggle particles.\n- 6 to toggle motion blur.\n- 7 to toggle Kuwahara filtering.\n- 8 to toggle cel shading.\n- 9 to toggle lookup table processing.\n- 0 to toggle between Phong and Blinn-Phong.\n- y to toggle SSAO.\n- u to toggle outlining.\n- i to toggle bloom.\n- o to toggle normal mapping.\n- p to toggle fog.\n- h to toggle depth of field.\n- j to toggle posterization.\n- k to toggle pixelization.\n- l to toggle sharpen.\n- n to toggle film grain.\n- m to toggle screen space reflection.\n- , to toggle screen space refraction.\n- . to toggle flow mapping.\n- / to toggle the sun animation.\n- \\\\ to toggle chromatic aberration.\n\n\n\n- r to reset the scene.\n\n\n\n- \\[ to decrease the fog near distance.\n- Shift+\\[ to increase the fog near distance.\n- ] to increase the fog far distance.\n- Shift+] to decrease the fog far distance.\n\n\n\n- Shift+- to decrease the amount of foam.\n- - to increase the amount of foam.\n\n\n\n- Shift+= to decrease the relative index of refraction.\n- = to increase the relative index of refraction.\n\n\n\n- Tab to move forward through the framebuffer textures.\n- Shift+Tab to move backward through the framebuffer textures.\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](building-the-demo.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](reference-frames.md)\n"
},
{
"path": "sections/screen-space-reflection.md",
"content": "[:arrow_backward:](chromatic-aberration.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](screen-space-refraction.md)\n\n# 3D Game Shaders For Beginners\n\n## Screen Space Reflection (SSR)\n\n
\n\n
\n\nAdding reflections can really ground your scene.\nWet and shiny objects spring to life as nothing makes\nsomething look wet or shiny quite like reflections.\nWith reflections, you can really sell the illusion of water and metallic objects.\n\nIn the [lighting](lighting.md) section, you simulated the reflected, mirror-like image of the light source.\nThis was the process of rendering the specular reflection.\nRecall that the specular light was computed using the reflected light direction.\nSimilarly, using screen space reflection or SSR, you can simulate the reflection of\nother objects in the scene instead of just the light source.\nInstead of the light ray coming from the source and bouncing off into the camera,\nthe light ray comes from some object in the scene and bounces off into the camera.\n\nSSR works by reflecting the screen image onto itself using only itself.\nCompare this to cube mapping which uses six screens or textures.\nIn cube mapping, you reflect a ray from some point in your scene\nto some point on the inside of a cube surrounding your scene.\nIn SSR, you reflect a ray from some point on your screen to some other point on your screen.\nBy reflecting your screen onto itself, you can create the illusion of reflection.\nThis illusion holds for the most part but SSR does fail in some cases as you'll see.\n\n### Ray Marching\n\nScreen space reflection uses a technique known as ray marching to determine the reflection for each fragment.\nRay marching is the process of iteratively extending or contracting the length or magnitude of some vector\nin order to probe or sample some space for information.\nThe ray in screen space reflection is the position vector reflected about the normal.\n\nIntuitively, a light ray hits some point in the scene,\nbounces off,\ntravels in the opposite direction of the reflected position vector,\nbounces off the current fragment,\ntravels in the opposite direction of the position vector,\nand hits the camera lens allowing you to see the color of some point in the scene reflected in the current fragment.\nSSR is the process of tracing the light ray's path in reverse.\nIt tries to find the reflected point the light ray bounced off of and hit the current fragment.\nWith each iteration,\nthe algorithm samples the scene's positions or depths,\nalong the reflection ray,\nasking each time if the ray intersected with the scene's geometry.\nIf there is an intersection,\nthat position in the scene is a potential candidate for being reflected by the current fragment.\n\nIdeally there would be some analytical method for determining the first intersection point exactly.\nThis first intersection point is the only valid point to reflect in the current fragment.\nInstead, this method is more like a game of battleship.\nYou can't see the intersections (if there are any) so you start at the base of the reflection ray and call out coordinates\nas you travel in the direction of the reflection.\nWith each call, you get back an answer of whether or not you hit something.\nIf you do hit something,\nyou try points around that area hoping to find the exact point of intersection.\n\n
\n\n
\n\nHere you see ray marching being used to calculate each fragment's reflected point.\nThe vertex normal is the bright green arrow,\nthe position vector is the bright blue arrow,\nand the bright red vector is the reflection ray marching through view space.\n\n### Vertex Positions\n\nLike SSAO, you'll need the vertex positions in view space.\nReferrer back to [SSAO](ssao.md#vertex-positions) for details.\n\n### Vertex Normals\n\nTo compute the reflections, you'll need the vertex normals in view space.\nReferrer back to [SSAO](ssao.md#vertex-normals) for details.\n\n
\n\n
\n\nHere you see SSR using the normal mapped normals instead of the vertex normals.\nNotice how the reflection follows the ripples in the water versus the more mirror\nlike reflection shown earlier.\n\nTo use the normal maps instead,\nyou'll need to transform the normal mapped normals from tangent space to view space\njust like you did in the lighting calculations.\nYou can see this being done in [normal.frag](../demonstration/shaders/fragment/normal.frag).\n\n### Position Transformations\n\n
\n\n
\n\nJust like\n[SSAO](ssao.md),\nSSR goes back and forth between the screen and view space.\nYou'll need the camera lens' projection matrix to transform points in view space to clip space.\nFrom clip space, you'll have to transform the points again to UV space.\nOnce in UV space,\nyou can sample a vertex/fragment position from the scene\nwhich will be the closest position in the scene to your sample.\nThis is the _screen space_ part in _screen space reflection_\nsince the \"screen\" is a texture UV mapped over a screen shaped rectangle.\n\n### Reflected UV Coordinates\n\nThere are a few ways you can implement SSR.\nThe example code starts the reflection process by computing a reflected UV coordinate for each screen fragment.\nYou could skip this part and go straight to computing the reflected color instead, using the final rendering of the scene.\n\nRecall that UV coordinates range from zero to one for both U and V.\nThe screen is just a 2D texture UV mapped over a screen-sized rectangle.\nKnowing this, the example code doesn't actually need the final rendering of the scene\nto compute the reflections.\nIt can instead calculate what UV coordinate each screen pixel will eventually use.\nThese calculated UV coordinates can be saved to a framebuffer texture\nand used later when the scene has been rendered.\n\n
\n\n
\n\nHere you see the reflected UV coordinates.\nWithout even rendering the scene yet,\nyou can get a good feel for what the reflections will look like.\n\n```c\n//...\n\nuniform mat4 lensProjection;\n\nuniform sampler2D positionTexture;\nuniform sampler2D normalTexture;\n\n//...\n```\n\nYou'll need the camera lens' projection matrix as well as the interpolated vertex positions and normals in view space.\n\n```c\n // ...\n\n float maxDistance = 15;\n float resolution = 0.3;\n int steps = 10;\n float thickness = 0.5;\n\n // ...\n```\n\nLike the other effects, SSR has a few parameters you can adjust.\nDepending on the complexity of the scene, it may take you awhile to find the right settings.\nGetting screen space reflections to look just right tends to be difficult when reflecting complex geometry.\n\nThe `maxDistance` parameter controls how far a fragment can reflect.\nIn other words, it controls the maximum length or magnitude of the reflection ray.\n\nThe `resolution` parameter controls how many fragments are skipped while traveling or marching the reflection ray during the first pass.\nThis first pass is to find a point along the ray's direction where the ray enters or goes behind some geometry in the scene.\nThink of this first pass as the rough pass.\nNote that the `resolution` ranges from zero to one.\nZero will result in no reflections while one will travel fragment-by-fragment along the ray's direction.\nA `resolution` of one can slow down your FPS considerably especially with a large `maxDistance`.\n\nThe `steps` parameter controls how many iterations occur during the second pass.\nThis second pass is to find the exact point along the reflection ray's direction\nwhere the ray immediately hits or intersects with some geometry in the scene.\nThink of this second pass as the refinement pass.\n\nThe `thickness` controls the cutoff between what counts as a possible reflection hit and what does not.\nIdeally, you'd like to have the ray immediately stop at some camera-captured position or depth in the scene.\nThis would be the exact point where the light ray bounced off, hit your current fragment, and then bounced off into the camera.\nUnfortunately the calculations are not always that precise so `thickness` provides some wiggle room or tolerance.\nYou'll want the thickness to be as small as possible—just a short distance beyond a sampled position or depth.\n\n
\n\n
\n\nYou'll find that as the thickness gets larger, the reflections tend to smear in places.\n\n
\n\n
\n\nGoing in the other direction, as the thickness gets smaller,\nthe reflections become noisy with tiny little holes and narrow gaps.\n\n```c\n // ...\n\n vec2 texSize = textureSize(positionTexture, 0).xy;\n vec2 texCoord = gl_FragCoord.xy / texSize;\n\n vec4 positionFrom = texture(positionTexture, texCoord);\n vec3 unitPositionFrom = normalize(positionFrom.xyz);\n vec3 normal = normalize(texture(normalTexture, texCoord).xyz);\n vec3 pivot = normalize(reflect(unitPositionFrom, normal));\n\n // ...\n```\n\nGather the current fragment's position, normal, and reflection about the normal.\n`positionFrom` is a vector from the camera position to the current fragment position.\n`normal` is a vector pointing in the direction of the interpolated vertex normal for the current fragment.\n`pivot` is the reflection ray or vector pointing in the reflected direction of the `positionFrom` vector.\nIt currently has a length or magnitude of one.\n\n\n```c\n // ...\n\n vec4 startView = vec4(positionFrom.xyz + (pivot * 0), 1);\n vec4 endView = vec4(positionFrom.xyz + (pivot * maxDistance), 1);\n\n // ...\n```\n\nCalculate the start and end point of the reflection ray in view space.\n\n```c\n // ...\n\n vec4 startFrag = startView;\n // Project to screen space.\n startFrag = lensProjection * startFrag;\n // Perform the perspective divide.\n startFrag.xyz /= startFrag.w;\n // Convert the screen-space XY coordinates to UV coordinates.\n startFrag.xy = startFrag.xy * 0.5 + 0.5;\n // Convert the UV coordinates to fragment/pixel coordnates.\n startFrag.xy *= texSize;\n\n vec4 endFrag = endView;\n endFrag = lensProjection * endFrag;\n endFrag.xyz /= endFrag.w;\n endFrag.xy = endFrag.xy * 0.5 + 0.5;\n endFrag.xy *= texSize;\n\n // ...\n```\n\nProject or transform these start and end points from view space to screen space.\nThese points are now fragment positions which correspond to pixel positions on the screen.\nNow that you know where the ray starts and ends on the screen, you can travel or march along its direction in screen space.\nThink of the ray as a line drawn on the screen.\nYou'll travel along this line using it to sample the fragment positions stored in the position framebuffer texture.\n\n
\n\n
\n\nNote that you could march the ray through view space\nbut this may under or over sample scene positions found in the position framebuffer texture.\nRecall that the position framebuffer texture is the size and shape of the screen.\nEvery screen fragment or pixel corresponds to some position captured by the camera.\nA reflection ray may travel a long distance in view space, but in screen space, it may only travel through a few pixels.\nYou can only sample the screen's pixels for positions\nso it is inefficient to potentially sample the same pixels over and over again while marching in view space.\nBy marching in screen space, you'll more efficiently sample the fragments or pixels the ray actually occupies or covers.\n\n```c\n // ...\n\n vec2 frag = startFrag.xy;\n uv.xy = frag / texSize;\n\n // ...\n```\n\nThe first pass will begin at the starting fragment position of the reflection ray.\nConvert the fragment position to a UV coordinate by dividing the fragment's coordinates by the position texture's dimensions.\n\n```c\n // ...\n\n float deltaX = endFrag.x - startFrag.x;\n float deltaY = endFrag.y - startFrag.y;\n\n // ...\n```\n\nCalculate the delta or difference between the X and Y coordinates of the end and start fragments.\nThis will be how many pixels the ray line occupies in the X and Y dimension of the screen.\n\n
\n\n
\n\n```c\n // ...\n\n float useX = abs(deltaX) >= abs(deltaY) ? 1 : 0;\n float delta = mix(abs(deltaY), abs(deltaX), useX) * clamp(resolution, 0, 1);\n\n // ...\n```\n\nTo handle all of the various different ways (vertical, horizontal, diagonal, etc.) the line can be oriented,\nyou'll need to keep track of and use the larger difference.\nThe larger difference will help you determine\nhow much to travel in the X and Y direction each iteration,\nhow many iterations are needed to travel the entire line,\nand what percentage of the line does the current position represent.\n\n`useX` is either one or zero.\nIt is used to pick the X or Y dimension depending on which delta is bigger.\n`delta` is the larger delta of the two X and Y deltas.\nIt is used to determine how much to march in either dimension each iteration and how many iterations to take during the first pass.\n\n```c\n // ...\n\n vec2 increment = vec2(deltaX, deltaY) / max(delta, 0.001);\n\n // ...\n```\n\nCalculate how much to increment the X and Y position by using the larger of the two deltas.\nIf the two deltas are the same, each will increment by one each iteration.\nIf one delta is larger than the other, the larger delta will increment by one while the smaller one will increment by less than one.\nThis assumes the `resolution` is one.\nIf the resolution is less than one, the algorithm will skip over fragments.\n\n```c\nstartFrag = ( 1, 4)\nendFrag = (10, 14)\n\ndeltaX = (10 - 1) = 9\ndeltaY = (14 - 4) = 10\n\nresolution = 0.5\n\ndelta = 10 * 0.5 = 5\n\nincrement = (deltaX, deltaY) / delta\n = ( 9, 10) / 5\n = ( 9 / 5, 2)\n```\n\nFor example, say the `resolution` is 0.5.\nThe larger dimension will increment by two fragments instead of one.\n\n\n```c\n // ...\n\n float search0 = 0;\n float search1 = 0;\n\n // ...\n```\n\nTo move from the start fragment to the end fragment, the algorithm uses linear interpolation.\n\n```c\ncurrent position x = (start x) * (1 - search1) + (end x) * search1;\ncurrent position y = (start y) * (1 - search1) + (end y) * search1;\n```\n\n`search1` ranges from zero to one.\nWhen `search1` is zero, the current position is the start fragment.\nWhen `search1` is one, the current position is the end fragment.\nFor any other value, the current position is somewhere between the start and end fragment.\n\n`search0` is used to remember the last position on the line where the ray missed or didn't intersect with any geometry.\nThe algorithm will later use `search0` in the second pass to help refine the point at which the ray touches the scene's geometry.\n\n```c\n // ...\n\n int hit0 = 0;\n int hit1 = 0;\n\n // ...\n```\n\n`hit0` indicates there was an intersection during the first pass.\n`hit1` indicates there was an intersection during the second pass.\n\n```c\n // ...\n\n float viewDistance = startView.y;\n float depth = thickness;\n\n // ...\n```\n\nThe `viewDistance` value is how far away from the camera the current point on the ray is.\nRecall that for Panda3D, the Y dimension goes in and out of the screen in view space.\nFor other systems, the Z dimension goes in and out of the screen in view space.\nIn any case, `viewDistance` is how far away from the camera the ray currently is.\nNote that if you use the depth buffer, instead of the vertex positions in view space, the `viewDistance` would be the Z depth.\n\nMake sure not to confuse the `viewDistance` value with the Y dimension of the line being traveled across the screen.\nThe `viewDistance` goes from the camera into scene while the Y dimension of the line travels up or down the screen.\n\nThe `depth` is the view distance difference between the current ray point and scene position.\nIt tells you how far behind or in front of the scene the ray currently is.\nRemember that the scene positions are the interpolated vertex positions stored in the position framebuffer texture.\n\n```c\n // ...\n\n for (i = 0; i < int(delta); ++i) {\n\n // ...\n```\n\nYou can now begin the first pass.\nThe first pass runs while `i` is less than the `delta` value.\nWhen `i` reaches `delta`, the algorithm has traveled the entire length of the line.\nRemember that `delta` is the larger of the two X and Y deltas.\n\n
\n\n
\n\n```c\n // ...\n\n frag += increment;\n uv.xy = frag / texSize;\n positionTo = texture(positionTexture, uv.xy);\n\n // ...\n```\n\nAdvance the current fragment position closer to the end fragment.\nUse this new fragment position to look up a scene position stored in the position framebuffer texture.\n\n```c\n // ...\n\n search1 =\n mix\n ( (frag.y - startFrag.y) / deltaY\n , (frag.x - startFrag.x) / deltaX\n , useX\n );\n\n // ...\n```\n\nCalculate the percentage or portion of the line the current fragment represents.\nIf `useX` is zero, use the Y dimension of the line.\nIf `useX` is one, use the X dimension of the line.\n\nWhen `frag` equals `startFrag`,\n`search1` equals zero since `frag - startFrag` is zero.\nWhen `frag` equals `endFrag`,\n`search1` is one since `frag - startFrag` equals `delta`.\n\n`search1` is the percentage or portion of the line the current position represents.\nYou'll need this to interpolate between the ray's view-space start and end distances from the camera.\n\n```c\n // ...\n\n viewDistance = (startView.y * endView.y) / mix(endView.y, startView.y, search1);\n\n // ...\n```\n\nUsing `search1`,\ninterpolate the view distance (distance from the camera in view space) for the current position you're at on the reflection ray.\n\n```c\n// Incorrect.\nviewDistance = mix(startView.y, endView.y, search1);\n\n// Correct.\nviewDistance = (startView.y * endView.y) / mix(endView.y, startView.y, search1);\n```\n\nYou may be tempted to just interpolate between the view distances of the start and end view-space positions\nbut this will give you the wrong view distance for the current position on the reflection ray.\nInstead, you'll need to perform\n[perspective-correct interpolation](https://www.comp.nus.edu.sg/~lowkl/publications/lowk_persp_interp_techrep.pdf)\nwhich you see here.\n\n```c\n // ...\n\n depth = viewDistance - positionTo.y;\n\n // ...\n```\n\nCalculate the difference between the ray's view distance at this point and the sampled view distance of the scene at this point.\n\n```c\n // ...\n\n if (depth > 0 && depth < thickness) {\n hit0 = 1;\n break;\n } else {\n search0 = search1;\n }\n\n // ...\n```\n\nIf the difference is between zero and the thickness,\nthis is a hit.\nSet `hit0` to one and exit the first pass.\nIf the difference is not between zero and the thickness,\nthis is a miss.\nSet `search0` to equal `search1` to remember this position as the last known miss.\nContinue marching the ray towards the end fragment.\n\n```c\n // ...\n\n search1 = search0 + ((search1 - search0) / 2);\n\n // ...\n```\n\nAt this point you have finished the first pass.\nSet the `search1` position to be halfway between the position of the last miss and the position of the last hit.\n\n```c\n // ...\n\n steps *= hit0;\n\n for (i = 0; i < steps; ++i) {\n\n // ...\n```\n\nYou can now begin the second pass.\nIf the reflection ray didn't hit anything in the first pass, skip the second pass.\n\n```c\n // ...\n\n frag = mix(startFrag.xy, endFrag.xy, search1);\n uv.xy = frag / texSize;\n positionTo = texture(positionTexture, uv.xy);\n\n // ...\n```\n\nAs you did in the first pass,\nuse the current position on the ray line to sample a position from the scene.\n\n```c\n // ...\n\n viewDistance = (startView.y * endView.y) / mix(endView.y, startView.y, search1);\n depth = viewDistance - positionTo.y;\n\n // ...\n```\n\nInterpolate the view distance for the current ray line position\nand calculate the camera distance difference between the ray at this point and the scene.\n\n```c\n // ...\n\n if (depth > 0 && depth < thickness) {\n hit1 = 1;\n search1 = search0 + ((search1 - search0) / 2);\n } else {\n float temp = search1;\n search1 = search1 + ((search1 - search0) / 2);\n search0 = temp;\n }\n\n // ...\n```\n\nIf the depth is within bounds, this is a hit.\nSet `hit1` to one and set `search1` to be halfway between the last known miss position and this current hit position.\nIf the depth is not within bounds, this is a miss.\nSet `search1` to be halfway between this current miss position and the last known hit position.\nMove `search0` to this current miss position.\nContinue this back and forth search while `i` is less than `steps`.\n\n```c\n // ...\n\n float visibility =\n hit1\n\n // ...\n```\n\nYou're now done with the second and final pass but before you can output the reflected UV coordinates,\nyou'll need to calculate the `visibility` of the reflection.\nThe `visibility` ranges from zero to one.\nIf there wasn't a hit in the second pass, the `visibility` is zero.\n\n```c\n // ...\n\n * positionTo.w\n\n // ...\n```\n\nIf the reflected scene position's alpha or `w` component is zero,\nthe `visibility` is zero.\nNote that if `w` is zero, there was no scene position at that point.\n\n
\n\n
\n\n```c\n // ...\n\n * ( 1\n - max\n ( dot(-unitPositionFrom, pivot)\n , 0\n )\n )\n\n // ...\n```\n\nOne of the ways in which screen space reflection can fail is when the reflection ray points in the general direction of the camera.\nIf the reflection ray points towards the camera and hits something, it's most likely hitting the back side of something facing away\nfrom the camera.\n\nTo handle this failure case, you'll need to gradually fade out the reflection based\non how much the reflection vector points to the camera's position.\nIf the reflection vector points directly in the opposite direction of the position vector,\nthe visibility is zero.\nAny other direction results in the visibility being greater than zero.\n\nRemember to normalize both vectors when taking the dot product.\n`unitPositionFrom` is the normalized position vector.\nIt has a length or magnitude of one.\n\n```c\n // ...\n\n * ( 1\n - clamp\n ( depth / thickness\n , 0\n , 1\n )\n )\n\n // ...\n```\n\nAs you sample scene positions along the reflection ray,\nyou're hoping to find the exact point where the reflection ray first intersects with the scene's geometry.\nUnfortunately, you may not find this particular point.\nFade out the reflection the further it is from the intersection point you did find.\n\n```c\n // ...\n\n * ( 1\n - clamp\n ( length(positionTo - positionFrom)\n / maxDistance\n , 0\n , 1\n )\n )\n\n // ...\n```\n\nFade out the reflection based on how far way the reflected point is from the initial starting point.\nThis will fade out the reflection instead of it ending abruptly as it reaches `maxDistance`.\n\n
\n\n
\n\n\n```c\n // ...\n\n * (uv.x < 0 || uv.x > 1 ? 0 : 1)\n * (uv.y < 0 || uv.y > 1 ? 0 : 1);\n\n // ...\n```\n\nIf the reflected UV coordinates are out of bounds, set the `visibility` to zero.\nThis occurs when the reflection ray travels outside the camera's frustum.\n\n```c\n visibility = clamp(visibility, 0, 1);\n\n uv.ba = vec2(visibility);\n```\n\nSet the blue and alpha component to the visibility as the UV coordinates only need the RG or XY components of the final vector.\n\n```c\n // ...\n\n fragColor = uv;\n\n // ...\n```\n\nThe final fragment color is the reflected UV coordinates and the visibility.\n\n### Specular Map\n\n
\n\n
\n\nIn addition to the reflected UV coordinates, you'll also need a specular map.\nThe example code creates one using the fragment's material specular properties.\n\n```c\n// ...\n\n#define MAX_SHININESS 127.75\n\nuniform struct\n { vec3 specular\n ; float shininess\n ;\n } p3d_Material;\n\nout vec4 fragColor;\n\nvoid main() {\n fragColor =\n vec4\n ( p3d_Material.specular\n , clamp(p3d_Material.shininess / MAX_SHININESS, 0, 1)\n );\n}\n```\n\nThe specular fragment shader is quite simple.\nUsing the fragment's material,\nthe shader outputs the specular color and uses the alpha channel for the shininess.\nThe shininess is mapped to a range of zero to one.\nIn Blender, the maximum specular hardness or shininess is 511.\nWhen exporting from Blender to Panda3D, 511 is exported as 127.75.\nFeel free to adjust the shininess to range of zero to one however you see fit for your particular stack.\n\nThe example code generates a specular map from the material specular properties but you\ncould create one in GIMP, for example, and attach that as a texture to your 3D model.\nFor instance,\nsay your 3D treasure chest has shiny brackets on it but nothing else should reflect the environment.\nYou can paint the brackets some shade of gray and the rest of the treasure chest black.\nThis will mask off the brackets, allowing your shader to render the reflections on only the brackets\nand nothing else.\n\n### Scene Colors\n\n
\n\n
\n\nYou'll need to render the parts of the scene you wish to reflect and store this in a framebuffer texture.\nThis is typically just the scene without any reflections.\n\n### Reflected Scene Colors\n\n
\n\n
\n\nHere you see the reflected colors saved to a framebuffer texture.\n\n```c\n// ...\n\nuniform sampler2D uvTexture;\nuniform sampler2D colorTexture;\n\n// ...\n```\n\nOnce you have the reflected UV coordinates, looking up the reflected colors is fairly easy.\nYou'll need the reflected UV coordinates texture and the color texture containing the colors you wish to reflect.\n\n```c\n // ...\n\n vec2 texSize = textureSize(uvTexture, 0).xy;\n vec2 texCoord = gl_FragCoord.xy / texSize;\n\n vec4 uv = texture(uvTexture, texCoord);\n vec4 color = texture(colorTexture, uv.xy);\n\n // ...\n```\n\nUsing the UV coordinates for the current fragment, look up the reflected color.\n\n```c\n // ...\n\n float alpha = clamp(uv.b, 0, 1);\n\n // ...\n```\n\nRecall that the reflected UV texture stored the visibility in the B or blue component.\nThis is the alpha channel for the reflected colors framebuffer texture.\n\n```c\n // ...\n\n fragColor = vec4(mix(vec3(0), color.rgb, alpha), alpha);\n\n // ...\n```\n\nThe fragment color is a mix between no reflection and the reflected color based on the visibility.\nThe visibility was computed during the reflected UV coordinates step.\n\n### Blurred Reflected Scene Colors\n\n
\n\n
\n\nNow blur the reflected scene colors and store this in a framebuffer texture.\nThe blurring is done using a box blur.\nRefer to the [SSAO blurring](ssao.md#blurring) step for details.\n\nThe blurred reflected colors are used for surfaces that have a less than mirror like finish.\nThese surfaces have tiny little hills and valleys that tend to diffuse or blur the reflection.\nI'll cover this more during the roughness calculation.\n\n### Reflections\n\n```c\n// ...\n\nuniform sampler2D colorTexture;\nuniform sampler2D colorBlurTexture;\nuniform sampler2D specularTexture;\n\n// ...\n```\n\nTo generate the final reflections, you'll need the three framebuffer textures computed earlier.\nYou'll need the reflected colors, the blurred reflected colors, and the specular map.\n\n```c\n // ...\n\n vec4 specular = texture(specularTexture, texCoord);\n vec4 color = texture(colorTexture, texCoord);\n vec4 colorBlur = texture(colorBlurTexture, texCoord);\n\n // ...\n```\n\nLook up the specular amount and shininess, the reflected scene color, and the blurred reflected scene color.\n\n```c\n // ...\n\n float specularAmount = dot(specular.rgb, vec3(1)) / 3;\n\n if (specularAmount <= 0) { fragColor = vec4(0); return; }\n\n // ...\n```\n\nMap the specular color to a greyscale value.\nIf the specular amount is none, set the frag color to nothing and return.\n\nLater on, you'll multiply the final reflection color by the specular amount.\nMultiplying by the specular amount allows you to control how much a material reflects its environment\nsimply by brightening or darkening the greyscale value in the specular map.\n\n```c\n dot(specular.rgb, vec3(1)) == (specular.r + specular.g + specular.b);\n```\n\nUsing the dot product to produce the greyscale value is just a short way of summing the three color components.\n\n```c\n // ...\n\n float roughness = 1 - min(specular.a, 1);\n\n // ...\n```\n\nCalculate the roughness based on the shininess value set during the specular map step.\nRecall that the shininess value was saved in the alpha channel of the specular map.\nThe shininess determined how spread out or blurred the specular reflection was.\nSimilarly, the `roughness` determines how blurred the reflection is.\nA roughness of one will produce the blurred reflection color.\nA roughness of zero will produce the non-blurred reflection color.\nDoing it this way allows you to control how blurred the reflection is just by changing\nthe material's shininess value.\n\nThe example code generates a roughness map from the material specular properties but you\ncould create one in GIMP, for example, and attach that as a texture to your 3D model.\nFor instance, say you have a tiled floor that has polished tiles and scratched up tiles.\nThe polished tiles could be painted a more translucent white while the scratched up tiles could be painted a more opaque white.\nThe more translucent/transparent the greyscale value, the more the shader will use the blurred reflected color.\nThe scratched tiles will have a blurry reflection while the polished tiles will have a mirror like reflection.\n\n```c\n // ...\n\n fragColor = mix(color, colorBlur, roughness) * specularAmount;\n\n // ...\n```\n\nMix the reflected color and blurred reflected color based on the roughness.\nMultiply that vector by the specular amount and then set that value as the fragment color.\n\nThe reflection color is a mix between the reflected scene color and the blurred reflected scene color based on the roughness.\nA high roughness will produce a blurry reflection meaning the surface is rough.\nA low roughness will produce a clear reflection meaning the surface is smooth.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [base.vert](../demonstration/shaders/vertex/base.vert)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [position.frag](../demonstration/shaders/fragment/position.frag)\n- [normal.frag](../demonstration/shaders/fragment/normal.frag)\n- [material-specular.frag](../demonstration/shaders/fragment/material-specular.frag)\n- [screen-space-reflection.frag](../demonstration/shaders/fragment/screen-space-reflection.frag)\n- [reflection-color.frag](../demonstration/shaders/fragment/reflection-color.frag)\n- [reflection.frag](../demonstration/shaders/fragment/reflection.frag)\n- [box-blur.frag](../demonstration/shaders/fragment/box-blur.frag)\n- [base-combine.frag](../demonstration/shaders/fragment/base-combine.frag)\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](chromatic-aberration.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](screen-space-refraction.md)\n"
},
{
"path": "sections/screen-space-refraction.md",
"content": "[:arrow_backward:](screen-space-reflection.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](foam.md)\n\n# 3D Game Shaders For Beginners\n\n## Screen Space Refraction (SSR)\n\n
\n\n
\n\nScreen space refraction,\nmuch like\n[screen space reflection](screen-space-reflection.md),\nadds a touch of realism you can't find anywhere else.\nGlass, plastic, water, and other transparent/translucent materials spring to life.\n\n[Screen space reflection](screen-space-reflection.md)\nand screen space refraction work almost identically expect for one major difference.\nInstead of using the reflected vector, screen space refraction uses the\n[refracted vector](http://asawicki.info/news_1301_reflect_and_refract_functions.html).\nIt's a slight change in code but a big difference visually.\n\n### Vertex Positions\n\nLike SSAO, you'll need the vertex positions in view space.\nReferrer back to [SSAO](ssao.md#vertex-positions) for details.\n\nHowever,\nunlike SSAO,\nyou'll need the scene's vertex positions with and without the refractive objects.\nRefractive surfaces are translucent, meaning you can see through them.\nSince you can see through them, you'll need the vertex positions behind the refractive surface.\nHaving both the foreground and background vertex positions will allow you to calculate\nUV coordinates and depth.\n\n### Vertex Normals\n\nTo compute the refractions, you'll need the scene's foreground vertex normals in view space.\nThe background vertex normals aren't needed unless you need to incorporate the background surface detail\nwhile calculating the refracted UV coordinates and distances.\nReferrer back to [SSAO](ssao.md#vertex-normals) for details.\n\n
\n\n
\n\nHere you see the water refracting the light with and without normal maps.\nIf available, be sure to use the normal mapped normals instead of the vertex normals.\nThe smoother and flatter the surface, the harder it is to tell the light is being refracted.\nThere will be some distortion but not enough to make it worthwhile.\n\nTo use the normal maps instead,\nyou'll need to transform the normal mapped normals from tangent space to view space\njust like you did in the [lighting](normal-mapping.md#fragment) calculations.\nYou can see this being done in [normal.frag](../demonstration/shaders/fragment/normal.frag).\n\n### Position Transformations\n\n
\n\n
\n\nJust like\n[SSAO](ssao.md) and [screen space reflection](screen-space-reflection.md),\nscreen space refraction goes back and forth between the screen and view space.\nYou'll need the camera lens' projection matrix to transform points in view space to clip space.\nFrom clip space, you'll have to transform the points again to UV space.\nOnce in UV space,\nyou can sample a vertex/fragment position from the scene\nwhich will be the closest position in the scene to your sample.\nThis is the _screen space_ part in _screen space refraction_\nsince the \"screen\" is a texture UV mapped over a screen shaped rectangle.\n\n### Refracted UV Coordinates\n\nRecall that UV coordinates range from zero to one for both U and V.\nThe screen is just a 2D texture UV mapped over a screen-sized rectangle.\nKnowing this, the example code doesn't actually need the final rendering of the scene\nto compute the refraction.\nIt can instead calculate what UV coordinate each screen pixel will eventually use.\nThese calculated UV coordinates can be saved to a framebuffer texture\nand used later when the scene has been rendered.\n\nThe process of refracting the UV coordinates is very similar to the process of\n[reflecting the UV coordinates](screen-space-reflection.md#reflected-uv-coordinates).\nBelow are the adjustments you'll need to turn reflection into refraction.\n\n```c\n// ...\n\nuniform sampler2D positionFromTexture;\nuniform sampler2D positionToTexture;\nuniform sampler2D normalFromTexture;\n\n// ...\n```\n\nReflection only deals with what is in front of the reflective surface.\nRefraction, however, deals with what is behind the refractive surface.\nTo accommodate this, you'll need both the vertex positions of the scene\nwith the refracting surfaces taken out and the vertex positions of the scene\nwith the refracting surfaces left in.\n\n
\n\n
\n\n`positionFromTexture` are the scene's vertex positions with the refracting surfaces left in.\n`positionToTexture` are the scene's vertex positions with the refracting surfaces taken out.\n`normalFromTexture` are the scene's vertex normals with the refraction surfaces left in.\nThere's no need for the vertex normals behind the refractive surfaces unless\nyou want to incorporate the surface detail for the background geometry.\n\n```c\n// ...\n\nuniform vec2 rior;\n\n// ...\n```\n\nRefraction has one more adjustable parameter than reflection.\n`rior` is the relative index of refraction or relative refractive index.\nIt is the ratio of the refraction indexes of two mediums.\nSo for example, going from water to air is `1 / 1.33 ≈ 0.75`.\nThe numerator is the refractive index of the medium the light is leaving and\nthe denominator is the refractive index of the medium the light is entering.\nAn `rior` of one means the light passes right through without being refracted or bent.\nAs `rior` grows, the refraction will become a\n[reflection](https://en.wikipedia.org/wiki/Total_internal_reflection).\n\nThere's no requirement that `rior` must adhere to the real world.\nThe demo uses `1.05`.\nThis is completely unrealistic (light does not travel faster through water than air)\nbut the realistic setting produced too many artifacts.\nIn the end, the distortion only has to be believable—not realistic.\n\n
\n\n
\n\n`rior` values above one tend to elongate the refraction while\nnumbers below one tend to shrink the refraction.\n\nAs it was with screen space reflection,\nthe screen doesn't have the entire geometry of the scene.\nA refracted ray may march through the screen space and never hit a captured surface.\nOr it may hit a surface but it's the backside not captured by the camera.\nWhen this happened during reflection, the fragment was left blank.\nThis indicated no reflection or not enough information to determine a reflection.\nLeaving the fragment blank was fine for reflection since the reflective surface would fill in the gaps.\n\n
\n\n
\n\nFor refraction, however, we must set the fragment to some UV.\nIf the fragment is left blank,\nthe refractive surface will contain holes that let the detail behind it come through.\nThis would be okay for a completely transparent surface but usually\nthe refractive surface will have some tint to it, reflection, etc.\n\n```c\n // ...\n\n vec2 texSize = textureSize(positionFromTexture, 0).xy;\n vec2 texCoord = gl_FragCoord.xy / texSize;\n\n vec4 uv = vec4(texCoord.xy, 1, 1);\n\n // ...\n```\n\nThe best choice is to select the UV as if the `rior` was one.\nThis will leave the UV coordinate unchanged,\nallowing the background to show through instead\nof there being a hole in the refractive surface.\n\n
\n\n
\n\nHere you see the refracted UV texture for the mill scene.\nThe wheel and waterway disturb what is otherwise a smooth gradient.\nThe disruptions shift the UV coordinates from their screen position to their refracted screen position.\n\n```c\n // ...\n\n vec3 unitPositionFrom = normalize(positionFrom.xyz);\n vec3 normalFrom = normalize(texture(normalFromTexture, texCoord).xyz);\n vec3 pivot = normalize(refract(unitPositionFrom, normalFrom, rior.x));\n\n // ...\n```\n\nThe most important difference is the calculation of the refracted vector versus the reflected vector.\nBoth use the unit position and normal but `refract` takes an additional parameter specifying the relative refractive index.\n\n```c\n // ...\n\n frag += increment;\n uv.xy = frag / texSize;\n positionTo = texture(positionToTexture, uv.xy);\n\n // ...\n```\n\nThe `positionTo`, sampled by the `uv` coordinates, uses the `positionToTexture`.\nFor reflection,\nyou only need one framebuffer texture containing the scene's interpolated vertex positions in view space.\nHowever,\nfor refraction,\n`positionToTexture` contains the vertex positions of the scene minus the refractive surfaces since\nthe refraction ray typically goes behind the surface.\nIf `positionFromTexture` and `positionToTexture` were the same for refraction,\nthe refracted ray would hit the refractive surface instead of what is behind it.\n\n### Refraction Mask\n\n
\n\n
\n\nYou'll need a mask to filter out the non-refractive parts of the scene.\nThis mask will determine which fragment does and does not receive a refracted color.\nYou could use this mask during the refracted UV calculation step or later when you\nactually sample the colors at the refracted UV coordinates.\n\nThe mill scene uses the models' material specular as a mask.\nFor the demo's purposes,\nthe specular map is sufficient but you may want to use a more specialized map.\nRefer back to [screen space reflection](screen-space-reflection.md#specular-map) for how to render the specular map.\n\n## Background Colors\n\n
\n\n
\n\nYou'll need to render the parts of the scene behind the refractive objects.\nThis can be done by hiding the refractive objects and then rendering the scene to a framebuffer texture.\n\n### Foreground Colors\n\n
\n\n
\n\n```c\n// ...\n\nuniform sampler2D uvTexture;\nuniform sampler2D refractionMaskTexture;\nuniform sampler2D positionFromTexture;\nuniform sampler2D positionToTexture;\nuniform sampler2D backgroundColorTexture;\n\n// ...\n```\n\nTo render the actual refractions or foreground colors,\nyou'll need the refracted UV coordinates,\nrefraction mask,\nthe foreground and background vertex positions,\nand the background colors.\n\n```c\n // ...\n\n vec3 tintColor = vec3(0.27, 0.58, 0.92, 0.3);\n float depthMax = 2;\n\n // ...\n```\n\n`tintColor` and `depthMax` are adjustable parameters.\n`tintColor` colorizes the background color.\n`depthMax` ranges from zero to infinity.\nWhen the distance between the foreground and background position reaches `depthMax`,\nthe foreground color will be the fully tinted background color.\nAt distance zero, the foreground will be the background color.\n\n```c\n // ...\n\n vec2 texSize = textureSize(backgroundColorTexture, 0).xy;\n vec2 texCoord = gl_FragCoord.xy / texSize;\n\n vec4 uv = texture(uvTexture, texCoord);\n vec4 mask = texture(maskTexture, texCoord);\n vec4 positionFrom = texture(positionFromTexture, texCoord);\n vec4 positionTo = texture(positionToTexture, uv.xy);\n vec4 backgroundColor = texture(backgroundColorTexture, uv.xy);\n\n if (refractionMask.r <= 0) { fragColor = vec4(0); return; }\n\n // ...\n```\n\nPull out the uv coordinates,\nmask,\nbackground position,\nforeground position,\nand the background color.\n\nIf the refraction mask is turned off for this fragment, return nothing.\n\n```c\n // ...\n\n float depth = length(positionTo.xyz - positionFrom.xyz);\n float mixture = clamp(depth / depthMax, 0, 1);\n\n vec3 shallowColor = backgroundColor.rgb;\n vec3 deepColor = mix(shallowColor, tintColor.rgb, tintColor.a);\n vec3 foregroundColor = mix(shallowColor, deepColor, mixture);\n\n // ...\n```\n\n
\n\n
\n\nCalculate the depth or distance between the foreground position and the background position.\nAt zero depth, the foreground color will be the shallow color.\nAt `depthMax`, the foreground color will be the deep color.\nThe deep color is the background color tinted with `tintColor`.\n\n```c\n // ...\n\n fragColor = mix(vec4(0), vec4(foregroundColor, 1), uv.b);\n\n // ...\n```\n\nRecall that the blue channel, in the refracted UV texture, is set to the visibility.\nThe visibility declines as the refracted ray points back at the camera.\nWhile the visibility should always be one, it is put here for completeness.\nAs the visibility lessens, the fragment color will receive less and less of the foreground color.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [base.vert](../demonstration/shaders/vertex/base.vert)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [position.frag](../demonstration/shaders/fragment/position.frag)\n- [normal.frag](../demonstration/shaders/fragment/normal.frag)\n- [material-specular.frag](../demonstration/shaders/fragment/material-specular.frag)\n- [screen-space-refraction.frag](../demonstration/shaders/fragment/screen-space-refraction.frag)\n- [refraction.frag](../demonstration/shaders/fragment/refraction.frag)\n- [base-combine.frag](../demonstration/shaders/fragment/base-combine.frag)\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](screen-space-reflection.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](foam.md)\n"
},
{
"path": "sections/setup.md",
"content": "[:arrow_backward:](gamma-correction.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](building-the-demo.md)\n\n# 3D Game Shaders For Beginners\n\n## Setup\n\n
\n\n
\n\nBelow is the setup used to develop and test the example code.\n\n### Environment\n\nThe example code was developed and tested using the following environment.\n\n- Linux manjaro 5.10.42-1-MANJARO\n- OpenGL renderer string: GeForce GTX 970/PCIe/SSE2\n- OpenGL version string: 4.6.0 NVIDIA 465.31\n- g++ (GCC) 11.1.0\n- Panda3D 1.10.9\n\n### Materials\n\nEach [Blender](https://blender.org) material used to build `mill-scene.egg` has five textures\nin the following order.\n\n- Diffuse\n- Normal\n- Specular\n- Reflection\n- Refraction\n\nBy having the same maps in the same positions for all models,\nthe shaders can be generalized, reducing the need to duplicate code.\n\nIf an object uses its vertex normals, a \"flat blue\" normal map is used.\n\n
\n\n
\n\nHere is an example of a flat normal map.\nThe only color it contains is flat blue `(red = 128, green = 128, blue = 255)`.\nThis color represents a unit (length one) normal pointing in the positive z-axis `(0, 0, 1)`.\n\n```c\n(0, 0, 1) =\n ( round((0 * 0.5 + 0.5) * 255)\n , round((0 * 0.5 + 0.5) * 255)\n , round((1 * 0.5 + 0.5) * 255)\n ) =\n (128, 128, 255) =\n ( round(128 / 255 * 2 - 1)\n , round(128 / 255 * 2 - 1)\n , round(255 / 255 * 2 - 1)\n ) =\n (0, 0, 1)\n```\n\nHere you see the unit normal `(0, 0, 1)`\nconverted to flat blue `(128, 128, 255)`\nand flat blue converted to the unit normal.\nYou'll learn more about this in the [normal mapping](normal-mapping.md) technique.\n\n
\n\n
\n\nUp above is one of the specular maps used.\nThe red and blue channel work to control the amount of specular reflection seen based on the camera angle.\nThe green channel controls the shininess factor.\nYou'll learn more about this in the [lighting](lighting.md) and [fresnel factor](fresnel-factor.md) sections.\n\nThe reflection and refraction textures mask off the objects that are either reflective, refractive, or both.\nFor the reflection texture, the red channel controls the amount of reflection and the green channel controls how\nclear or blurry the reflection is.\n\n### Panda3D\n\nThe example code uses\n[Panda3D](https://www.panda3d.org/)\nas the glue between the shaders.\nThis has no real influence over the techniques described,\nmeaning you'll be able to take what you learn here and apply it to your stack or game engine of choice.\nPanda3D does provide some conveniences.\nI have pointed these out so you can either find an equivalent convenience provided by your stack or\nreplicate it yourself, if your stack doesn't provide something equivalent.\n\nThree Panda3D configurations were changed for the purposes of the demo program.\nYou can find these in [config.prc](../demonstration/config.prc).\nThe configurations changed were\n`gl-coordinate-system default`,\n`textures-power-2 down`, and\n`textures-auto-power-2 1`.\nRefer to the\n[Panda3D configuration](http://www.panda3d.org/manual/?title=Configuring_Panda3D)\npage in the manual for more details.\n\nPanda3D defaults to a z-up, right-handed coordinate system while OpenGL uses a y-up, right-handed system.\n`gl-coordinate-system default` keeps you from having to translate between the two inside your shaders.\n`textures-auto-power-2 1` allows us to use texture sizes that are not a power of two if the system supports it.\nThis comes in handy when doing SSAO and other screen/window sized related techniques since the screen/window size\nis usually not a power of two.\n`textures-power-2 down` downsizes our textures to a power of two if the system only supports texture sizes being a power of two.\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](gamma-correction.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](building-the-demo.md)\n"
},
{
"path": "sections/sharpen.md",
"content": "[:arrow_backward:](pixelization.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](dilation.md)\n\n# 3D Game Shaders For Beginners\n\n## Sharpen\n\n
\n\n
\n\nThe sharpen effect increases the contrast at the edges of the image.\nThis comes in handy when your graphics are bit too soft.\n\n```c\n // ...\n\n float amount = 0.8;\n\n // ...\n```\n\nYou can control how sharp the result is by adjusting the amount.\nAn amount of zero leaves the image untouched.\nTry negative values for an odd look.\n\n```c\n // ...\n\n float neighbor = amount * -1;\n float center = amount * 4 + 1;\n\n // ...\n```\n\nNeighboring fragments are multiplied by `amount * -1`.\nThe current fragment is multiplied by `amount * 4 + 1`.\n\n```c\n // ...\n\n vec3 color =\n texture(sharpenTexture, vec2(gl_FragCoord.x + 0, gl_FragCoord.y + 1) / texSize).rgb\n * neighbor\n\n + texture(sharpenTexture, vec2(gl_FragCoord.x - 1, gl_FragCoord.y + 0) / texSize).rgb\n * neighbor\n + texture(sharpenTexture, vec2(gl_FragCoord.x + 0, gl_FragCoord.y + 0) / texSize).rgb\n * center\n + texture(sharpenTexture, vec2(gl_FragCoord.x + 1, gl_FragCoord.y + 0) / texSize).rgb\n * neighbor\n\n + texture(sharpenTexture, vec2(gl_FragCoord.x + 0, gl_FragCoord.y - 1) / texSize).rgb\n * neighbor\n ;\n\n // ...\n```\n\nThe neighboring fragments are up, down, left, and right.\nAfter multiplying both the neighbors and the current fragment by their particular values, sum the result.\n\n```c\n // ...\n\n fragColor = vec4(color, texture(sharpenTexture, texCoord).a);\n\n // ...\n```\n\nThis sum is the final fragment color.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [sharpen.frag](../demonstration/shaders/fragment/sharpen.frag)\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](pixelization.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](dilation.md)\n"
},
{
"path": "sections/ssao.md",
"content": "[:arrow_backward:](bloom.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](motion-blur.md)\n\n# 3D Game Shaders For Beginners\n\n## Screen Space Ambient Occlusion (SSAO)\n\n
\n\n
\n\nSSAO is one of those effects you never knew you needed and can't live without once you have it.\nIt can take a scene from mediocre to wow!\nFor fairly static scenes, you can bake ambient occlusion into a texture but for more dynamic scenes, you'll need a shader.\nSSAO is one of the more fairly involved shading techniques, but once you pull it off, you'll feel like a shader master.\n\nBy using only a handful of textures, SSAO can approximate the\n[ambient occlusion](https://en.wikipedia.org/wiki/Ambient_occlusion)\nof a scene.\nThis is faster than trying to compute the ambient occlusion by going through all of the scene's geometry.\nThese handful of textures all originate in screen space giving screen space ambient occlusion its name.\n\n### Inputs\n\nThe SSAO shader will need the following inputs.\n\n- Vertex position vectors in view space.\n- Vertex normal vectors in view space.\n- Sample vectors in tangent space.\n- Noise vectors in tangent space.\n- The camera lens' projection matrix.\n\n### Vertex Positions\n\n
\n\n
\n\nStoring the vertex positions into a framebuffer texture is not a necessity.\nYou can recreate them from the [camera's depth buffer](http://theorangeduck.com/page/pure-depth-ssao).\nThis being a beginners guide, I'll avoid this optimization and keep it straight forward.\nFeel free to use the depth buffer, however, for your implementation.\n\n```cpp\nPT(Texture) depthTexture =\n new Texture(\"depthTexture\");\ndepthTexture->set_format\n ( Texture::Format::F_depth_component32\n );\n\nPT(GraphicsOutput) depthBuffer =\n graphicsOutput->make_texture_buffer\n ( \"depthBuffer\"\n , 0\n , 0\n , depthTexture\n );\ndepthBuffer->set_clear_color\n ( LVecBase4f(0, 0, 0, 0)\n );\n\nNodePath depthCameraNP =\n window->make_camera();\nDCAST(Camera, depthCameraNP.node())->set_lens\n ( window->get_camera(0)->get_lens()\n );\nPT(DisplayRegion) depthBufferRegion =\n depthBuffer->make_display_region\n ( 0\n , 1\n , 0\n , 1\n );\ndepthBufferRegion->set_camera(depthCameraNP);\n```\n\nIf you do decide to use the depth buffer, here's how you can set it up using Panda3D.\n\n```c\nin vec4 vertexPosition;\n\nout vec4 fragColor;\n\nvoid main() {\n fragColor = vertexPosition;\n}\n```\n\nHere's the simple shader used to render out the view space vertex positions into a framebuffer texture.\nThe more involved work is setting up the framebuffer texture such that the fragment vector components it receives\nare not clamped to `[0, 1]` and that each one has a high enough precision (a high enough number of bits).\nFor example, if a particular interpolated vertex position is `<-139.444444566, 0.00000034343, 2.5>`,\nyou don't want it stored into the texture as `<0.0, 0.0, 1.0>`.\n\n```c\n // ...\n\n FrameBufferProperties fbp = FrameBufferProperties::get_default();\n\n // ...\n\n fbp.set_rgba_bits(32, 32, 32, 32);\n fbp.set_rgb_color(true);\n fbp.set_float_color(true);\n\n // ...\n```\n\nHere's how the example code sets up the framebuffer texture to store the vertex positions.\nIt wants 32 bits per red, green, blue, and alpha components and disables clamping the values to `[0, 1]`\nThe `set_rgba_bits(32, 32, 32, 32)` call sets the bits and also disables the clamping.\n\n```c\n glTexImage2D\n ( GL_TEXTURE_2D\n , 0\n , GL_RGB32F\n , 1200\n , 900\n , 0\n , GL_RGB\n , GL_FLOAT\n , nullptr\n );\n```\n\nHere's the equivalent OpenGL call.\n`GL_RGB32F` sets the bits and also disables the clamping.\n\n
\nIf the color buffer is fixed-point, the components of the source and destination\nvalues and blend factors are each clamped to [0, 1] or [−1, 1] respectively for\nan unsigned normalized or signed normalized color buffer prior to evaluating the blend\nequation.\nIf the color buffer is floating-point, no clamping occurs.\n \n \n\n
\n\n
\n\n
\n\nHere you see the vertex positions with y being the up vector.\n\nRecall that Panda3D sets z as the up vector but OpenGL uses y as the up vector.\nThe position shader outputs the vertex positions with z being up since Panda3D\nwas configured with `gl-coordinate-system default`.\n\n### Vertex Normals\n\n
\n\n
\n\nYou'll need the vertex normals to correctly orient the samples you'll take in the SSAO shader.\nThe example code generates multiple sample vectors distributed in a hemisphere\nbut you could use a sphere and do away with the need for normals all together.\n\n```c\nin vec3 vertexNormal;\n\nout vec4 fragColor;\n\nvoid main() {\n vec3 normal = normalize(vertexNormal);\n\n fragColor = vec4(normal, 1);\n}\n```\n\nLike the position shader, the normal shader is simple as well.\nBe sure to normalize the vertex normal and remember that they are in view space.\n\n
\n\n
\n\nHere you see the vertex normals with y being the up vector.\n\nRecall that Panda3D sets z as the up vector but OpenGL uses y as the up vector.\nThe normal shader outputs the vertex positions with z being up since Panda3D\nwas configured with `gl-coordinate-system default`.\n\n
\n\n
\n\nHere you see SSAO being used with the normal maps instead of the vertex normals.\nThis adds an extra level of detail and will pair nicely with the normal mapped lighting.\n\n```c\n // ...\n\n normal =\n normalize\n ( normalTex.rgb\n * 2.0\n - 1.0\n );\n normal =\n normalize\n ( mat3\n ( tangent\n , binormal\n , vertexNormal\n )\n * normal\n );\n\n // ...\n```\n\nTo use the normal maps instead,\nyou'll need to transform the normal mapped normals from tangent space to view space\njust like you did in the lighting calculations.\n\n### Samples\n\nTo determine the amount of ambient occlusion for any particular fragment,\nyou'll need to sample the surrounding area.\nThe more samples you use, the better the approximation at the cost of performance.\n\n```cpp\n // ...\n\n for (int i = 0; i < numberOfSamples; ++i) {\n LVecBase3f sample =\n LVecBase3f\n ( randomFloats(generator) * 2.0 - 1.0\n , randomFloats(generator) * 2.0 - 1.0\n , randomFloats(generator)\n ).normalized();\n\n float rand = randomFloats(generator);\n sample[0] *= rand;\n sample[1] *= rand;\n sample[2] *= rand;\n\n float scale = (float) i / (float) numberOfSamples;\n scale = lerp(0.1, 1.0, scale * scale);\n sample[0] *= scale;\n sample[1] *= scale;\n sample[2] *= scale;\n\n ssaoSamples.push_back(sample);\n }\n\n // ...\n```\n\nThe example code generates a number of random samples distributed in a hemisphere.\nThese `ssaoSamples` will be sent to the SSAO shader.\n\n```cpp\n LVecBase3f sample =\n LVecBase3f\n ( randomFloats(generator) * 2.0 - 1.0\n , randomFloats(generator) * 2.0 - 1.0\n , randomFloats(generator) * 2.0 - 1.0\n ).normalized();\n```\n\nIf you'd like to distribute your samples in a sphere instead,\nchange the random `z` component to range from negative one to one.\n\n### Noise\n\n```c\n // ...\n\n for (int i = 0; i < numberOfNoise; ++i) {\n LVecBase3f noise =\n LVecBase3f\n ( randomFloats(generator) * 2.0 - 1.0\n , randomFloats(generator) * 2.0 - 1.0\n , 0.0\n );\n\n ssaoNoise.push_back(noise);\n }\n\n // ...\n```\n\nTo get a good sweep of the sampled area, you'll need to generate some noise vectors.\nThese noise vectors will randomly tilt the hemisphere around the current fragment.\n\n### Ambient Occlusion\n\n
\n\n
\n\nSSAO works by sampling the view space around a fragment.\nThe more samples that are below a surface, the darker the fragment color.\nThese samples are positioned at the fragment and pointed in the general direction of the vertex normal.\nEach sample is used to look up a position in the position framebuffer texture.\nThe position returned is compared to the sample.\nIf the sample is farther away from the camera than the position, the sample counts towards the fragment being occluded.\n\n
\n\n
\n\nHere you see the space above the surface being sampled for occlusion.\n\n```c\n // ...\n\n float radius = 1;\n float bias = 0.01;\n float magnitude = 1.5;\n float contrast = 1.5;\n\n // ...\n```\n\nLike some of the other techniques,\nthe SSAO shader has a few control knobs you can tweak to get the exact look you're going for.\nThe `bias` adds to the sample's distance from the camera.\nYou can use the bias to combat \"acne\".\nThe `radius` increases or decreases the coverage area of the sample space.\nThe `magnitude` either lightens or darkens the occlusion map.\nThe `contrast` either washes out or increases the starkness of the occlusion map.\n\n```c\n // ...\n\n vec4 position = texture(positionTexture, texCoord);\n vec3 normal = normalize(texture(normalTexture, texCoord).xyz);\n\n int noiseX = int(gl_FragCoord.x - 0.5) % 4;\n int noiseY = int(gl_FragCoord.y - 0.5) % 4;\n vec3 random = noise[noiseX + (noiseY * 4)];\n\n // ...\n```\n\nRetrieve the position, normal, and random vector for later use.\nRecall that the example code created a set number of random vectors.\nThe random vector is chosen based on the current fragment's screen position.\n\n```c\n // ...\n\n vec3 tangent = normalize(random - normal * dot(random, normal));\n vec3 binormal = cross(normal, tangent);\n mat3 tbn = mat3(tangent, binormal, normal);\n\n // ...\n```\n\nUsing the random and normal vectors, assemble the tangent, binormal, and normal matrix.\nYou'll need this matrix to transform the sample vectors from tangent space to view space.\n\n```c\n // ...\n\n float occlusion = NUM_SAMPLES;\n\n for (int i = 0; i < NUM_SAMPLES; ++i) {\n // ...\n }\n\n // ...\n```\n\nWith the matrix in hand, the shader can now loop through the samples, subtracting how many are not occluded.\n\n```c\n // ...\n\n vec3 samplePosition = tbn * samples[i];\n samplePosition = position.xyz + samplePosition * radius;\n\n // ...\n```\n\nUsing the matrix, position the sample near the vertex/fragment position and scale it by the radius.\n\n```c\n // ...\n\n vec4 offsetUV = vec4(samplePosition, 1.0);\n offsetUV = lensProjection * offsetUV;\n offsetUV.xyz /= offsetUV.w;\n offsetUV.xy = offsetUV.xy * 0.5 + 0.5;\n\n // ...\n```\n\nUsing the sample's position in view space, transform it from view space to clip space to UV space.\n\n```c\n-1 * 0.5 + 0.5 = 0\n 1 * 0.5 + 0.5 = 1\n```\n\nRecall that clip space components range from negative one to one and that UV coordinates range from zero to one.\nTo transform clip space coordinates to UV coordinates, multiply by one half and add one half.\n\n```c\n // ...\n\n vec4 offsetPosition = texture(positionTexture, offsetUV.xy);\n\n float occluded = 0;\n if (samplePosition.y + bias <= offsetPosition.y) { occluded = 0; } else { occluded = 1; }\n\n // ...\n```\n\nUsing the offset UV coordinates,\ncreated by projecting the 3D sample onto the 2D position texture,\nfind the corresponding position vector.\nThis takes you from view space to clip space to UV space back to view space.\nThe shader takes this round trip to find out if some geometry is behind, at, or in front of this sample.\nIf the sample is in front of or at some geometry, this sample doesn't count towards the fragment being occluded.\nIf the sample is behind some geometry, this sample counts towards the fragment being occluded.\n\n```c\n // ...\n\n float intensity =\n smoothstep\n ( 0.0\n , 1.0\n , radius\n / abs(position.y - offsetPosition.y)\n );\n occluded *= intensity;\n\n occlusion -= occluded;\n\n // ...\n```\n\nNow weight this sampled position by how far it is inside or outside the radius.\nFinally, subtract this sample from the occlusion factor since it assumes all of the samples are occluded before the loop.\n\n```c\n // ...\n\n occlusion /= NUM_SAMPLES;\n\n // ...\n\n fragColor = vec4(vec3(occlusion), position.a);\n\n // ...\n```\n\nDivide the occluded count by the number of samples to scale the occlusion factor from `[0, NUM_SAMPLES]` to `[0, 1]`.\nZero means full occlusion and one means no occlusion.\nNow assign the occlusion factor to the fragment's color and you're done.\n\n```c\n // ...\n\n fragColor = vec4(vec3(occlusion), position.a);\n\n // ...\n```\n\nFor the demo's purposes,\nthe example code sets the alpha channel to alpha channel of the position framebuffer texture to avoid covering up the background.\n\n### Blurring\n\n
\n\n
\n\nThe SSAO framebuffer texture is noisy as is.\nYou'll want to blur it to remove the noise.\nRefer back to the section on [blurring](blur.md).\nFor the best results, use a median or Kuwahara filter to preserve the sharp edges.\n\n### Ambient Color\n\n```c\n // ...\n\n vec2 ssaoBlurTexSize = textureSize(ssaoBlurTexture, 0).xy;\n vec2 ssaoBlurTexCoord = gl_FragCoord.xy / ssaoBlurTexSize;\n float ssao = texture(ssaoBlurTexture, ssaoBlurTexCoord).r;\n\n vec4 ambient = p3d_Material.ambient * p3d_LightModel.ambient * diffuseTex * ssao;\n\n // ...\n```\n\nThe final stop for SSAO is back in the lighting calculation.\nHere you see the occlusion factor being looked up in the\nSSAO framebuffer texture and then included in the ambient light calculation.\n\n### Source\n\n- [main.cxx](../demonstration/src/main.cxx)\n- [basic.vert](../demonstration/shaders/vertex/basic.vert)\n- [base.vert](../demonstration/shaders/vertex/base.vert)\n- [base.frag](../demonstration/shaders/fragment/base.frag)\n- [position.frag](../demonstration/shaders/fragment/position.frag)\n- [normal.frag](../demonstration/shaders/fragment/normal.frag)\n- [ssao.frag](../demonstration/shaders/fragment/ssao.frag)\n- [median-filter.frag](../demonstration/shaders/fragment/median-filter.frag)\n- [kuwahara-filter.frag](../demonstration/shaders/fragment/kuwahara-filter.frag)\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](bloom.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](motion-blur.md)\n"
},
{
"path": "sections/texturing.md",
"content": "[:arrow_backward:](render-to-texture.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](lighting.md)\n\n# 3D Game Shaders For Beginners\n\n## Texturing\n\n
\n\n
\n\nTexturing involves mapping some color or some other kind of vector to a fragment using UV coordinates.\nBoth U and V range from zero to one.\nEach vertex gets a UV coordinate and this is outputted in the vertex shader.\n\n
\n\n
\n\nThe fragment shader receives the UV coordinate interpolated.\nInterpolated meaning the UV coordinate for the fragment is somewhere between the UV coordinates\nfor the vertexes that make up the triangle face.\n\n### Vertex\n\n```c\n#version 150\n\nuniform mat4 p3d_ModelViewProjectionMatrix;\n\nin vec2 p3d_MultiTexCoord0;\n\nin vec4 p3d_Vertex;\n\nout vec2 texCoord;\n\nvoid main()\n{\n texCoord = p3d_MultiTexCoord0;\n\n gl_Position = p3d_ModelViewProjectionMatrix * p3d_Vertex;\n}\n```\n\nHere you see the vertex shader outputting the texture coordinate to the fragment shader.\nNotice how it's a two dimensional vector.\nOne dimension for U and one for V.\n\n### Fragment\n\n```c\n#version 150\n\nuniform sampler2D p3d_Texture0;\n\nin vec2 texCoord;\n\nout vec2 fragColor;\n\nvoid main()\n{\n texColor = texture(p3d_Texture0, texCoord);\n\n fragColor = texColor;\n}\n```\n\nHere you see the fragment shader looking up the color at its UV coordinate and outputting that as the fragment color.\n\n#### Screen Filled Texture\n\n```c\n#version 150\n\nuniform sampler2D screenSizedTexture;\n\nout vec2 fragColor;\n\nvoid main()\n{\n vec2 texSize = textureSize(texture, 0).xy;\n vec2 texCoord = gl_FragCoord.xy / texSize;\n\n texColor = texture(screenSizedTexture, texCoord);\n\n fragColor = texColor;\n}\n```\n\nWhen performing render to texture, the mesh is a flat rectangle with the same aspect ratio as the screen.\nBecause of this, you can calculate the UV coordinates knowing only\nA) the width and height of the screen sized texture being UV mapped to the rectangle and\nB) the fragment's x and y coordinate.\nTo map x to U, divide x by the width of the input texture.\nSimilarly, to map y to V, divide y by the height of the input texture.\nYou'll see this technique used in the example code.\n\n## Copyright\n\n(C) 2019 David Lettier\n \n[lettier.com](https://www.lettier.com)\n\n[:arrow_backward:](render-to-texture.md)\n[:arrow_double_up:](../README.md)\n[:arrow_up_small:](#)\n[:arrow_down_small:](#copyright)\n[:arrow_forward:](lighting.md)\n"
}
]
![\"3D](\"https://i.imgur.com/x8rtGr4.gif\" "\\"3D")
\n![\"Blinn-Phong\"](\"https://i.imgur.com/CFWEeGK.gif\" "\\"Blinn-Phong\\"")
\n![\"Blinn-Phong](\"https://i.imgur.com/vtqd1Ox.gif\" "\\"Blinn-Phong")
\n![\"Full](\"https://i.imgur.com/WZQqxEH.png\" "\\"Full")
\n![\"Blinn-Phong](\"https://i.imgur.com/kiSdJzt.png\" "\\"Blinn-Phong")
\n![\"Bloom\"](\"https://i.imgur.com/UxKRz2r.gif\" "\\"Bloom\\"")
\n![\"Bloom](\"https://i.imgur.com/m4yedrM.gif\" "\\"Bloom")
\n![\"Kuwahara](\"https://i.imgur.com/b5vw2AJ.gif\" "\\"Kuwahara")
\n![\"Box](\"https://i.imgur.com/uaXC1JM.gif\" "\\"Box")
\n![\"Blur](\"https://i.imgur.com/bRw0OkX.png\" "\\"Blur")
\n![\"Median](\"https://i.imgur.com/T1nEEn3.gif\" "\\"Median")
\n![\"Painterly\"](\"https://i.imgur.com/AnbzUmN.png\" "\\"Painterly\\"")
\n![\"Kuwahara](\"https://i.imgur.com/iuLbLKO.gif\" "\\"Kuwahara")
\n![\"Building](\"https://i.imgur.com/PQcDnIu.gif\" "\\"Building")
\n![\"Cel](\"https://i.imgur.com/W80Ke1y.gif\" "\\"Cel")
\n![\"Step](\"https://i.imgur.com/lyLweFc.png\" "\\"Step")
\n![\"Steps](\"https://i.imgur.com/EI6QJ60.png\" "\\"Steps")
\n![\"Step](\"https://i.imgur.com/7KK65mi.png\" "\\"Step")
\n![\"Chromatic](\"https://i.imgur.com/bawgERm.gif\" "\\"Chromatic")
\n![\"Chromatic](\"https://i.imgur.com/fNpMaPL.gif\" "\\"Chromatic")
\n![\"Deferred](\"https://i.imgur.com/rxTYLGv.png\" "\\"Deferred")
\n![\"G-buffer\"](\"https://i.imgur.com/uTimQJj.gif\" "\\"G-buffer\\"")
\n![\"Lighting](\"https://i.imgur.com/frYp6to.gif\" "\\"Lighting")
\n![\"Depth](\"https://i.imgur.com/DEa77Bh.gif\" "\\"Depth")
\n![\"Bokeh\"](\"https://i.imgur.com/aQ9Ga8J.gif\" "\\"Bokeh\\"")
\n![\"smoothstep\"](\"https://i.imgur.com/idDZr62.png\" "\\"smoothstep\\"")
\n![\"Dilation\"](\"https://i.imgur.com/z751O74.gif\" "\\"Dilation\\"")
\n![\"Dilation](\"https://i.imgur.com/X3uIyIL.png\" "\\"Dilation")
\n![\"Film](\"https://i.imgur.com/ct7mTv5.gif\" "\\"Film")
\n![\"Horizontal,](\"https://i.imgur.com/xqSIMCb.gif\" "\\"Horizontal,")
\n![\"Rain\"](\"https://i.imgur.com/4UXllmS.gif\" "\\"Rain\\"")
\n![\"Increasing](\"https://i.imgur.com/Mtt8BNg.gif\" "\\"Increasing")
\n![\"Flow](\"https://i.imgur.com/3WDO9xW.gif\" "\\"Flow")
\n![\"Flow](\"https://i.imgur.com/b9Vw94N.png\" "\\"Flow")
\n![\"Foam](\"https://i.imgur.com/N6TWBw8.gif\" "\\"Foam")
\n![\"Foam\"](\"https://i.imgur.com/SVLPYKn.gif\" "\\"Foam\\"")
\n![\"Lava](\"https://i.imgur.com/HCqvd8c.gif\" "\\"Lava")
\n![\"Easing](\"https://i.imgur.com/CDIPmin.png\" "\\"Easing")
\n![\"Fog\"](\"https://i.imgur.com/uNRxZl4.gif\" "\\"Fog\\"")
\n![\"Fresnel\"](\"https://i.imgur.com/3lQL51m.gif\" "\\"Fresnel\\"")
\n![\"\"](\"https://i.imgur.com/WolRRhX.png\" "\\"\\"")
\n![\"Specular](\"https://i.imgur.com/FnOhXxv.gif\" "\\"Specular")
\n![\"Fresnel](\"https://i.imgur.com/AAFI8p1.gif\" "\\"Fresnel")
\n![\"Gamma](\"https://i.imgur.com/IG7A6cj.gif\" "\\"Gamma")
\n![\"sRGB](\"https://i.imgur.com/a1U5oBq.png\" "\\"sRGB")
\n![\"Color](\"https://i.imgur.com/E5nkRfG.png\" "\\"Color")
\n![\"Gamma](\"https://i.imgur.com/TOEb0EC.gif\" "\\"Gamma")
\n![\"Non-color](\"https://i.imgur.com/reA2qjs.png\" "\\"Non-color")
\n![\"Perceptually](\"https://i.imgur.com/tRxkKNe.gif\" "\\"Perceptually")
\n![\"Not](\"https://i.imgur.com/4km0pdv.gif\" "\\"Not")
\n![\"\"](\"https://i.imgur.com/7b5MCBG.gif\" "\\"\\"")
\n![\"Output](\"https://i.imgur.com/V25UzMa.gif\" "\\"Output")
\n![\"Lighting\"](\"https://i.imgur.com/zQrA8tr.gif\" "\\"Lighting\\"")
\n![\"Phong](\"https://i.imgur.com/0pzNh5d.gif\" "\\"Phong")
\n![\"The](\"https://i.imgur.com/Nb78z96.gif\" "\\"The")
\n![\"Shininess\"](\"https://i.imgur.com/4r6wqLP.gif\" "\\"Shininess\\"")
\n![\"Attenuation\"](\"https://i.imgur.com/jyatr7l.png\" "\\"Attenuation\\"")
\n![\"LUT\"](\"https://i.imgur.com/WrPzVlW.gif\" "\\"LUT\\"")
\n![\"Neutral](\"https://i.imgur.com/NPdJNGj.png\" "\\"Neutral")
\n![\"LUT](\"https://i.imgur.com/KyxPm1r.png\" "\\"LUT")
\n![\"RGB](\"https://i.imgur.com/j2JmyQ2.png\" "\\"RGB")
\n![\"Mixing\"](\"https://i.imgur.com/uciq7Um.png\" "\\"Mixing\\"")
\n![\"Motion](\"https://i.imgur.com/eTnhpLr.gif\" "\\"Motion")
\n![\"Position](\"https://i.imgur.com/oQqdxM9.gif\" "\\"Position")
\n![\"Normal](\"https://i.imgur.com/M4eHo9I.gif\" "\\"Normal")
\n![\"From](\"https://i.imgur.com/nSY9AW4.gif\" "\\"From")
\n![\"Normal](\"https://i.imgur.com/jvkRPE7.gif\" "\\"Normal")
\n![\"Normal](\"https://i.imgur.com/tLIA6Hu.gif\" "\\"Normal")
\n![\"Replacing](\"https://i.imgur.com/EzHJPd4.gif\" "\\"Replacing")
\n![\"Outlined](\"https://i.imgur.com/EKaZRkR.gif\" "\\"Outlined")
\n![\"Edge](\"https://i.imgur.com/xAMRGhn.gif\" "\\"Edge")
\n![\"Outlines\"](\"https://i.imgur.com/yWNdPZe.gif\" "\\"Outlines\\"")
\n![\"Sketchy](\"https://i.imgur.com/FRBkSm1.gif\" "\\"Sketchy")
\n![\"Outline](\"https://i.imgur.com/qgZNtnN.png\" "\\"Outline")
\n![\"Squiggly](\"https://i.imgur.com/wk43ybP.png\" "\\"Squiggly")
\n![\"Pixelization\"](\"https://i.imgur.com/IbnyYZN.gif\" "\\"Pixelization\\"")
\n![\"Pixelization](\"https://i.imgur.com/WF5MmM0.gif\" "\\"Pixelization")
\n![\"Posterization\"](\"https://i.imgur.com/hpP5G9z.gif\" "\\"Posterization\\"")
\n![\"Model](\"https://i.imgur.com/8xptajU.gif\" "\\"Model")
\n![\"World](\"https://i.imgur.com/fHl4ohX.gif\" "\\"World")
\n![\"View](\"https://i.imgur.com/3b4SGGH.gif\" "\\"View")
\n![\"Clip](\"https://i.imgur.com/iSEWS9Y.png\" "\\"Clip")
\n![\"Frustum\"](\"https://i.imgur.com/MhgmOLv.gif\" "\\"Frustum\\"")
\n![\"Screen](\"https://i.imgur.com/bHHrjOl.png\" "\\"Screen")
\n![\"The](\"https://i.imgur.com/t3iLKhx.gif\" "\\"The")
\n![\"The](\"https://i.imgur.com/o8H6cTy.png\" "\\"The")
\n![\"Tabbing](\"https://i.imgur.com/L6Hwuxa.gif\" "\\"Tabbing")
\n![\"Rim](\"https://i.imgur.com/3erauzN.gif\" "\\"Rim")
\n![\"Rim](\"https://i.imgur.com/mMsQFbE.gif\" "\\"Rim")
\n![\"Running](\"https://i.imgur.com/y5XcReP.gif\" "\\"Running")
\n![\"Screen](\"https://i.imgur.com/AYG4tvg.gif\" "\\"Screen")
\n![\"SSR](\"https://i.imgur.com/wnAC7NI.gif\" "\\"SSR")
\n![\"SSR](\"https://i.imgur.com/1cE2vBY.gif\" "\\"SSR")
\n![\"Position](\"https://i.imgur.com/Qnsvkc0.gif\" "\\"Position")
\n![\"Reflected](\"https://i.imgur.com/pkQcGkD.gif\" "\\"Reflected")
\n![\"Larger](\"https://i.imgur.com/W163Mfw.png\" "\\"Larger")
\n![\"Smaller](\"https://i.imgur.com/T6uH4Yu.png\" "\\"Smaller")
\n![\"Screen](\"https://i.imgur.com/MpBR225.png\" "\\"Screen")
\n![\"The](\"https://i.imgur.com/Um4dzgL.png\" "\\"The")
\n![\"Reflection](\"https://i.imgur.com/7e2cOdZ.gif\" "\\"Reflection")
\n![\"Reflection](\"https://i.imgur.com/i0btBna.gif\" "\\"Reflection")
\n![\"Specular](\"https://i.imgur.com/iuFYVWB.gif\" "\\"Specular")
\n![\"Scene](\"https://i.imgur.com/diBSxPI.png\" "\\"Scene")
\n![\"Reflected](\"https://i.imgur.com/UPvgfDU.gif\" "\\"Reflected")
\n![\"Blurred](\"https://i.imgur.com/gVvx1Ei.png\" "\\"Blurred")
\n![\"Screen](\"https://i.imgur.com/8Mdcn4y.gif\" "\\"Screen")
\n![\"Normal](\"https://i.imgur.com/MZ2R8I6.gif\" "\\"Normal")
\n![\"Position](\"https://i.imgur.com/bXtXDyu.gif\" "\\"Position")
\n![\"Without](\"https://i.imgur.com/FjQtjsm.gif\" "\\"Without")
\n![\"Adjusting](\"https://i.imgur.com/dDOnobK.gif\" "\\"Adjusting")
\n![\"Refraction](\"https://i.imgur.com/vcQDAYU.gif\" "\\"Refraction")
\n![\"Refraction](\"https://i.imgur.com/9fybLUO.png\" "\\"Refraction")
\n![\"Background](\"https://i.imgur.com/AmT9RrU.gif\" "\\"Background")
\n![\"Foreground](\"https://i.imgur.com/6RPHULr.gif\" "\\"Foreground")
\n![\"Refraction](\"https://i.imgur.com/IEFKerB.gif\" "\\"Refraction")
\n![\"Setup\"](\"https://i.imgur.com/fYpIWNk.gif\" "\\"Setup\\"")
\n![\"A](\"https://i.imgur.com/tFmKgoH.png\" "\\"A")
\n![\"Specular](\"https://i.imgur.com/R9FgZKx.png\" "\\"Specular")
\n![\"Sharpen\"](\"https://i.imgur.com/VFDKNvl.gif\" "\\"Sharpen\\"")
\n![\"SSAO\"](\"https://i.imgur.com/o7lCukD.gif\" "\\"SSAO\\"")
\n![\"Panda3D](\"https://i.imgur.com/gr7IxKv.png\" "\\"Panda3D")
\n![\"OpenGL](\"https://i.imgur.com/V4nETME.png\" "\\"OpenGL")
\n![\"Panda3d](\"https://i.imgur.com/ilnbkzq.gif\" "\\"Panda3d")
\n![\"OpenGL](\"https://i.imgur.com/ucdx9Kp.gif\" "\\"OpenGL")
\n![\"SSAO](\"https://i.imgur.com/fiHXBex.gif\" "\\"SSAO")
\n![\"SSAO](\"https://i.imgur.com/KKt74VE.gif\" "\\"SSAO")
\n![\"SSAO](\"https://i.imgur.com/Nm4CJDN.gif\" "\\"SSAO")
\n![\"SSAO](\"https://i.imgur.com/QsqOhFR.gif\" "\\"SSAO")
\n![\"Diffuse](\"https://i.imgur.com/cqbgT8b.gif\" "\\"Diffuse")
\n![\"UV](\"https://i.imgur.com/JjAdNfk.png\" "\\"UV")
\n