In this chapter, we put together many of the techniques you have learned throughout the book to discuss some advanced uses of OpenGL ES 2.0. There are a large number of advanced rendering techniques that can be accomplished with the programmable flexibility of OpenGL ES 2.0. In this chapter, we cover the following advanced rendering techniques:
Per-fragment lighting.
Environment mapping.
Particle system with point sprites.
Image postprocessing.
Projective texturing.
Noise using a 3D texture.
Procedural textures.
In Chapter 8, “Vertex Shaders,” we covered the lighting equations that can be used in the vertex shader to calculate per-vertex lighting. Commonly, to achieve higher quality lighting, we seek to evaluate the lighting equations on a per-fragment basis. In this section, we cover an example of evaluating ambient, diffuse, and specular lighting on a per-fragment basis. The example we cover is a RenderMonkey workspace that can be found in Chapter_13/ PerFragmentLighting/PerFragmentLighting.rfx as pictured in Figure 13-1.
Before we get into the details of the shaders used in the RenderMonkey workspace, it’s necessary to first discuss the general approach that is used in the example. The simplest way to do lighting per-fragment would be to use the interpolated vertex normal in the fragment shader and then move the lighting computations into the fragment shader. However, for the diffuse term, this would really not yield much better results than doing the lighting per-vertex. There would be the advantage that the normal vector could be renormalized, which would remove artifacts due to linear interpolation, but the overall quality would be only minimally better. To really take advantage of the ability to do computations on a per-fragment basis, using a normal map to store per-texel normals can provide significantly more detail.
A normal map is a 2D texture that stores at each texel a normal vector. The red channel represents the x component, the green channel the y component, and the blue channel the z component. For a normal map stored as GL_RGB8 with GL_UNSIGNED_BYTE data, the values will all be in the range [0, 1]. To represent a normal, these values need to be scaled and biased in the shader to remap to [–1, 1]. The following block of fragment shader code shows how you would go about fetching from a normal map.
// Fetch the tangent space normal from normal map vec3 normal = texture2D(s_bumpMap, v_texcoord).xyz; // Scale and bias from [0, 1] to [-1, 1] and normalize normal = normalize(normal * 2.0 - 1.0);
As you can see, this small bit of shader code will fetch the color value from a texture map and then multiply the results by two and subtract one. The result is that the values are rescaled into the [–1, 1] range from the [0, 1] range. In addition, if the data in your normal map are not normalized, you will also need to normalize the results in the fragment shader. This step can be skipped if your normal map contains all unit vectors.
The other significant issue to tackle with per-fragment lighting has to do with in which space the normals in the texture are stored. To minimize computations in the fragment shader, we do not want to have to transform the result of the normal fetched from the normal map. One way to accomplish this would be to store world-space normals in your normal map. That is, the normal vectors in the normal map would each represent a world-space normal vector. Then, the light and direction vectors could be transformed into world space in the vertex shader and could be directly used with the value fetched from the normal map. However, there are significant issues with storing normal maps in world space. The most significant is that the object has to be assumed to be static because no transformation can happen on the object. Another significant issue is that the same surface oriented in different directions in space would not be able to share the same texels in the normal map, which can result in much larger maps.
A better solution than using world-space normal maps is to store normal maps in tangent space. The idea behind tangent space is that we define a space for each vertex by three coordinate axes: the normal, binormal, and tangent. The normals stored in the texture map are then all stored in this tangent space. Then, when we want to compute any lighting equations, we transform our incoming lighting vectors into the tangent space and those light vectors can then directly be used with the values in the normal map. The tangent space is typically computed as a preprocess and the binormal and tangent are added to the vertex attribute data. This work is done automatically by RenderMonkey, which computes a tangent space for any model that has a vertex normal and texture coordinates.
Once we have tangent space normal maps and tangent space vectors set up, we can proceed with per-fragment lighting. First, let’s take a look at the vertex shader in Example 13-1.
Example 13-1. Per-Fragment Lighting Vertex Shader
uniform mat4 u_matViewInverse;
uniform mat4 u_matViewProjection;
uniform vec3 u_lightPosition;
uniform vec3 u_eyePosition;
varying vec2 v_texcoord;
varying vec3 v_viewDirection;
varying vec3 v_lightDirection;
attribute vec4 a_vertex;
attribute vec2 a_texcoord0;
attribute vec3 a_normal;
attribute vec3 a_binormal;
attribute vec3 a_tangent;
void main(void)
{
// Transform eye vector into world space
vec3 eyePositionWorld =
(u_matViewInverse * vec4(u_eyePosition, 1.0)).xyz;
// Compute world space direction vector
vec3 viewDirectionWorld = eyePositionWorld - a_vertex.xyz;
// Transform light position into world space
vec3 lightPositionWorld =
(u_matViewInverse * vec4(u_lightPosition, 1.0)).xyz;
// Compute world space light direction vector
vec3 lightDirectionWorld = lightPositionWorld - a_vertex.xyz;
// Create the tangent matrix
mat3 tangentMat = mat3(a_tangent,
a_binormal,
a_normal);
// Transform the view and light vectors into tangent space
v_viewDirection = viewDirectionWorld * tangentMat;
v_lightDirection = lightDirectionWorld * tangentMat;
// Transform output position
gl_Position = u_matViewProjection * a_vertex;
// Pass through texture coordinate
v_texcoord = a_texcoord0.xy;
}We have two uniform matrices that we need as input to the vertex shader: u_matViewInverse and u_matViewProjection. The u_matViewInverse contains the inverse of the view matrix. This matrix is used to transform the light vector and eye vector (which are in view space) into world space. The first four statements in main are used to perform this transformation and compute the light vector and view vector in world space. The next step in the shader is to create a tangent matrix. The tangent space for the vertex is stored in three vertex attributes: a_normal, a_binormal, and a_tangent. These three vectors define the three coordinate axes of the tangent space for each vertex. We construct a 3 × 3 matrix out of these vectors to form the tangent matrix tangentMat.
The next step is to transform the view and direction vectors into tangent space by multiplying them by the tangentMat matrix. Remember, our purpose here is to get the view and direction vectors into the same space as the normals in the tangent-space normal map. By doing this transformation in the vertex shader, we avoid doing any transformations in the fragment shader. Finally, we compute the final output position and place it in gl_Position and pass the texture coordinate along to the fragment shader in v_texcoord.
Now we have the view and direction vector in view space and a texture coordinate passed in as varyings to the fragment shader. The next step is to actually light the fragments using the fragment shader as shown in Example 13-2.
Example 13-2. Per-Fragment Lighting Fragment Shader
precision mediump float;
uniform vec4 u_ambient;
uniform vec4 u_specular;
uniform vec4 u_diffuse;
uniform float u_specularPower;
uniform sampler2D s_baseMap;
uniform sampler2D s_bumpMap;
varying vec2 v_texcoord;
varying vec3 v_viewDirection;
varying vec3 v_lightDirection;
void main(void)
{
// Fetch basemap color
vec4 baseColor = texture2D(s_baseMap, v_texcoord);
// Fetch the tangent-space normal from normal map
vec3 normal = texture2D(s_bumpMap, v_texcoord).xyz;
// Scale and bias from [0, 1] to [-1, 1] and normalize
normal = normalize(normal * 2.0 - 1.0);
// Normalize the light direction and view direction
vec3 lightDirection = normalize(v_lightDirection);
vec3 viewDirection = normalize(v_viewDirection);
// Compute N.L
float nDotL = dot(normal, lightDirection);
// Compute reflection vector
vec3 reflection = (2.0 * normal * nDotL) - lightDirection;
// Compute R.V
float rDotV = max(0.0, dot(reflection, viewDirection));
// Compute Ambient term
vec4 ambient = u_ambient * baseColor;
// Compute Diffuse term
vec4 diffuse = u_diffuse * nDotL * baseColor;
// Compute Specular term
vec4 specular = u_specular * pow(rDotV, u_specularPower);
// Output final color
gl_FragColor = ambient + diffuse + specular;
}The first part of the fragment shader is a series of uniform declarations for the ambient, diffuse, and specular colors. These values are stored in the uniform variables u_ambient, u_diffuse, and u_specular. The shader is also configured with two samplers, s_baseMap and s_bumpMap, which are bound to a base color map and the normal map, respectively.
The first part of the fragment shader fetches the base color from the base map and the normal values from the normal map. As described earlier, the normal vector fetched from the texture map is scaled and biased and then normalized so that it is a unit vector with components in the [–1, 1] range. Next, the light vector and view vector are normalized and stored in lightDirection and viewDirection. The reason that normalization is necessary is because of the way varying variables are interpolated across a primitive. The varying variables are linearly interpolated across the primitive. When linear interpolation is done between two vectors, the results can become denormalized during interpolation. To compensate for this artifact, the vectors must be normalized in the fragment shader.
At this point in the fragment shader, we now have a normal, light vector, and direction vector all normalized and in the same space. This gives us the inputs that are needed to compute the lighting equations. The lighting computations performed in this shader are as follows:
Ambient = kAmbient × CBase
Diffuse = kDiffuse × N • L × CBase
Specular = kSpecular × pow(max(R • V, 0.0), kspecular Power
The k constants for ambient, diffuse, and specular colors come from the u_ambient, u_diffuse, and u_specular uniform variables. The CBase is the base color fetched from the base texture map. The dot product between the light vector and normal vector N • L is computed and stored in the nDotL variable in the shader. This value is used to compute the diffuse lighting term. Finally, the specular computation requires R,which is the reflection vector computed from the equation
R = 2 × N × (N • L) – L
Notice that the reflection vector also requires N • L, so the computation used for the diffuse lighting term can be reused in the reflection vector computation. Finally, the lighting terms are stored in the ambient, diffuse, and specular variables in the shader. These results are summed and finally stored in the gl_FragColor output variable. The result is a per-fragment lit object with normal data coming from the normal map.
Many variations are possible on per-fragment lighting. One common technique is to store the specular exponent in a texture along with a specular mask value. This allows the specular lighting to vary across a surface. The main purpose of this example is to give you an idea of the types of computations that are typically done for per-fragment lighting. The use of tangent space along with the computation of the lighting equations in the fragment shader is typical of many modern games. Of course, it is also possible to add additional lights, more material information, and much more.
The next rendering technique we cover—related to the previous technique—is performing environment mapping using a cubemap. The example we cover is the RenderMonkey workspace Chapter_13/Environment Mapping/ EnvironmentMapping.rfx. The results are shown in Figure 13-2.
The concept behind environment mapping is to render the reflection of the environment on an object. In Chapter 9, “Texturing,” we introduced you to cubemaps, which are commonly used to store environment maps. In the RenderMonkey example workspace, the environment of a mountain scene is stored in a cubemap. The way such cubemaps can be generated is by positioning a camera at the center of a scene and rendering along each of the positive and negative major axis directions using a 90-degree field of view. For reflections that change dynamically, one can render such a cubemap using a framebuffer object dynamically for each frame. For a static environment, this process can be done as a preprocess and the results stored in a static cubemap.
The vertex shader for the environment mapping example is provided in Example 13-3.
Example 13-3. Environment Mapping Vertex Shader
uniform mat4 u_matViewInverse;
uniform mat4 u_matViewProjection;
uniform vec3 u_lightPosition;
uniform vec3 u_eyePosition;
varying vec2 v_texcoord;
varying vec3 v_lightDirection;
varying vec3 v_normal;
varying vec3 v_binormal;
varying vec3 v_tangent;
attribute vec4 a_vertex;
attribute vec2 a_texcoord0;
attribute vec3 a_normal;
attribute vec3 a_binormal;
attribute vec3 a_tangent;
void main(void)
{
// Transform light position into world space
vec3 lightPositionWorld =
(u_matViewInverse * vec4(u_lightPosition, 1.0)).xyz;
// Compute world-space light direction vector
vec3 lightDirectionWorld = lightPositionWorld - a_vertex.xyz;
// Pass the world-space light vector to the fragment shader
v_lightDirection = lightDirectionWorld;
// Transform output position
gl_Position = u_matViewProjection * a_vertex;
// Pass through other attributes
v_texcoord = a_texcoord0.xy;
v_normal = a_normal;
v_binormal = a_binormal;
v_tangent = a_tangent;
}The vertex shader in this example is very similar to the previous per-fragment lighting example. The primary difference is that rather than transforming the light direction vector into tangent space, we keep the light vector in world space. The reason we must do this is because we ultimately want to fetch from the cubemap using a world-space reflection vector. As such, rather than transforming the light vectors into tangent space, we are going to transform the normal vector from tangent into world space. To do so, the vertex shader passes the normal, binormal, and tangent as varyings into the fragment shader so that a tangent matrix can be constructed.
The fragment shader listing for the environment mapping sample is provided in Example 13-4.
Example 13-4. Environment Mapping Fragment Shader
precision mediump float;
uniform vec4 u_ambient;
uniform vec4 u_specular;
uniform vec4 u_diffuse;
uniform float u_specularPower;
uniform sampler2D s_baseMap;
uniform sampler2D s_bumpMap;
uniform samplerCube s_envMap;
varying vec2 v_texcoord;
varying vec3 v_lightDirection;
varying vec3 v_normal;
varying vec3 v_binormal;
varying vec3 v_tangent;
void main(void)
{
// Fetch basemap color
vec4 baseColor = texture2D(s_baseMap, v_texcoord);
// Fetch the tangent-space normal from normal map
vec3 normal = texture2D(s_bumpMap, v_texcoord).xyz;
// Scale and bias from [0, 1] to [-1, 1]
normal = normal * 2.0 - 1.0;
// Construct a matrix to transform from tangent to world space
mat3 tangentToWorldMat = mat3(v_tangent,
v_binormal,
v_normal);
// Transform normal to world space and normalize
normal = normalize(tangentToWorldMat * normal);
// Normalize the light direction
vec3 lightDirection = normalize(v_lightDirection);
// Compute N.L
float nDotL = dot(normal, lightDirection);
// Compute reflection vector
vec3 reflection = (2.0 * normal * nDotL) - lightDirection;
// Use the reflection vector to fetch from the environment map
vec4 envColor = textureCube(s_envMap, reflection);
// Output final color
gl_FragColor = 0.25 * baseColor + envColor;
}In the fragment shader, you will notice that the normal vector is fetched from the normal map in the same way as in the per-fragment lighting example. The difference in this example is that rather than leaving the normal vector in tangent space, the fragment shader transforms the normal vector into world space. This is done by constructing the tangentToWorld matrix out of the v_tangent, v_binormal, and v_normal varying vectors and then multiplying that matrix by the fetched normal vector. The reflection vector is then calculated using the light direction vector and normal both in world space. The result of the computation is a reflection vector that is in world space, exactly what we need to fetch from the cubemap as an environment map. This vector is used to fetch into the environment map using the textureCube function with the reflection vector as a texture coordinate. Finally, the resultant gl_FragColor is written as a combination of the basemap color and the environment map color. The base color is attenuated by 0.25 for the purposes of this example so that the environment map is clearly visible.
This example shows the basics of environment mapping. This basic technique can be used to accomplish a large variety of effects. For example, one additional technique is attenuating the reflection using a fresnel term to more accurately model the reflection of light on a given material. As mentioned earlier, another common technique is to dynamically render a scene into a cubemap so that the environment reflection varies as an object moves through a scene and the scene itself changes. Using the basic technique we have shown you here, you can extend the technique to accomplish more advanced reflection effects.
The next example we cover is rendering a particle explosion using point sprites. The purpose of this example is to demonstrate how to animate a particle in a vertex shader and how to render particles using point sprites. The example we cover is the sample program in Chapter_13/ParticleSystem, the results of which are pictured in Figure 13-3.
Before diving into the code for this example, it’s helpful to cover at a high level the approach this sample uses. One of the goals of this sample was to show how to render a particle explosion without having any dynamic vertex data modified by the CPU. That is, with the exception of uniform variables, there are no changes to any of the vertex data as the explosion animates. To accomplish this goal, there are number of inputs that are fed into the shaders.
At initialization time, the program initializes the following values in a vertex array, one for each particle, based on a random value:
In addition, each explosion has several global settings that are passed in as uniforms:
Center position—The center of the explosion (the per-vertex positions are offset from this center).
Color—An overall color of the explosion.
Time—The current time in seconds.
With this information, the vertex and fragment shaders are completely responsible for the motion, fading, and rendering of the particles. Let’s begin by taking a look at the vertex shader code for the sample in Example 13-5.
Example 13-5. Particle System Vertex Shader
uniform float u_time;
uniform vec3 u_centerPosition;
attribute float a_lifetime;
attribute vec3 a_startPosition;
attribute vec3 a_endPosition;
varying float v_lifetime;
void main()
{
if(u_time <= a_lifetime)
{
gl_Position.xyz = a_startPosition + (u_time * a_endPosition);
gl_Position.xyz += u_centerPosition;
gl_Position.w = 1.0;
}
else
gl_Position = vec4(-1000, -1000, 0, 0);
v_lifetime = 1.0 - (u_time / a_lifetime);
v_lifetime = clamp(v_lifetime, 0.0, 1.0);
gl_PointSize = (v_lifetime * v_lifetime) * 40.0;
}The first input to the vertex shader is the uniform variable u_time. This variable is set to the current elapsed time in seconds by the application. The value is reset to 0.0 when the time exceeds the length of a single explosion. The next input to the vertex shader is the uniform variable u_centerPosition. This variable is set to the center location of the explosion at the start of a new explosion. The setup code for u_time and u_centerPosition is in the Update() function of the example program, which is provided in Example 13-6.
Example 13-6. Update Function for Particle System Sample
void Update (ESContext *esContext, float deltaTime)
{
UserData *userData = esContext->userData;
userData->time += deltaTime;
if(userData->time >= 1.0f)
{
float centerPos[3];
float color[4];
userData->time = 0.0f;
// Pick a new start location and color
centerPos[0] = ((float)(rand() % 10000) / 10000.0f) - 0.5f;
centerPos[1] = ((float)(rand() % 10000) / 10000.0f) - 0.5f;
centerPos[2] = ((float)(rand() % 10000) / 10000.0f) - 0.5f;
glUniform3fv(userData->centerPositionLoc, 1, ¢erPos[0]);
// Random color
color[0] = ((float)(rand() % 10000) / 20000.0f) + 0.5f;
color[1] = ((float)(rand() % 10000) / 20000.0f) + 0.5f;
color[2] = ((float)(rand() % 10000) / 20000.0f) + 0.5f;
color[3] = 0.5;
glUniform4fv(userData->colorLoc, 1, &color[0]);
}
// Load uniform time variable
glUniform1f(userData->timeLoc, userData->time);
}As you can see, the Update() function resets the time after one second elapses and then sets up a new center location and time for another explosion. The function also keeps the u_time variable up-to-date each frame.
Returning to the vertex shader, the vertex attribute inputs to the vertex shader are the particle lifetime, particle start position, and end position. These are all initialized to randomly seeded values in the Init function in the program. The body of the vertex shader first checks to see whether a particle’s lifetime has expired. If so, the gl_Position variable is set to the value ( –1000, –1000 ), which is just a quick way of forcing the point to be off the screen. Because the point will be clipped, all of the subsequent processing for the expired point sprites can be skipped. If the particle is still alive, its position is set to be a linear interpolated value between the start and end positions. Next, the vertex shader passes the remaining lifetime of the particle down into the fragment shader in the varying variable v_lifetime. The lifetime will be used in the fragment shader to fade the particle as it ends its life. The final piece of the vertex shader sets the point size to be based on the remaining lifetime of the particle by setting the gl_PointSize built-in variable. This has the effect of scaling the particles down as they reach the end of their life.
The fragment shader code for the example program is provided in Example 13-7.
The first input to the fragment shader is the u_color uniform variable, which is set up at the beginning of each explosion by the Update function. Next, the v_lifetime varying variable set by the vertex shader is declared in the fragment shader. There is also a sampler declared to which a 2D texture image of smoke is bound.
The fragment shader itself is relatively simple. The texture fetch uses the gl_PointCoord variable as a texture coordinate. This is a special variable for point sprites that is set to fixed values for the corners of the point sprite (this was described in Chapter 7, “Primitive Assembly and Rasterization,” on drawing primitives). One could also extend the fragment shader to rotate the point sprite coordinates if rotation of the sprite was required. This requires extra fragment shader instructions, but increases the flexibility of the point sprite.
The texture color is attenuated by the u_color variable and the alpha is attenuated by the particle lifetime. The application also enables alpha blending with the following blend function.
glEnable ( GL_BLEND ); glBlendFunc ( GL_SRC_ALPHA, GL_ONE );
The result of this is that the alpha produced in the fragment shader is modulated with the fragment color. This value is then added into whatever values are stored in the destination of the fragment. The result is to get an additive blend effect for the particle system. Note that various particle effects will use different alpha blending modes to accomplish the desired effect.
The code to actually draw the particles is given in Example 13-8.
Example 13-8. Draw Function for Particle System Sample
void Draw(ESContext *esContext)
{
UserData *userData = esContext->userData;
// Set the viewport
glViewport(0, 0, esContext->width, esContext->height);
// Clear the color buffer
glClear(GL_COLOR_BUFFER_BIT);
// Use the program object
glUseProgram(userData->programObject);
// Load the vertex attributes
glVertexAttribPointer(userData->lifetimeLoc, 1, GL_FLOAT,
GL_FALSE, PARTICLE_SIZE * sizeof(GLfloat),
userData->particleData);
glVertexAttribPointer(userData->endPositionLoc, 3, GL_FLOAT,
GL_FALSE, PARTICLE_SIZE * sizeof(GLfloat),
&userData->particleData[1]);
glVertexAttribPointer(userData->startPositionLoc, 3, GL_FLOAT,
GL_FALSE, PARTICLE_SIZE * sizeof(GLfloat),
&userData->particleData[4]);
glEnableVertexAttribArray(userData->lifetimeLoc);
glEnableVertexAttribArray(userData->endPositionLoc);
glEnableVertexAttribArray(userData->startPositionLoc);
// Blend particles
glEnable(GL_BLEND);
glBlendFunc(GL_SRC_ALPHA, GL_ONE);
// Bind the texture
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, userData->textureId);
glEnable(GL_TEXTURE_2D);
// Set the sampler texture unit to 0
glUniform1i(userData->samplerLoc, 0);
glDrawArrays(GL_POINTS, 0, NUM_PARTICLES);
eglSwapBuffers(esContext->eglDisplay, esContext->eglSurface);
}The Draw function begins by setting the viewport and clearing the screen. It then selects the program object to use and loads the vertex data using glVertexAttribPointer. Note that because the values of the vertex array never change, this example could have used vertex buffer objects rather than client-side vertex arrays. In general, this is recommended for any vertex data that does not change because it reduces the vertex bandwidth used. Vertex buffer objects were not used in this example merely to keep the code a bit simpler. After setting the vertex arrays, the function enables the blend function, binds the smoke texture, and then uses glDrawArrays to draw the particles.
Unlike triangles, there is no connectivity for point sprites, so using glDrawElements does not really provide any advantage for rendering point sprites in this example. However, often particle systems need to be sorted by depth from back to front to get proper alpha blending results. In such cases, one approach that can be used is to sort the element array to modify the draw order. This is an efficient approach because it requires minimal bandwidth across the bus per-frame (only the index data need be changed, which is almost always smaller than the vertex data).
This example has shown you a number of techniques that can be useful in rendering particle systems using point sprites. The particles were animated entirely on the GPU using the vertex shader. The sizes of the particles were attenuated based on particle lifetime using the gl_PointSize variable. In addition, the point sprites were rendered with a texture using the gl_PointCoord built-in texture coordinate variable. These are the fundamental elements needed to implement a particle system using OpenGL ES 2.0.
The next example we cover is image postprocessing. Using a combination of framebuffer objects and shaders, it is possible to perform a wide variety of image postprocessing techniques. The first example we cover is the Simple Bloom effect in the RenderMonkey workspace in Chapter_13/PostProcess, results of which are pictured in Figure 13-4.
This example renders a single textured cube into a framebuffer object and then uses the color attachment as a texture in a subsequent pass. A full-screen quad is drawn to the screen using the rendered texture as a source. A fragment shader is run over the full-screen quad, which performs a blur filter. In general, many types of postprocess techniques can be accomplished using this pattern:
Render the scene into an off-screen framebuffer object (FBO).
Bind the FBO texture as a source and render a full-screen quad to the screen.
Execute a fragment shader that performs filtering across the quad.
Some algorithms require performing multiple passes over an image and some require more complicated inputs. However, the general idea is to use a fragment shader over a full-screen quad that performs a postprocessing algorithm.
The fragment shader used on the full-screen quad in the blurring example is provided in Example 13-9.
Example 13-9. Blur Fragment Shader
precision mediump float;
uniform sampler2D renderTexture;
varying vec2 v_texCoord;
uniform float u_blurStep;
void main(void)
{
vec4 sample0,
sample1,
sample2,
sample3;
float step = u_blurStep / 100.0;
sample0 = texture2D(renderTexture,
vec2(v_texCoord.x - step,
v_texCoord.y - step));
sample1 = texture2D(renderTexture,
vec2(v_texCoord.x + step,
v_texCoord.y + step));
sample2 = texture2D(renderTexture,
vec2(v_texCoord.x + step,
v_texCoord.y - step));
sample3 = texture2D(renderTexture,
vec2(v_texCoord.x - step,
v_texCoord.y + step));
gl_FragColor = (sample0 + sample1 + sample2 + sample3) / 4.0;
}This shader begins by computing the step variable that is based on the u_blurStep uniform variable. The step is used to determine how much to offset the texture coordinate when fetching samples from the image. Overall, four different samples are taken from the image and they are averaged together at the end of the shader. The step is used to offset the texture coordinate in four directions such that four samples in each diagonal direction from the center are taken. The larger the step, the more the image is blurred. One possible optimization to this shader would be to compute the offset texture coordinates in the vertex shader and pass them into varyings in the fragment shader. This would reduce the amount of computation done per fragment.
Now that we have looked at a simple image postprocessing technique, let’s take a look at a slightly more complicated one. Using the blurring technique we introduced in the previous example, we can implement an effect known as light bloom. Light bloom is what happens when the eye views a bright light contrasted with a darker surface. The effect is that the light color bleeds into the darker surface. The example we cover is the Bloom effect in the RenderMonkey workspace in Chapter_13/PostProcess, the results of which are pictured in Figure 13-5.
As you can see from the screenshot, the background color bleeds over the car model. The algorithm works as follows:
Clear an off-screen render target (rt0) and draw the object in black.
Blur the off-screen render target (rt0) into another render target (rt1) using a blur step of 1.0.
Blur the off-screen render target (rt1) back into original render target (rt0) using a blur step of 2.0.
NOTE: For more blur, repeat steps 2 and 3 for the amount of blur, increasing the blur step each time
Render the object to the backbuffer.
Blend the final render target with the backbuffer.
The process this algorithm uses is illustrated in Figure 13-6, which shows each of the steps that go into producing the final image.
As you can see from Figure 13-6, the object is first rendered in black to the render target. That render target is then blurred into a second render target in the next pass. The blurred render target is then blurred again with an expanded blur kernel back into the original render target. At the end, that blurred render target is blended with the original scene. The concept of using two render targets to do successive blurring is often referred to as ping-ponging. The amount of bloom can be increased by ping-ponging the blur targets over and over. The shader code for the blur steps is the same as from the previous example. The only difference is that the blur step is being increased for each pass.
There are a large variety of other image postprocessing algorithms that can be performed using a combination of FBOs and shaders. Some other common techniques include tone mapping, selective blurring, distortion, screen transitions, and depth of field. Using the techniques we have shown you here, you can start to implement other postprocessing algorithms using shaders.
A common technique that is used in many effects such as shadow mapping and reflections is the use of projective texturing. To introduce you to the use of projective texturing, we cover an example of rendering a projective spotlight. Most of the complication in using projective texturing is in the mathematics that goes into calculating the projective texture coordinates. The method we show you here would also be the same method you would use in shadow mapping or reflections. The example we cover is the projective spotlight RenderMonkey workspace in Chapter_13/ProjectiveSpotlight, the results of which are pictured in Figure 13-7.
The example uses the 2D texture image pictured in Figure 13-8 and applies it to the surface of terrain geometry using projective texturing. Projective spotlights were a very common technique used to emulate per-pixel spotlight falloff before shaders were introduced to GPUs. Projective spotlights can still provide an attractive solution because of the high level of efficiency. Applying the projective texture only takes a single texture fetch instruction in the fragment shader and some setup in the vertex shader. In addition, the 2D texture image that is projected can contain really any picture, so there are many possible effects that can be accomplished.
So what exactly do we mean by projective texturing? At its most basic, projective texturing is the use of a 3D texture coordinate to look up into a 2D texture image. The (s, t) coordinates are divided by the (r) coordinate such that a texel is fetched using (s/r, t/r). There is a special built-in function in the OpenGL ES Shading Language to do projective texturing called texture2DProj.
| |
| a sampler bound to a texture unit specifying the texture to fetch from |
| a 3D texture coordinate used to fetch from the texture map. The (x, y) arguments are divided by (z) such that the fetch occurs at (x/z, y/z) |
| an optional LOD bias to apply |
The idea behind projective lighting is to transform the position of an object into the projective view space of a light. The projective light space position, after application of a scale and bias, can then be used as a projective texture coordinate. The vertex shader in the RenderMonkey example workspace does the work of transforming the position into the projective view space of a light.
There are three matrices that we need to transform the position into projective view space of the light and get a projective texture coordinate:
Light projection—. projection matrix of the light source using the field of view, aspect ratio, and near and far planes of the light.
Light view—. The view matrix of the light source. This would be constructed just as if the light were a camera.
Bias matrix—. A matrix that transforms the light-space projected position into a 3D projective texture coordinate.
The light projection matrix would be constructed just like any other projection matrix using the light’s parameters for field of view (FOV), aspect ratio (aspect), and near (zNear) and far plane (zFar) distances.
The light view matrix is constructed by using the three primary axis directions that define the light’s view axes and the light’s position. We refer to the axes as the right, up, and look vectors.
After transforming the object’s position by the view and projection matrices, we must then turn the coordinates into projective texture coordinates. This is accomplished using a 3 × 3 bias matrix on the (x, y, z) components of the position in projective light space. The bias matrix does a linear transformation to go from the [–1, 1] range to the [0, 1] range. Having the coordinates in the [0, 1] range is necessary for the values to be used as texture coordinates.
Typically, the matrix to transform the position into a projective texture coordinate would be computed on the CPU by concatenating the projection, view, and bias matrices together (using a 4 × 4 version of the bias matrix). This would then be loaded into a single uniform matrix that could transform the position in the vertex shader. However, in the example, we perform this computation in the vertex shader for illustrative purposes.
Now that we have covered the basic mathematics, we can examine the vertex shader in Example 13-10.
Example 13-10. Projective Texturing Vertex Shader
uniform float u_time_0_X;
uniform mat4 u_matProjection;
uniform mat4 u_matViewProjection;
attribute vec4 a_vertex;
attribute vec2 a_texCoord0;
attribute vec3 a_normal;
varying vec2 v_texCoord;
varying vec3 v_projTexCoord;
varying vec3 v_normal;
varying vec3 v_lightDir;
void main(void)
{
gl_Position = u_matViewProjection * a_vertex;
v_texCoord = a_texCoord0.xy;
// Compute a light position based on time
vec3 lightPos;
lightPos.x = cos(u_time_0_X);
lightPos.z = sin(u_time_0_X);
lightPos.xz = 100.0 * normalize(lightPos.xz);
lightPos.y = 100.0;
// Compute the light coordinate axes
vec3 look = -normalize(lightPos);
vec3 right = cross(vec3(0.0, 0.0, 1.0), look);
vec3 up = cross(look, right);
// Create a view matrix for the light
mat4 lightView = mat4(right, dot(right, -lightPos),
up, dot(up, -lightPos),
look, dot(look, -lightPos),
0.0, 0.0, 0.0, 1.0);
// Transform position into light view space
vec4 objPosLight = a_vertex * lightView;
// Transform position into projective light view space
objPosLight = u_matProjection * objPosLight;
// Create bias matrix
mat3 biasMatrix = mat3(0.5, 0.0, 0.5,
0.0, -0.5, 0.5,
0.0, 0.0, 1.0);
// Compute projective texture coordinates
v_projTexCoord = objPosLight.xyz * biasMatrix;
v_lightDir = normalize(a_vertex.xyz - lightPos);
v_normal = a_normal;
}The first operation this shader does is to transform the position by the u_matViewProjection matrix and output the texture coordinate for the basemap to the v_texCoord varying. The next thing the shader does is to compute a position for the light based on time. This bit of the code can really be ignored, but it was added to animate the light in the vertex shader. In a typical application, this would be done on the CPU and not in the shader.
Based on the position of the light, the vertex shader then computes the three coordinate axis vectors for the light into the look, right, and up variables. Those vectors are used to create a view matrix for the light in the lightView variable using the equations previously described. The input position for the object is then transformed by the lightView matrix, which transforms the position into light space. The next step is to use the perspective matrix to transform the light space position into projected light space. Rather than creating a new perspective matrix for the light, this example uses the u_matProjection matrix for the camera. Typically, a real application would want to create its own projection matrix for the light based on how big the cone angle and falloff distance is.
Once the position is in projective light space, a biasMatrix is then created to transform the position into a projective texture coordinate. The final projective texture coordinate is stored in the vec3 varying variable v_projTexCoord. In addition, the vertex shader also passes the light direction and normal vectors into the fragment shader in the v_lightDir and v_normal varyings. These will be used to determine if a fragment is facing the light source or not in order to mask off the projective texture for fragments facing away from the light.
The fragment shader performs the actual projective texture fetch that applies the projective spotlight texture to the surface.
Example 13-11. Projective Texturing Fragment Shader
precision mediump float;
uniform sampler2D baseMap;
uniform sampler2D spotLight;
varying vec2 v_texCoord;
varying vec3 v_projTexCoord;
varying vec3 v_normal;
varying vec3 v_lightDir;
void main(void)
{
// Projective fetch of spotlight
vec4 spotLightColor = texture2DProj(spotLight, v_projTexCoord);
// Basemap
vec4 baseColor = texture2D(baseMap, v_texCoord);
// Compute N.L
float nDotL = max(0.0, -dot(v_normal, v_lightDir));
gl_FragColor = spotLightColor * baseColor * 2.0 * nDotL;
}The first operation the fragment shader performs is to do the projective texture fetch using texture2DProj. As you can see, the projective texture coordinate that was computed during the vertex shader and passed in the varying v_projTexCoord is used to perform the projective texture fetch. The wrap modes for the projective texture are set to GL_CLAMP_TO_EDGE and the min/mag filters are both set to GL_LINEAR. The fragment shader then fetches the color from the basemap using the v_texCoord varying. Next, the shader computes the dot product between the light direction and normal vector. This is used to attenuate the final color so that the projective spotlight is not applied to fragments that are facing away from the light. Finally, all of the components are multiplied together (and scaled by 2.0 to increase the brightness). This gives us our final image from Figure 13-7 of the terrain lit by the projective spotlight.
As mentioned at the beginning of this section, the important takeaway from this example is the set of computations that go into computing a projective texture coordinate. The computation shown here would be the exact same computation that you would use to produce a coordinate to fetch from a shadow map. Similarly, rendering reflections with projective texturing requires that you transform the position into the projective view space of the reflection camera. You would do the same thing we have done here, but substitute the light matrices for the reflection camera matrices. Projective texturing is a very powerful tool in advanced effects and you should now understand the basics of how to use it.
The next rendering technique we cover is using a 3D texture for noise. In Chapter 9, we introduced you to the basics of 3D textures. As you will recall, a 3D texture is essentially a stack of 2D texture slices representing a 3D volume. There are many possible uses of 3D textures and one of them is the representation of noise. In this section, we show an example of using a 3D volume of noise to create a wispy fog effect. This example builds on the linear fog example from Chapter 10, “Fragment Shaders.” The example we cover is the RenderMonkey workspace in Chapter_13/Noise3D, the results of which are pictured in Figure 13-9.
The application of noise is a very common technique in a large variety of 3D effects. The OpenGL Shading Language (not OpenGL ES Shading Language) had functions for computing noise in one, two, three, or four dimensions. The functions return a pseudorandom continuous noise value that is repeatable based on the input value. However, the problem with these functions was that they were very expensive to implement. Most programmable GPUs did not implement noise functions natively in hardware, which meant the noise computations had to be implemented using shader instructions (or worse, in software on the CPU). It takes a lot of shader instructions to implement these noise functions, so the performance was too slow to be used in most real-time fragment shaders. This, by the way, is the reason that the OpenGL ES working group decided to drop noise from the OpenGL ES shading language (although vendors are still free to expose it through an extension).
Although computing noise in the fragment shader is prohibitively expensive, we can work around the problem using a 3D texture. It is possible to easily produce acceptable quality noise by precomputing the noise and placing the results in a 3D texture. There are a number of algorithms that can be used to generate noise. A list of references and links described at the end of this chapter can be used to read about the various noise algorithms. We discuss a specific algorithm that generates a lattice-based gradient noise. Ken Perlin’s noise function (Perlin, 1985) is a lattice-based gradient noise. This is a very commonly used method for generating noise. For example, a lattice-based gradient noise is implemented by the noise function in the Renderman shading language.
The gradient noise algorithm takes a 3D coordinate as input and returns a floating-point noise value. To generate this noise value given an input (x, y, z), we map the x, y, and z values to appropriate integer locations in a lattice. The number of cells in a lattice is programmable and for our implementation is set to 256 cells. For each cell in the lattice, we need to generate and store a pseudorandom gradient vector. Example 13-12 describes how these gradient vectors are generated.
Example 13-12. Generating Gradient Vectors
// permTable describes a random permutation of
// 8-bit values from 0 to 255.
static unsigned char permTable[256] = {
0xE1, 0x9B, 0xD2, 0x6C, 0xAF, 0xC7, 0xDD, 0x90,
0xCB, 0x74, 0x46, 0xD5, 0x45, 0x9E, 0x21, 0xFC,
0x05, 0x52, 0xAD, 0x85, 0xDE, 0x8B, 0xAE, 0x1B,
0x09, 0x47, 0x5A, 0xF6, 0x4B, 0x82, 0x5B, 0xBF,
0xA9, 0x8A, 0x02, 0x97, 0xC2, 0xEB, 0x51, 0x07,
0x19, 0x71, 0xE4, 0x9F, 0xCD, 0xFD, 0x86, 0x8E,
0xF8, 0x41, 0xE0, 0xD9, 0x16, 0x79, 0xE5, 0x3F,
0x59, 0x67, 0x60, 0x68, 0x9C, 0x11, 0xC9, 0x81,
0x24, 0x08, 0xA5, 0x6E, 0xED, 0x75, 0xE7, 0x38,
0x84, 0xD3, 0x98, 0x14, 0xB5, 0x6F, 0xEF, 0xDA,
0xAA, 0xA3, 0x33, 0xAC, 0x9D, 0x2F, 0x50, 0xD4,
0xB0, 0xFA, 0x57, 0x31, 0x63, 0xF2, 0x88, 0xBD,
0xA2, 0x73, 0x2C, 0x2B, 0x7C, 0x5E, 0x96, 0x10,
0x8D, 0xF7, 0x20, 0x0A, 0xC6, 0xDF, 0xFF, 0x48,
0x35, 0x83, 0x54, 0x39, 0xDC, 0xC5, 0x3A, 0x32,
0xD0, 0x0B, 0xF1, 0x1C, 0x03, 0xC0, 0x3E, 0xCA,
0x12, 0xD7, 0x99, 0x18, 0x4C, 0x29, 0x0F, 0xB3,
0x27, 0x2E, 0x37, 0x06, 0x80, 0xA7, 0x17, 0xBC,
0x6A, 0x22, 0xBB, 0x8C, 0xA4, 0x49, 0x70, 0xB6,
0xF4, 0xC3, 0xE3, 0x0D, 0x23, 0x4D, 0xC4, 0xB9,
0x1A, 0xC8, 0xE2, 0x77, 0x1F, 0x7B, 0xA8, 0x7D,
0xF9, 0x44, 0xB7, 0xE6, 0xB1, 0x87, 0xA0, 0xB4,
0x0C, 0x01, 0xF3, 0x94, 0x66, 0xA6, 0x26, 0xEE,
0xFB, 0x25, 0xF0, 0x7E, 0x40, 0x4A, 0xA1, 0x28,
0xB8, 0x95, 0xAB, 0xB2, 0x65, 0x42, 0x1D, 0x3B,
0x92, 0x3D, 0xFE, 0x6B, 0x2A, 0x56, 0x9A, 0x04,
0xEC, 0xE8, 0x78, 0x15, 0xE9, 0xD1, 0x2D, 0x62,
0xC1, 0x72, 0x4E, 0x13, 0xCE, 0x0E, 0x76, 0x7F,
0x30, 0x4F, 0x93, 0x55, 0x1E, 0xCF, 0xDB, 0x36,
0x58, 0xEA, 0xBE, 0x7A, 0x5F, 0x43, 0x8F, 0x6D,
0x89, 0xD6, 0x91, 0x5D, 0x5C, 0x64, 0xF5, 0x00,
0xD8, 0xBA, 0x3C, 0x53, 0x69, 0x61, 0xCC, 0x34,
};
#define NOISE_TABLE_MASK 255
// lattice gradients 3D noise
static float gradientTable[256*3];
#define FLOOR(x) ((int)(x) - ((x) < 0 && (x) != (int)(x)))
#define smoothstep(t) (t * t * (3.0f - 2.0f * t))
#define lerp(t, a, b) (a + t * (b - a))
void
initNoiseTable()
{
long rnd;
int i;
double a;
float x, y, z, r, theta;
float gradients[256*3];
unsigned int *p, *psrc;
srandom(0);
// build gradient table for 3D noise
for(i=0; i<256; i++)
{
// calculate 1 - 2 * random number
rnd = random();
a = (random() & 0x7FFFFFFF) / (double) 0x7FFFFFFF;
z = (float)(1.0 - 2.0 * a);
r = (float)sqrt(1.0 - z * z); // r is radius of circle
rnd = random();
a = (float)((random() & 0x7FFFFFFF) / (double) 0x7FFFFFFF);
theta = (float)(2.0 * M_PI * a);
x = (float)(r * (float)cos(a));
y = (float)(r * (float)sin(a));
gradients[i*3] = x;
gradients[i*3+1] = y;
gradients[i*3+2] = z;
}
// use the index in the permutation table to load the
// gradient values from gradients to gradientTable
p = (unsigned int *)gradientTable;
psrc = (unsigned int *)gradients;
for (i=0; i<256; i++)
{
int indx = permTable[i];
p[i*3] = psrc[indx*3];
p[i*3+1] = psrc[indx*3+1];
p[i*3+2] = psrc[indx*3+2];
}
}Example 13-13 describes how the gradient noise is calculated using the pseudorandom gradient vectors and an input 3D coordinate.
Example 13-13. 3D Noise
//
// generate the value of gradient noise for a given lattice point
//
// (ix, iy, iz) specifies the 3D lattice position
// (fx, fy, fz) specifies the fractional part
//
static float
glattice3D(int ix, int iy, int iz, float fx, float fy, float fz)
{
float *g;
int indx, y, z;
z = permTable[iz & NOISE_TABLE_MASK];
y = permTable[(iy + z) & NOISE_TABLE_MASK];
indx = (ix + y) & NOISE_TABLE_MASK;
g = &gradientTable[indx*3];
return (g[0]*fx + g[1]*fy + g[2]*fz);
}
//
// generate the 3D noise value
// f describes input (x, y, z) position for which the noise value
// needs to be computed. noise3D returns the scalar noise value
//
float
noise3D(float *f)
{
int ix, iy, iz;
float fx0, fx1, fy0, fy1, fz0, fz1;
float wx, wy, wz;
float vx0, vx1, vy0, vy1, vz0, vz1;
ix = FLOOR(f[0]);
fx0 = f[0] - ix;
fx1 = fx0 - 1;
wx = smoothstep(fx0);
iy = FLOOR(f[1]);
fy0 = f[1] - iy;
fy1 = fy0 - 1;
wy = smoothstep(fy0);
iz = FLOOR(f[2]);
fz0 = f[2] - iz;
fz1 = fz0 - 1;
wz = smoothstep(fz0);
vx0 = glattice3D(ix, iy, iz, fx0, fy0, fz0);
vx1 = glattice3D(ix+1, iy, iz, fx1, fy0, fz0);
vy0 = lerp(wx, vx0, vx1);
vx0 = glattice3D(ix, iy+1, iz, fx0, fy1, fz0);
vx1 = glattice3D(ix+1, iy+1, iz, fx1, fy1, fz0);
vy1 = lerp(wx, vx0, vx1);
vz0 = lerp(wy, vy0, vy1);
vx0 = glattice3D(ix, iy, iz+1, fx0, fy0, fz1);
vx1 = glattice3D(ix+1, iy, iz+1, fx1, fy0, fz1);
vy0 = lerp(wx, vx0, vx1);
vx0 = glattice3D(ix, iy+1, iz+1, fx0, fy1, fz1);
vx1 = glattice3D(ix+1, iy+1, iz+1, fx1, fy1, fz1);
vy1 = lerp(wx, vx0, vx1);
vz1 = lerp(wy, vy0, vy1);
return lerp(wz, vz0, vz1);;
}The noise3D function returns a value between –1.0 and 1.0. The value of gradient noise is always 0 at the integer lattice points. For points in between, trilinear interpolation of gradient values across the eight integer lattice points that surround the point is used to generate the scalar noise value. Figure 13-10 shows a 2D slice of the gradient noise using the preceding algorithm.
Once we have created a 3D noise volume, it is very easy to use it for a variety of different effects. In the case of the wispy fog effect, the idea is simple: scroll the 3D noise texture in all three dimensions based on time and use the value from the texture to distort the fog factor. Let’s take a look at the fragment shader in Example 13-14.
Example 13-14. Noise Distorted Fog Fragment Shader
#extension GL_OES_texture_3D : enable
precision mediump float;
uniform vec4 u_fogColor;
uniform float u_fogMaxDist;
uniform float u_fogMinDist;
uniform float u_time;
uniform sampler2D baseMap;
uniform sampler3D noiseVolume;
varying vec2 v_texCoord;
varying float v_eyeDist;
float computeLinearFogFactor()
{
float factor;
// Compute linear fog equation
factor = (u_fogMaxDist - v_eyeDist) /
(u_fogMaxDist - u_fogMinDist);
return factor;
}
void main(void)
{
float fogFactor = computeLinearFogFactor();
vec4 baseColor = texture2D(baseMap, v_texCoord);
// Distort fog factor by noise
vec3 noiseCoord;
noiseCoord.xy = v_texCoord.xy - (u_time * 0.1);
noiseCoord.z = u_time * 0.1;
fogFactor += texture3D(noiseVolume, noiseCoord).r;
fogFactor = clamp(fogFactor, 0.0, 1.0);
// Compute final color as a lerp with fog factor
gl_FragColor = baseColor * fogFactor +
u_fogColor * (1.0 - fogFactor);
}The first thing this shader does is to enable the 3D texture extension using the #extension mechanism. If an implementation does not support the GL_OES_texture_3D extension, then this shader will fail to compile. This shader is very similar to our linear fog example in Chapter 10. The primary difference is that the linear fog factor is distorted by the 3D noise texture. The shader computes a 3D texture coordinate based on time and places it in noiseCoord. The u_time uniform variable is tied to the current time and is updated each frame. The 3D texture is set up with wrap modes in s, t, and r of GL_REPEAT so that the noise volume scrolls smoothly on the surface. The (s, t) coordinates are based on the coordinates for the base texture and are scrolled in both directions. The (r) coordinate is based purely on time so that it is continuously scrolled.
The 3D texture is a single-channel (GL_LUMINANCE) texture so only the red component of the texture is used (the green and blue channels have the same value as the red channel). The value fetched from the volume is added to the computed fogFactor and then used to linearly interpolate between the fog color and base color. The result is a wispy fog appearing to roll in from the hilltops. The speed of the fog can be increased easily by applying a scale to the u_time variable when scrolling the 3D texture coordinates.
There are a number of different effects you can achieve by using a 3D texture to represent noise. Some other examples of using noise include representing dust in a light volume, adding a more natural appearance to a procedural texture, and simulating water waves. Using a 3D texture is a great way to save performance and still achieve high-quality visual effects. It is unlikely that you can expect handheld devices to compute noise functions in the fragment shader and have enough performance to run at a high frame rate. As such, having a precomputed noise volume will be a very valuable trick to have in your toolkit for creating effects.
The next topic we cover is the generation of procedural textures. Textures are typically described as a 2D image, a cubemap, or a 3D image. These images store color or depth values. Built-in functions defined in the OpenGL ES shading language take a texture coordinate, a texture object referred to as a sampler, and return a color or depth value. Procedural texturing refers to textures that are described as a procedure instead of as an image. The procedure describes the algorithm that will generate a texture color or depth value given a set of inputs.
The following are some of the benefits of procedural textures:
Much more compact representation versus a stored texture image. All you need to store is the code that describes the procedural texture. This will typically be much smaller in size over a stored image.
Procedural textures, unlike stored images, have no fixed resolution. They can therefore be applied to the surface without loss of detail. This means that we will not see issues such as reduced detail as we zoom onto a surface that uses a procedural texture. We will, however, encounter these issues when using a stored texture image because of its fixed resolution.
The disadvantages of procedural textures are as follows:
The procedural texture code could require quite a few instructions and could result in fragment shaders that might not compile because of fragment shader size restrictions. As OpenGL ES 2.0 primarily runs on handheld and embedded devices, this can be a serious problem limiting the algorithms developers can use for procedural effects. This problem should get better as handheld devices become more capable in their ability to do graphics.
Although the procedural texture might have a smaller footprint than a stored texture, it might take a lot more cycles to execute the procedural texture versus doing a lookup in the stored texture. With procedural textures you are dealing with instruction bandwidth versus memory bandwidth for stored textures. Both the instruction and memory bandwidth are at a premium on handheld devices and a developer must carefully choose which approach to take.
Repeatability might be hard to achieve. Differences in arithmetic precision and in the implementation of built-in functions across OpenGL ES 2.0 implementations can make this a difficult problem to deal with.
Procedural textures can have serious aliasing artifacts. Most of these artifacts can be resolved but they result in additional instructions to the procedural texture code, which can impact the performance of a shader.
The decision whether to use a procedural or a stored texture will need to be made based on careful analysis of the performance and memory bandwidth requirements of each.
We now look at a simple example that demonstrates procedural textures. We are very familiar with how we would use a checkerboard texture image to draw a checkerboard pattern on an object. We now look at a procedural texture implementation that renders a checkerboard pattern on an object. The example we cover is the Checker.rfx RenderMonkey workspace in Chapter_13/ProceduralTextures. Examples 13-15 and 13-16 describe the vertex and fragment shader that implements the checkerboard texture procedurally.
Example 13-15. Checker Vertex Shader
uniform mat4 mvp_matrix; // combined modelview + projection
matrix
attribute vec4 a_position; // input vertex position
attribute vec2 a_st; // input texture coordinate
varying vec2 v_st; // output texture coordinate
void
main()
{
v_st = a_st;
gl_Position = mvp_matrix * a_position;
}The vertex shader code in Example 13-15 is really straightforward. It transforms the position using the combined model view and projection matrix and passes the texture coordinate (a_st) to the fragment shader as a varying variable (v_st).
Example 13-16. Checker Fragment Shader
#ifdef GL_ES
precision highp float;
#endif
// frequency of the checkerboard pattern
uniform int u_frequency;
// alternate colors that make the checkerboard pattern
uniform vec4 u_color0;
uniform vec4 u_color1;
varying vec2 v_st;
void
main()
{
vec2 tcmod = mod(v_st * float(u_frequency), 1.0);
if(tcmod.s < 0.5)
{
if(tcmod.t < 0.5)
gl_FragColor = u_color1;
else
gl_FragColor = u_color0;
}
else
{
if(tcmod.t < 0.5)
gl_FragColor = u_color0;
else
gl_FragColor = u_color1;
}
gl_FragColor = mix(color1, color0, delta);
}The fragment shader code in Example 13-16 uses the v_st texture coordinate to draw the texture pattern. Although easy to understand, the fragment shader might have poor performance because the multiple conditional checks are being done on values that can be different over fragments being executed in parallel. This can impact performance as the number of vertices or fragments executed in parallel by the GPU is reduced. Example 13-17 is a version of the fragment shader that no longer uses any conditional checks.
Example 13-17. Checker Fragment Shader
#ifdef GL_ES
precision highp float;
#endif
// frequency of the checkerboard pattern
uniform int u_frequency;
// alternate colors that make the checkerboard pattern
uniform vec4 u_color0;
uniform vec4 u_color1;
varying vec2 v_st;
void
main()
{
vec2 texcoord = mod(floor(v_st * float(u_frequency * 2)), 2.0);
float delta = abs(texcoord.x - texcoord.y);
gl_FragColor = mix(u_color1, u_color0, delta);
}Figure 13-11 shows the checkerboard image rendered using the fragment shader in Example 13-17 with u_frequency = 10, u_color0 set to black, and u_color1 set to white.
As you can see this was really easy to implement. We do see quite a bit of aliasing, which is never acceptable. With a texture checker image, aliasing issues are overcome using mipmapping and applying preferably a trilinear or bilinear filter. We now look at how to render an antialiased checkerboard pattern. To antialias a procedural texture, we need the built-in functions implemented by the GL_OES_standard_derivatives extension. Refer to Appendix B for a detailed description of the built-in functions implemented by this extension.
In Advanced RenderMan: Creating CGI for Motion Pictures, Anthony Apodaca and Larry Gritz give a very thorough explanation on how to implement analytic antialiasing of procedural textures. We use the techniques described in this book to implement the antialiased checker fragment shader. Example 13-18 describes the antialiased checker fragment shader code from the CheckerAA.rfx RenderMonkey workspace in Chapter13/ ProceduralTextures.
Example 13-18. Antialiased Checker Fragment Shader
#ifdef GL_ES
precision highp float;
#extension GL_OES_standard_derivatives : enable
#endif
uniform int u_frequency;
uniform vec4 u_color0;
uniform vec4 u_color1;
varying vec2 v_st;
void
main()
{
vec4 color;
vec2 st_width;
vec2 fuzz;
vec2 check_pos;
float fuzz_max;
// calculate the filter width.
st_width = fwidth(v_st);
fuzz = st_width * float(u_frequency) * 2.0;
fuzz_max = max(fuzz.s, fuzz.t);
// get the place in the pattern where we are sampling
check_pos = fract(v_st * float(u_frequency));
if (fuzz_max <= 0.5)
{
// if the filter width is small enough, compute the pattern
// color by performing a smooth interpolation between the
// computed color and the average color.
vec2 p = smoothstep(vec2(0.5), fuzz + vec2(0.5), check_pos)
+ (1.0 - smoothstep(vec2(0.0), fuzz, check_pos));
color = mix(u_color0, u_color1,
p.x * p.y + (1.0 - p.x) * (1.0 - p.y));
color = mix(color, (u_color0 + u_color1)/2.0,
smoothstep(0.125, 0.5, fuzz_max));
}
else
{
// filter is too wide. just use the average color.
color = (u_color0 + u_color1)/2.0;
}
gl_FragColor = color;
}Figure 13-12 shows the checkerboard image rendered using the antialiased checker fragment shader in Example 13-18 with u_frequency = 10, u_color0 set to black, and u_color1 set to white.
To antialias the checkerboard procedural texture, we need to estimate the average value of the texture over an area covered by the pixel. Given a function g(v) that represents a procedural texture, we need to calculate the average value of g(v) of the region covered by this pixel. To determine this region, we need to know the rate of change of g(v). The GL_OES_standard_derivatives extension allows us to compute the rate of change of g(v) in x and y using the functions dFdx and dFdy. The rate of change, called the gradient vector is given by [dFdx(g(v)), dFdy(g(v))]. The magnitude of the gradient vector is computed as sqrt((dFdx(g(v))2 + dFdx(g(v))2). This can also be approximated by abs(dFdx(g(v))) + abs(dFdy(g(v))). The function fwidth can be used to compute the magnitude of this gradient vector. This approach works fine if g(v) is a scalar expression. If g(v) is a point, we will need to compute the cross-product of dFdx(g(v)) and dFdy(g(v)). In the case of the checkerboard texture example, we need to compute the magnitude of the v_st.x and v_st.y scalar expressions and therefore the function fwidth can be used to compute the filter widths for v_st.x and v_st.y.
Let w be the filter width computed by fwidth. We need to know two additional things about the procedural texture:
The smallest value of filter width
ksuch that the procedural textureg(v)will not show any aliasing artifacts for filter widths< k/2.The average value of the procedural texture
g(v)over very large widths.
If w < k/2, we should not see any aliasing artifacts. If w > k/2 (i.e., the filter width is too large), aliasing will occur. We use the average value of g(v) in this case. For other values of w, we use a smoothstep to fade between the true function and average value.
This, hopefully, provided good insight into how to use procedural textures and how to resolve aliasing artifacts that become apparent when using procedural textures. The generation of procedural textures for many different applications is a very large subject. The following list of references provides some good places to start if you are interested in reading more on procedural texture generation.
Anthony A. Apodaca and Larry Gritz. Advanced Renderman: Creating CGI for Motion Pictures (Morgan Kaufmann, 1999).
David S. Ebert, F. Kenton Musgrave, Darwyn Peachey, Ken Perlin, and Steven Worley. Texturing and Modeling: A Procedural Approach, 3rd ed. (Morgan Kaufmann, 2002).
K. Perlin. An image synthesizer. Computer Graphics (SIGGRAPH 1985 Proceedings, pp. 287–296, July 1985).
K. Perlin. Improving noise. Computer Graphics (SIGGRAPH 2002 Proceedings, pp. 681–682).
K. Perlin. Making Noise. www.noisemachine.com/talk1/.
Pixar. The Renderman Interface Specification, Version 3.2. July 2000. renderman.pixar.com/products/rispec/index.htm.
Randi J. Rost. OpenGL Shading Language, 2nd ed. (Addison-Wesley Professional, 2006).