So, you want to learn the magic that puts the gold into modern games?

Note: This series is an update to the previous XNA Shader Programming series I have written for XNA 3.0. If you know both XNA 3.0 and XNA 4.0, it should be no problem for you to understand the shaders from the other series if you want to move on faster than I can update the old ones.

My name is Petri Wilhelmsen and is a member of Dark Codex Studios. We usually participate in various competitions regarding graphics/game development, at The Gathering, Assembly, Solskogen, Dream-Build-Play, NGA and so on.

The XNA Shader Programming series will cover many different aspects of XNA, and how to write HLSL shaders using XNA and your GPU. I will start with some basic theory, and then move over to a more practical approach to shader programming.

The theory part will not be very detailed, but should be enough for you to get started with Shaders and be able to experiment for yourself. It will cover the basics around HLSL, how the HLSL language works and some keywords that is worth knowing about.

Today I will cover XNA and HLSL, as well as a simple ambient lighting algorithm.

Prerequisites
Some programming in XNA (this tutorial is about shader programming and not XNA in it self), as I wont go much into details about loading textures, 3d models, matrices and some math.

2001: A shader odyssey – A brief history of shaders
Before DirectX8, GPU’s had a fixed way to transform pixels and vertices, called “The fixed pipeline”. This made it impossible to developers to change how pixels and vertices was transformed and processed after passing them to the GPU, and made games looked quite similar graphics wise.

In 2001, DirectX8 introduced the vertex and pixel shaders, as a utility that developers could use to decide how the vertices and pixels should be processed when going through the pipeline, giving them a lot of flexibility.
An assembly language was used to program the shaders, something that made it pretty hard to be a shader developers, and shader model 1.0 was the only supported version. But this changed once DirectX9 was released, giving developers the opportunity to develop shaders in a high level language, called High Level Shading Language( HLSL ), replacing the assembly shading language with something that looked more like the C-language. This made shaders much easier to write, read and understand.

DirectX10.0 introduced a new shader, the Geometry Shader, and was a part of Shader Model 4.0. But this required a new state-of-the-art graphics card, and Windows Vista.

The latest addition to the DirectX series is the DirectX 11 including a tesselator, DirectCompute for paralell programming and much more.

XNA supports Shader Model 1.0 to 3.0, but works on XP, Vista and XBox360!

Taking the red pill
So, the question is.. What is a shader? Well, a shader is simply a set of instructions that will be run on you graphics processing unit (GPU), performing specific tasks of you need. This makes it possible to developer small/tiny applications that make you in control of three stages in the graphics pipeline: The vertex shader stage, the geometry shader stage, and the pixel shader stage.

Fig 1 –High level Programmable pipeline

As you can see in fig 1, you are able to program all the green squares, and the rest is fixed, meaning that you cannot control them. The geometry shader is not supported by Xbox360 and XNA, so it will not be covered in this article. Rather, let’s take a quick tour through the vertex and pixel shaders (don’t get frustrated if you don’t understand the code yet, it will be covered in a later section).

Vertex shader
Vertex shaders are used to manipulate vertex-data, per vertex. This can for example be a shader that makes a model “fatter” during rendering by moving vertexes along their normals to a new position for every vertex in the model (deform shaders).
Vertex shaders get input from a vertex structure defined in the application code, and load this from the vertex buffer, passed into the shader. This describes what properties each vertex will have during shading: Position, Color, Normal, Tangent and so on.

The vertex shader sends its output for later use to the pixel shader. To define what data the vertex shader will pass to the next stage can be done by defining a structure in the shader, containing the data you want to store, and make the vertex shader return this instance, or by defining parameters in the shader, using the out keyword. Output can be Position, Fog, Color, Texture coordinates, Tangets, Light position and so on.

An example of a simple Vertex Shader that transforms an object to a position on the screen can be seen below.

struct VertexShaderInput
{
    float4 Position : POSITION0;
};

struct VertexShaderOutput
{
    float4 Position : POSITION0;
};

VertexShaderOutput VertexShaderFunction(VertexShaderInput input)
{
    VertexShaderOutput output;

    float4 worldPosition = mul(input.Position, World);
    float4 viewPosition = mul(worldPosition, View);
    output.Position = mul(viewPosition, Projection);

    return output;
}

Pixel shader
The Pixel shader manipulates all pixels (per pixel) on a given model/object/collection of vertices. This can be a metal box, where we want to customize the lighting algorithm on, colors and so on. The pixel shader gets data from the vertex shader’s output values, like position, normals and texture coordinates, and interpolates these values to the different pixels. A very simple and small pixel shader can look like the snippet below.

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0
{
    return float4(1,0,0,1);
}

The code colors everything that flows through the shader RED.

The pixel shader can have two output values, Color and Depth.

All of the stages displayed in Fig 1 are working together in order to synthesize images and display them on the monitor.

So, are you ready to take control of the GPU using shaders? In that case, sit back, you are about to join a long ride and be reborn in the world of shaders.

HLSL
High Level Shading Language(HLSL) is used to develop shaders using a language similar to C. Just as in C, HLSL gives you tools like declaring variables, functions, data types, testing( if/else/for/do/while and so on) and much more, in order to create a logic for processing vertices and pixels. Below is a table of some keywords that exists in HLSL. This is not all of them, but some of the most important ones.

HSLS offers a huge set of functions that can be used to solve complex equations. As we go through this article, we will cover many, but for now, here is a list with just a handful of them. It’s important to learn all of them in order to create high-performance shaders without re-implementing the wheel.

For a complete list: http://msdn2.microsoft.com/en-us/library/bb509611.aspx

Effect files
Effect files (.fx) makes shader developing in HSLS easier. You can think of them as containers where you can store shader functionality, including vertex-, geometry- and pixel shaders. This includes global variables, functions, structures, vertex shader functions, pixel shader functions, different techniques/passes, textures and so on.

We have already seen how to declare variables and structures in a shader, but what is this technique/passes thing?  It’s pretty simple. One Shader can have one or more techniques. One Technique is a piece of functionality that represents one functionality of a given .fx file. Each technique can have a unique name, and from the game/application, we can select what technique in the shader we want to used when rendering a given geometry, by setting the CurrentTechnique property of the Effect class like this:

effect.CurrentTechnique = effect.Techniques[“AmbientLight”];

One .fx file represents one effect. On the line above, we tell the effect to use the technique “AmbientLight”. One technique can have one or more passes, and we must remember to process all passes in order to archive the result we want.

This is an example of a shader containing one technique named “AmbientLight” and one pass named “P0”:

technique AmbientLight
{
    pass P0
    {
        VertexShader = compile vs_1_1 VS();
        PixelShader = compile ps_1_1 PS();
    }
}

This is an example of a shader containing one technique and two passes:
technique Shader
{
pass P0
    {
        VertexShader = compile vs_1_1 VS();
        PixelShader = compile ps_1_1 PS();
    }
pass P1
    {
        VertexShader = compile vs_1_1 VS_Other();
        PixelShader = compile ps_1_1 PS_Other();
    }
}

This is an example of a shader containing two techniques and one pass:
technique Shader_11
{
    pass P0
    {
        VertexShader = compile vs_1_1 VS();
        PixelShader = compile ps_1_1 PS();
    }
}

technique Shader_2a
{
    pass P0
    {
        VertexShader = compile vs_1_1 VS2();
        PixelShader = compile ps_2_a PS2();
    }
}

We can see that a technique got two functions, one for the pixel shader and one for the vertex shader:
VertexShader = compile vs_1_1 VS2();
PixelShader = compile ps_1_1 PS2();

This tells us that the technique will use VS2() function as the vertex shader, PS2() function as the pixel shader, and the shader requires shader model 1.1 or higher. This makes it possible to have a different and more complex shader for GPUs supporting higher shader model versions, and simpler shaders for older hardware that only support the earlier shader models.

XNA and shaders
It’s really easy to implement shaders in XNA. In fact, only a few lines of code are needed to load and use a shader. Here is a list of steps that can be followed when making a shader, each of them covered in detail below:
1. Make the shader
2. Put the shade file (.fx) in “Contents”
3. Make an instance of the Effect class
4. Initiate the instance of the Effect class.
5. Select what technique you want to use
6. Pass different parameters to the shader
7. Draw the scene/object

The steps in a bit more detail:

1.When making a shader, several programs like notepad, the visual studio editor and so on can be used. There are also some shader IDEs available, and personally I like to use nVidias FX Composer: http://developer.nvidia.com/object/fx_composer_home.html

2. When the shader is created, drag it into the ”Content” folder, so it gets an asset name. The asset name will be the same as the filename of the fx file, but you can edit this asset name so it better suits your needs.

3. XNA Framework includes an Effect class that is used to load and compile the shaders. To make an instance of this class, write the following line of code:

Effect effect;
Effect is a part of the “Microsoft.Xna.Framework.Graphics” library, so remember to add this line of code to the using statement block:
using Microsoft.Xna.Framework.Graphics;

4. To initiate the shader, we can use the Content property to either load if from the project or from a file:

effect = Content.Load<Effect>(“Shader”);
In the line above, “Shader” is the asset name of the shader you added to the Contents folder.

5. Select what technique you want to use:
effect.CurrentTechnique = effect.Techniques[“AmbientLight”];

6. Pass the parameters you want to set in the shader.

First you need to create a EffectParamter object:
EffectParameter projectionParameter;

Then in the LoadContent() function you bind the parameter object with a variable in the shader:
projectionParameter = effect.Parameters[“Projection”];

Now, you can set the parameter by using the SetValue function like this:
projectionParameter.SetValue(projection);

where projection is a matrix (here: representing the projection matrix).

7. All that is left is to go through all the different passes in the shader and render your object. This is done by using a loop:

for (int i = 0; i < effect.CurrentTechnique.Passes.Count; i++)
{
    //EffectPass.Apply will update the device to
    //begin using the state information defined in the current pass
    effect.CurrentTechnique.Passes[i].Apply();

    //sampleMesh contains all of the information required to draw
    //the current mesh
    graphics.GraphicsDevice.DrawIndexedPrimitives(
        PrimitiveType.TriangleList, 0, 0,
        meshPart.NumVertices, meshPart.StartIndex, meshPart.PrimitiveCount);
}

Technique: Ambient light
So, you now got the answer to what a shader is! Let’s take that in to use and create your first real shader! The first shader you will write is a really simple one that just transforms the vertexes and calculates the ambient light on the model.
But wait… What is the “Ambient light” thing we are talking about?

Well, ambient light is the basic light in a scene that’s “just there”. If you go into a complete dark room, the ambient light is typically zero, but when walking outside there is almost always some light that makes it possible to see. This light got no direction and is there to make sure objects that are facing a light source, will have a basic color.

A scene with only a yellow ambient light can be seen in figure 2. The scene consists of a black background color and an un-textured zombie model. We will make this zombie scene look a lot better as we are going through this series.

Fig 2. Scene lit with ambient light.

Before we can implement the ambient light technique, we need to understand it. The formula for Ambient light can be seen in 1.1 below.
I = Aintensity x Acolor ( 1.1)

I is the final light color on a given pixel, Aintensity is the intensity of the light (usually between 0.0 (0%) and 1.0 (100%)), and Acolor is the color of the ambient light. This color can be a hardcoded value, a parameter or a texture.

Ok, let’s start implementing the shader. First of all, we need a matrix that represents the world matrix, the view matrix and the projection matrix:

float4x4 World;
float4x4 View;
float4x4 Projection;

float4 AmbientColor;
float AmbientIntensity;

You might also notice that we added two variables in the end, the AmbientColor and AmbientIntensity. I don’t think I need to explain those now?

Next, we create a structure that will contain the shader inputs, and outputs.
These are used in the Vertex Shader itself.

The structures contains a variable of the type float4 with the name Position. The : POSITION in the end tells the GPU what register to put this value in. So, what is a register? Well, a register is simply just a container in the GPU that contains data. The GPU got different registers to put position data, normal, texture coordinates and so on, and when defining a variable that the shader will pass to the pixel shader, we must also decide where in the GPU this value is stored.

When declaring the Vertex Shader function “VertexShaderFunction” (works like the main() function for Vertex Shaders), we specify that the function should return a VertexShaderOutput object (you can name the structure yourself, it could be VSOut and so on), and it takes in the VertexShaderInput structure. All that the shader needs to to is to create an instance of the VertexShaderOutput structure, set its members and return it.

struct VertexShaderInput
{
    float4 Position : POSITION0;
};

struct VertexShaderOutput
{
    float4 Position : POSITION0;
};

VertexShaderOutput VertexShaderFunction(VertexShaderInput input)
{
    VertexShaderOutput output;

    float4 worldPosition = mul(input.Position, World);
    float4 viewPosition = mul(worldPosition, View);
    output.Position = mul(viewPosition, Projection);

    return output;
}

This is a basic “pass-through” shader. It will take the position of the vertex in the model and transform it to the correct space.

Next we create the Pixel Shader function, where we will compute the Ambient Light. This takes the output from the Vertex Shader as input. It’s a simple function that takes the pixel it is working on, and color it to the AmbientColor*AmbientIntensity. Simply: A color, the color of the given pixel. Remember, pixel shaders works with one and one pixel at a time.

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0
{
    return AmbientColor*AmbientIntensity;
}

The last thing we need to do is to create the Technique and the passes. This shader will contain one Technique with one pass.

technique Technique1
{
    pass Pass1
    {
        VertexShader = compile vs_2_0 VertexShaderFunction();
        PixelShader = compile ps_2_0 PixelShaderFunction();
    }
}

Ok, thats it!

Now, i recommend you to look at the source code and play around with the values in order to understand how to setup and implement a shader using XNA. Especially, take a look at how the shader is loaded and what parts is needed. I have tried to minimize everything so the code mostly only consists of Shader related code. The camera spins around the model by using a Cos/Sin function on the view matrix.

 Download Source (XNA 4.0)