The programmable pipe – or shaders

A modern "shader" graphics pipeline

What all of this means is that a graphics card with a GPU based on a fixed function vertex and pixel pipeline can do nearly all of the work in converting the 3D model into a 2D representation.

The programmer has to specify the vertices of each triangle in the mesh, set up arrays of numbers specifying the position of the camera, lights, surface textures and so on and finally configure processes such as texture filtering and fog effects. After this the GPU more or less does all of the work.

This is excellent but it does tend to produce 3D graphics that look the same. Even if you are a creative programmer there isn't much you can do to actually show off your talents with a fixed graphics pipeline. This is where shaders come in.

Instead of having a fixed function T&L vertex processor and a fixed function pixel processor, the modern 3D graphics cards have programmable hardware in the pipeline. A program running on the vertex processing pipeline is called a "vertex shader" and one running on the pixel part of the pipeline is called a "pixel shader".

This makes it possible to implement any sort of improvement you care to think up in the GPU and create graphics that are better than the average.

For example a programmer is now free to use a vertex shader to improve on the lighting effects algorithms in use to create halos, backlighting and dynamic shadows. A pixel shader can be used to improve the quality of texture and surface rendering.

Exciting stuff but the problem is that the first implementations of the shader concept were very difficult to use. Most programs don't use the GPU directly but go through a "graphics engine" such as DirectX or OpenGL, a program designed to make the GPU easier to use.

The first breakthrough in making shaders easier to use was the introduction of a shading script in the Quake III graphics engine, but OpenGL and DirectX 9 introduced high level shading languages of their own. The current leader in shader languages is Pixar's RenderMan, but this is currently too advanced for the average GPU found in a PC graphics card. However shaders are slowely but surely working their way into all areas of graphics programming. For example, Silverlight web programmers can now use shaders directly to create special effects.

GPU advances

Not only have shader languages undergone rapid development, so has the GPU hardware.

The first advanced graphics cards implemented Vertex Shader vs_1_1, which limited processing to 128 instructions; the very latest cards support vs_3_0 with a minimum of 512 instructions and lots of additional instructions.

Pixel shaders have been through a similar development with the early ps_1_1 system supporting just 12 instructions and today's ps_3_0 supporting a minimum of 512.

In addition graphics hardware is free to implement the processing pipeline in different ways. For example, the latest graphics card from nVida, part of the GeForce series, use "superscalar" architecture, i.e. multiple GPU pipelines, to speed up the Shader Model 3.0 and high dynamic range HDR rendering. HDR uses 32-bit floating point numbers processed in hardware to give a wider dynamic range for the rendered image allowing detail in the dark and highlights of an image.

To give you some idea of just how powerful the latest Shader Model 3 based graphics cards are, consider the problem of creating the effect of moving water in a game. Previous attempts at this have always looked artificial and have involved a lot of "soft focus". By writing an appropriate shader the dynamic surface of water can be simulated by solving the equations that govern its behaviour.

You may not think of fire as water but both can be simulated as fluids. In this case the flames are simulated by the GPU

In fact this approach allows the game world to introduce fluids of any sort that move realistically and look realistic. The GPU as equation solver is such a powerful idea that it is being used in science and technology as an alternative to building a super computer.

We haven't quite got to the stage of "realtime cinematic rendering" but it's not far off. What is also important is that, as more and more of the graphics tasks are given to the GPU to perform, the CPU becomes free to be used to implement more advanced game features such as opponents that have artificial intelligence or gaming strategies.

This Shader editor makes it easy to try out effects programs. The Shader program is in the window on the left and it applies a "wood" texture using Shader model 2.

Going further

For information on how to program 3D graphics in DirectX see:
www.microsoft.com/directx

and for OpenGL:
www.opengl.org

http://developer.nvidia.com/

and for game programming in general:

http://www.gamedev.net/

<ASIN:0321515269>

<ASIN:0321335597>

<ASIN:0321228324>

<ASIN:B001H5V6LO>

<ASIN:3836490730>