Some more 3D engine-related news

Although I’ve mainly been working on the OpenGL sample for the BHM File Format, I also had some ideas on how to improve my Direct3D codebase. Namely, Direct3D 10 and 11 allow updating of shader constants by mapping a struct directly on the 3D device. In Direct3D 9, you had to use the constant table that the shader compiler generated, and you had to update each constant individually, by first retrieving a handle to the constant, and then calling a setter function on the constant table.

The new Direct3D 10+ way is not only more efficient (only one call required to update an entire block of shader constants), it is also more elegant to implement in C/C++ code. I ran into the same problem with OpenGL, as they use a similar system as Direct3D 9 to update shader constants. There is a new ARB_Uniform_Buffer_Object extension that gives you a way to update the constants in a way similar to Direct3D 10+, but it was introduced only a few months ago, in OpenGL 3.1. This isn’t supported everywhere yet (Intel’s drivers don’t support it, and Apple doesn’t support it). So it’s best to stick with the old method for now, for best compatibility.

So I wanted something that would simulate the Direct3D 10+ way of filling a struct and sending it off to the GPU, where the handling of the shader variables was abstracted away. I figured I could build a mock ID3DBuffer object which would take care of this… but I had to somehow combine the struct and the shader variable names and types, so that the proper values could be read from the struct, and set to the constant table with the setter function for the correct datatype.

I decided to take the input description system in Direct3D as an example on how to describe the struct in a way that it can be translated automatically. I had already shortly discussed how I translate input descriptions and vertex declarations between Direct3D 9 and 10+ earlier, this idea is going to be somewhat similar… I came up with this description format for structs with shader constants:

typedef struct D3D_CONST_ELEMENT_DESC
    LPCSTR SemanticName;
    UINT Count;
    D3D_CONST_FORMAT Format;
    UINT AlignedByteOffset;

A small example of a constant buffer struct and its description:

    D3DXCOLOR ambient;
    D3DXCOLOR diffuse;
    D3DXCOLOR specular;

D3D_CONST_ELEMENT_DESC SkinnedSurface9::psDesc[] =
        { "ambient", 1, D3D_CONST_FORMAT_VECTOR, 0 },
        { "diffuse", 1, D3D_CONST_FORMAT_VECTOR, 16 },
        { "specular", 1, D3D_CONST_FORMAT_VECTOR, 32 },

The struct can be shared between all versions of Direct3D now, and the description is only required for Direct3D 9. It can be silently ignored in 10 and 11. This way I can use a single code path for all APIs.

In Direct3D 9 I can now build the ID3DConstBuffer object:

pPshConstBuffer = new ID3DConstBuffer( psDesc, _countof(psDesc), pPshConstTable, p3D->pD3DDevice );

This will initialize itself by allocating a struct internally, and getting the handles to all constants named in the declaration.
You then use it exactly like you would in D3D10/11:

p3D->Map( pPshConstBuffer, D3D_MAP_WRITE_DISCARD, (void**)&pPshConstData);
// Set pixelshader constants


Map() will return a pointer to the internal structure, so you fill the data inside the buffer directly. After the struct is filled by the application, the buffer object can automatically read the variables from the struct and call the setter functions on the constant table, to copy the data to the GPU.

Some more thoughts on OpenGL…

The more I use OpenGL, the more I am being confronted with how much legacy there is in the API, and how limited this legacy functionality is. The problem with OpenGL seems to be that most tutorials and articles are quite dated, and build mostly on the legacy functionality. Microsoft already took the step to cut all legacy fixedfunction functionality from the API in Direct3D 10, and started with a clean slate, supporting only shaders.

In OpenGL, although they have deprecated a lot of legacy functionality, you still have access to it because of the backwards compatibility of the API (with Direct3D, each major version has its own interfaces, where the interface names include the version, such as ID3D10Device or ID3D11Device, so technically it’s a completely different API, it’s not an extension of an older version). However, in my opinion, everyone who wants to use OpenGL, should treat it as if it is a lightweight shader-only API, just like Direct3D 10, and ignore the legacy pipeline functionality. I will try to explain why.

The thing with the legacy pipeline is that it is very limited. When I originally designed my Direct3D 9 engine, I designed it around the fixed function pipeline, and supported shaders as an option. When I wanted to implement more advanced shading, such as matrix palette skinning, I could still combine it with the fixed function pipeline, because it supports setting up to 256 world matrices, and each matrix can be accessed individually. OpenGL does not have this luxury. The matrix handling is very limited. There are basically only two matrices that can be manipulated. Those are the modelview matrix and the projection matrix. You cannot set a palette of world matrices, and you cannot retrieve the current world (model) matrix or view (camera) matrix individually, as they are seen as a single matrix stack, and always multiplied together. This is a bit of a shame, as OpenGL even has integration of the matrix state with GLSL, so you can use these matrices directly in your shader code. This would have been a nice feature, if you could access the matrices the way you want them, but sadly you can’t.

Therefore, to save yourself a lot of trouble and headache, it’s better to just pretend the whole matrix pipeline doesn’t even exist in OpenGL at all. Just build your own matrix handling framework, and set the matrices to shader constants yourself, so you can access as many world matrices as you want, and have direct access to the view matrix, perhaps even an inverted view matrix, and other types of matrices that may come in handy for skinning.

In Direct3D 9 I had built this as an extension on the existing pipeline. I could easily call up common matrix variations. Then in Direct3D 10/11 I could use the same system as the main pipeline (it has no pipeline of itself, so the only difference is that the matrices are never sent to the D3D pipeline. They are only stored internally). My Java software engine also used a very similar system. I will now implement something similar for OpenGL. An easy way to globally store and retrieve matrices, which can then be used for various tasks in various places in the rendering process, without having to know everything about the objects and their hierarchies.

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s