Valve to bring Steam and Left 4 Dead 2 to linux

There have been rumours about a linux version of Steam for a long time, but a few days ago, Valve finally made an official announcement of Steam for linux. At the same time, they are also making Left 4 Dead 2 available on linux. I would like take this opportunity to comment on Valve’s move, and linux and OpenGL in general.

The first thing that stands out is their choice of Ubuntu. This is much the same reason as why most games are developed for Windows rather than linux: it is the largest platform. You develop for the largest platform first, because that’s the largest potential market. Then you look at other platforms to see if they are attractive enough to support as well. This generally has little to do with the technical merits of the platform in itself. For example, the Nintendo Wii, Sony PlayStation 3 and Microsoft Xbox 360 are nowhere near as powerful as a modern Windows PC. However, because they are very large markets, they remain interesting to develop for. Likewise, Valve’s choice for Ubuntu should not be interpreted as Valve saying that Ubuntu is the best linux distribution out there. It just means that it is the most interesting distribution for them to develop for.

Standardization

It also indicates a problem within the linux world: the diversity. Since linux distributions are not very standardized, it is very hard to distribute binaries that work on all linux distributions. Different distributions will use different libraries, or put things in different locations, or use different versions of the same software, etc. And Steam is primarily a platform to distribute binaries, so the only choice that Valve has is to take it one distribution at a time. Compare this to OS X or Windows, and there is a big difference there. Currently there are 3 major versions of Windows in common use (XP, Vista and 7), in both 32-bit and 64-bit versions, and at various states of being up-to-date (not everyone installs service packs right away… or even at all). Likewise I’m sure that not everyone uses the latest version of OS X ever. However, these OSes have a very stable ABI (application binary interface), and generally a lot of care is taken to make sure that newer versions of libraries remain backward-compatible with older ones. As a result, the same binary code can easily be run on a wide range of different versions and installations of the OS. With linux, the distributions are closer together on paper. For example, the difference between a Windows XP kernel and a Windows 7 kernel is far greater than the few minor revisions difference of the kernel in the latest Ubuntu and Fedora release.

nVidia and AMD already suffer similar problems with their binary blob drivers: small changes to the linux kernel and/or to Xorg often require them to release new drivers, because the old ones are no longer compatible with the new code. While open source is clearly an important factor in an OS such as linux, it seems that the developers have been using it as a crutch: “Sure, we can change this, even though it won’t be compatible with older versions. People can just recompile all their applications and it will work again.” Open source is no reason not to have a stable and standardized environment. Hopefully Steam will have a positive effect on the situation. If these linux games catch on somewhat, perhaps the developers will be a bit more careful when it comes to binary compatibility. Breaking compatibility with a major linux game could make your distribution very unpopular. Likewise, with Ubuntu becoming the de facto games distribution, it might result in other distributions trying to conform to this de facto standard, so that Ubuntu games can also be played on their distributions.

If you build it, they will come…

Valve will provide Steam and Left 4 Dead 2 for linux. But that’s it, for now. Steam will be able to distribute linux games, but these games still need to be provided by the developers. So we will have to see. Games for OS X are somewhat more common than for linux, and OS X has had Steam support for a while now. However, many developers still don’t provide OS X ports of their games. And the ones that do, often go the ‘economy way’ and just use Cider (the OS X version of Wine) as a wrapper around the Windows version of the game to make it work on OS X. At least it’s better than nothing, and I expect at least some developers to take the same approach to linux.

The better solution is the native port, obviously. This is what Valve is going for. In a way it is an interesting move. OS X is only about 10% of the total market, and linux is even less (about 2-3%). However, for that small part of the market, they had to completely rewrite their Steam platform and Source engine. Initially both were entirely Windows-only. The original version of the Steam client used the IE browser component. Valve had to rewrite the client to use Webkit instead, before they could port it to OS X. Once they did that, porting the client to linux was not that difficult anymore, since OS X and linux are not that different. Both are UNIX-like environments, and Webkit supports both platforms (as well as Windows itself).

The original Source engine was also Windows-only, making use of the DirectX 9 API. Here, Valve also had to choose different APIs to get the engine to work on OS X. An important part of that is the Direct3D component, which they replaced with OpenGL. Like Webkit, OpenGL is supported on all three platforms. This may have implications for the future of the Source engine. As I already pointed out when Valve started supporting OS X, their Source engine is still limited to DirectX 9 on Windows, which is getting extremely long in the tooth, with D3D11 already being available for quite a while, and D3D11.1 just around the corner with Windows 8. The difference between D3D9 and D3D10+ is huge, so supporting the newer APIs requires a big rewrite of the engine anyway. About as big as an OpenGL rewrite. However, OpenGL enables the same functionality on Windows as D3D11 would. So it seems like an easier way forward to use OpenGL on Windows as well. It would also solve the problem that XP poses with D3D10+. Most games still want to support Windows XP today, so even if they include a D3D10+ renderer, they also have a D3D9 renderer for XP compatibility, as D3D10+ is only available on Vista and newer OSes. So if Valve sticks to Direct3D on Windows, that would mean that they’d have to support 3 rendering APIs: D3D9 for XP, D3D10+ for Vista and newer, and OpenGL for OS X/linux. Where OpenGL could support all platforms (although obviously they’d still need different renderpaths inside the engine, to deal with different versions of OpenGL, different hardware capabilities, and some platform-specific details).

Since Valve chose to do true native ports of Steam and Source for both OS X and linux, that means that they have had to do quite a significant rewrite, for only a small potential marketshare (less than 15%). Their aim is most probably not directly related to sales, as it will take a while to get a return on the investment they made when porting their code. Perhaps they feel that a more portable infrastructure will pay off in the future. There are rumours of Valve developing a game console based on linux, so who knows.

OpenGL vs Direct3D

So that brings us to the question: just how good are these native ports? The graphics layer is the biggest issue here, in terms of porting effort, and also the resulting performance and compatibility. People are quick to point out that on paper, OpenGL can do everything that Direct3D can. But I’ve already discussed some of the practical differences before. One of the differences is that with Direct3D, the display driver only implements the bare functionality at the lowest level. The actual API is implemented on top of that by a common DirectX runtime, which is provided by Microsoft. With OpenGL, the entire API is implemented by the driver. This means that your GPU can only work with the latest version of the API when the vendor supplies a driver for that. In practice, this is not always the case, even if the hardware is capable of it.

For example, Intel has notoriously poor support for OpenGL functionality. They have picked things up recently, with the Sandy Bridge and Ivy Bridge IGPs, but the earlier DX9 and DX10-capable GPUs don’t expose anywhere near their full featureset under OpenGL, and are stuck at OpenGL 2.1 or worse.

Likewise, under OS X, Apple likes to keep some control of the OpenGL runtime. As a result, they are a few versions behind. They only started supporting OpenGL 3.2 recently, in OS X 10.7. OS 10.6.5 still only supported OpenGL 2.1. At the time of writing, OpenGL 4.2 is already out. On *nix there is a similar situation for the open source drivers. These open source drivers are part of the MesaGL project, which provides an open source OpenGL-compatible environment on *nix platforms. They currently only implement version 3.0 of the API.

By contrast, in Windows, anyone with Vista or newer can simply update to the latest version of the DirectX API, regardless of video driver. Also, in Windows, both AMD and nVidia provide a full OpenGL 4.2 implementation in their drivers, and Intel provides version 4.0 for their HD2500 and HD4000 GPUs (the Ivy Bridge ones). Which is rather ironic. While OpenGL is not the primary graphics API on Windows, the drivers are still more up-to-date than the OSes where OpenGL is the only API for accelerated graphics. But even here, you are faced with the fact that not all drivers implement the same API level (and even if they do, they are separate implementations, so you may run into some compatibility issues between them. Code that compiles for vendor A will not compile for vendor B).

Some practical examples

Let’s look at some real-world examples then, from the OS X/OpenGL ports of Valve games. Just look at Anandtech’s preview, or this discussion on the Steam forums. We see that neither the graphics quality nor the performance are quite on par with the Windows version. In fact, someone brings up the MacBenchmarks website in the discussion. On this site, people compare the OS X version of the Steam games with the Windows version running on the same machine, via Bootcamp. The performance differences are nothing short of staggering in some cases. We see that in some cases Windows is about twice as fast (and that is after some performance updates).

Another view on the differences between Direct3D and OpenGL is ID Software’s Rage. One of very few high-profile OpenGL games to be released for Windows in recent years. There are two things that stood out in this release:

1. The game suffered from severe bugs. It would not run at all on Intel or AMD GPUs. It worked on nVidia GPUs, although there were some issues with texture quality (one could ask how such a game could possibly have passed QA and have been released in this state, but still…). After a few patches for the game, and going through various beta drivers, it finally started to work on AMD GPUs as well, and eventually also some of Intel’s GPUs (but not most of them, since it requires OpenGL 3.3 API level support).
2. The graphics were far from spectacular. They are not that much better than Doom 3 or various older DirectX 9 titles, such as Half Life 2.

The result of this is that while this game COULD have run on older/lower-end hardware, such as Intel’s older Direct3D 10 or even Direct3D 9 chips, the OpenGL API requirement locks them out (and even for people who have hardware that DOES support the game, they had to go through some driver updates to actually get it working).

Also, at this point I’d like to point out a pet-peeve of mine:

Direct3D versioning has two aspects: the hardware level and the API level. The hardware does not necessarily have to be of the same level as the API. For example, D3D11 will work with most hardware of D3D9-level or higher. D3D9 would work with D3D7 or higher, etc. Many people don’t seem to understand this, and think a newer API can’t be used on older hardware for some reason. The only time this was true, was for Direct3D 10, which was mainly to remove a lot of legacy from the API going forward. OpenGL has gone through a similar transition by introducing core profiles. OpenGL ES also does not have legacy support. Microsoft actually reverted that decision to some extent, and added ‘downlevel functionality’ to the API, so that you could now at least use the new D3D API on most D3D9 hardware, but still without the legacy of the fixed-function pipeline.

Since Direct3D is the only major API that defines hardware levels this way (OpenGL development has been reduced to merely adding extensions based on new functionality that is already in hardware, and supported by Direct3D), these hardware levels have become the de-facto way of describing hardware.

As such, it is common to refer to a PlayStation 3 as being a ‘DirectX 9’ device. Technically it does not use DirectX, but the GPU was obviously designed for DirectX 9, and that is the featureset it implements in hardware. For some reason, many people don’t quite get this. They think you can’t refer to a D3D hardware level for any device that does not use D3D. It’s not like the GPU cares about the API used, no more than that your CPU cares about what OS you use.

Conclusions?

So what to conclude from all this? Well, I’m not sure really. It’s clear that OpenGL is not in quite as mature a state as Direct3D is at the moment, not even on Windows. Will Valve’s efforts make a change in that situation? That is impossible to say at this moment. For OS X it does not seem to be that much of a success. Although the games certainly are playable under OS X, it appears that people still prefer to play these games via Bootcamp. I suppose that linux users will be less likely to dual-boot to Windows, but they are also a smaller market than OS X… So their choices will not be all that decisive in the greater scheme of things. As for Valve… Perhaps their real motive is something different than just bringing their games to linux? They could be looking at their own console, a SteamBox, so to say, based on linux, so they won’t have to deal with third parties such as Microsoft. Or perhaps they are gearing up for another platform altogether, perhaps Android? At the current rate of hardware development, it is not unlikely that Android devices will soon be powerful enough to play DX9-level games and beyond. Imagine just connecting your tablet or smartphone to a TV and some controllers, and go. It doesn’t necessarily have to perform as well as Windows games (the current consoles don’t do that either, and they’re still successful), as long as the games themselves are playable.

Advertisement