Valve to bring Steam and Left 4 Dead 2 to linux

There have been rumours about a linux version of Steam for a long time, but a few days ago, Valve finally made an official announcement of Steam for linux. At the same time, they are also making Left 4 Dead 2 available on linux. I would like take this opportunity to comment on Valve’s move, and linux and OpenGL in general.

The first thing that stands out is their choice of Ubuntu. This is much the same reason as why most games are developed for Windows rather than linux: it is the largest platform. You develop for the largest platform first, because that’s the largest potential market. Then you look at other platforms to see if they are attractive enough to support as well. This generally has little to do with the technical merits of the platform in itself. For example, the Nintendo Wii, Sony PlayStation 3 and Microsoft Xbox 360 are nowhere near as powerful as a modern Windows PC. However, because they are very large markets, they remain interesting to develop for. Likewise, Valve’s choice for Ubuntu should not be interpreted as Valve saying that Ubuntu is the best linux distribution out there. It just means that it is the most interesting distribution for them to develop for.

Standardization

It also indicates a problem within the linux world: the diversity. Since linux distributions are not very standardized, it is very hard to distribute binaries that work on all linux distributions. Different distributions will use different libraries, or put things in different locations, or use different versions of the same software, etc. And Steam is primarily a platform to distribute binaries, so the only choice that Valve has is to take it one distribution at a time. Compare this to OS X or Windows, and there is a big difference there. Currently there are 3 major versions of Windows in common use (XP, Vista and 7), in both 32-bit and 64-bit versions, and at various states of being up-to-date (not everyone installs service packs right away… or even at all). Likewise I’m sure that not everyone uses the latest version of OS X ever. However, these OSes have a very stable ABI (application binary interface), and generally a lot of care is taken to make sure that newer versions of libraries remain backward-compatible with older ones. As a result, the same binary code can easily be run on a wide range of different versions and installations of the OS. With linux, the distributions are closer together on paper. For example, the difference between a Windows XP kernel and a Windows 7 kernel is far greater than the few minor revisions difference of the kernel in the latest Ubuntu and Fedora release.

nVidia and AMD already suffer similar problems with their binary blob drivers: small changes to the linux kernel and/or to Xorg often require them to release new drivers, because the old ones are no longer compatible with the new code. While open source is clearly an important factor in an OS such as linux, it seems that the developers have been using it as a crutch: “Sure, we can change this, even though it won’t be compatible with older versions. People can just recompile all their applications and it will work again.” Open source is no reason not to have a stable and standardized environment. Hopefully Steam will have a positive effect on the situation. If these linux games catch on somewhat, perhaps the developers will be a bit more careful when it comes to binary compatibility. Breaking compatibility with a major linux game could make your distribution very unpopular. Likewise, with Ubuntu becoming the de facto games distribution, it might result in other distributions trying to conform to this de facto standard, so that Ubuntu games can also be played on their distributions.

If you build it, they will come…

Valve will provide Steam and Left 4 Dead 2 for linux. But that’s it, for now. Steam will be able to distribute linux games, but these games still need to be provided by the developers. So we will have to see. Games for OS X are somewhat more common than for linux, and OS X has had Steam support for a while now. However, many developers still don’t provide OS X ports of their games. And the ones that do, often go the ‘economy way’ and just use Cider (the OS X version of Wine) as a wrapper around the Windows version of the game to make it work on OS X. At least it’s better than nothing, and I expect at least some developers to take the same approach to linux.

The better solution is the native port, obviously. This is what Valve is going for. In a way it is an interesting move. OS X is only about 10% of the total market, and linux is even less (about 2-3%). However, for that small part of the market, they had to completely rewrite their Steam platform and Source engine. Initially both were entirely Windows-only. The original version of the Steam client used the IE browser component. Valve had to rewrite the client to use Webkit instead, before they could port it to OS X. Once they did that, porting the client to linux was not that difficult anymore, since OS X and linux are not that different. Both are UNIX-like environments, and Webkit supports both platforms (as well as Windows itself).

The original Source engine was also Windows-only, making use of the DirectX 9 API. Here, Valve also had to choose different APIs to get the engine to work on OS X. An important part of that is the Direct3D component, which they replaced with OpenGL. Like Webkit, OpenGL is supported on all three platforms. This may have implications for the future of the Source engine. As I already pointed out when Valve started supporting OS X, their Source engine is still limited to DirectX 9 on Windows, which is getting extremely long in the tooth, with D3D11 already being available for quite a while, and D3D11.1 just around the corner with Windows 8. The difference between D3D9 and D3D10+ is huge, so supporting the newer APIs requires a big rewrite of the engine anyway. About as big as an OpenGL rewrite. However, OpenGL enables the same functionality on Windows as D3D11 would. So it seems like an easier way forward to use OpenGL on Windows as well. It would also solve the problem that XP poses with D3D10+. Most games still want to support Windows XP today, so even if they include a D3D10+ renderer, they also have a D3D9 renderer for XP compatibility, as D3D10+ is only available on Vista and newer OSes. So if Valve sticks to Direct3D on Windows, that would mean that they’d have to support 3 rendering APIs: D3D9 for XP, D3D10+ for Vista and newer, and OpenGL for OS X/linux. Where OpenGL could support all platforms (although obviously they’d still need different renderpaths inside the engine, to deal with different versions of OpenGL, different hardware capabilities, and some platform-specific details).

Since Valve chose to do true native ports of Steam and Source for both OS X and linux, that means that they have had to do quite a significant rewrite, for only a small potential marketshare (less than 15%). Their aim is most probably not directly related to sales, as it will take a while to get a return on the investment they made when porting their code. Perhaps they feel that a more portable infrastructure will pay off in the future. There are rumours of Valve developing a game console based on linux, so who knows.

OpenGL vs Direct3D

So that brings us to the question: just how good are these native ports? The graphics layer is the biggest issue here, in terms of porting effort, and also the resulting performance and compatibility. People are quick to point out that on paper, OpenGL can do everything that Direct3D can. But I’ve already discussed some of the practical differences before. One of the differences is that with Direct3D, the display driver only implements the bare functionality at the lowest level. The actual API is implemented on top of that by a common DirectX runtime, which is provided by Microsoft. With OpenGL, the entire API is implemented by the driver. This means that your GPU can only work with the latest version of the API when the vendor supplies a driver for that. In practice, this is not always the case, even if the hardware is capable of it.

For example, Intel has notoriously poor support for OpenGL functionality. They have picked things up recently, with the Sandy Bridge and Ivy Bridge IGPs, but the earlier DX9 and DX10-capable GPUs don’t expose anywhere near their full featureset under OpenGL, and are stuck at OpenGL 2.1 or worse.

Likewise, under OS X, Apple likes to keep some control of the OpenGL runtime. As a result, they are a few versions behind. They only started supporting OpenGL 3.2 recently, in OS X 10.7. OS 10.6.5 still only supported OpenGL 2.1. At the time of writing, OpenGL 4.2 is already out. On *nix there is a similar situation for the open source drivers. These open source drivers are part of the MesaGL project, which provides an open source OpenGL-compatible environment on *nix platforms. They currently only implement version 3.0 of the API.

By contrast, in Windows, anyone with Vista or newer can simply update to the latest version of the DirectX API, regardless of video driver. Also, in Windows, both AMD and nVidia provide a full OpenGL 4.2 implementation in their drivers, and Intel provides version 4.0 for their HD2500 and HD4000 GPUs (the Ivy Bridge ones). Which is rather ironic. While OpenGL is not the primary graphics API on Windows, the drivers are still more up-to-date than the OSes where OpenGL is the only API for accelerated graphics. But even here, you are faced with the fact that not all drivers implement the same API level (and even if they do, they are separate implementations, so you may run into some compatibility issues between them. Code that compiles for vendor A will not compile for vendor B).

Some practical examples

Let’s look at some real-world examples then, from the OS X/OpenGL ports of Valve games. Just look at Anandtech’s preview, or this discussion on the Steam forums. We see that neither the graphics quality nor the performance are quite on par with the Windows version. In fact, someone brings up the MacBenchmarks website in the discussion. On this site, people compare the OS X version of the Steam games with the Windows version running on the same machine, via Bootcamp. The performance differences are nothing short of staggering in some cases. We see that in some cases Windows is about twice as fast (and that is after some performance updates).

Another view on the differences between Direct3D and OpenGL is ID Software’s Rage. One of very few high-profile OpenGL games to be released for Windows in recent years. There are two things that stood out in this release:

  1. The game suffered from severe bugs. It would not run at all on Intel or AMD GPUs. It worked on nVidia GPUs, although there were some issues with texture quality (one could ask how such a game could possibly have passed QA and have been released in this state, but still…). After a few patches for the game, and going through various beta drivers, it finally started to work on AMD GPUs as well, and eventually also some of Intel’s GPUs (but not most of them, since it requires OpenGL 3.3 API level support).
  2. The graphics were far from spectacular. They are not that much better than Doom 3 or various older DirectX 9 titles, such as Half Life 2.

The result of this is that while this game COULD have run on older/lower-end hardware, such as Intel’s older Direct3D 10 or even Direct3D 9 chips, the OpenGL API requirement locks them out (and even for people who have hardware that DOES support the game, they had to go through some driver updates to actually get it working).

Also, at this point I’d like to point out a pet-peeve of mine:

Direct3D versioning has two aspects: the hardware level and the API level. The hardware does not necessarily have to be of the same level as the API. For example, D3D11 will work with most hardware of D3D9-level or higher. D3D9 would work with D3D7 or higher, etc. Many people don’t seem to understand this, and think a newer API can’t be used on older hardware for some reason. The only time this was true, was for Direct3D 10, which was mainly to remove a lot of legacy from the API going forward. OpenGL has gone through a similar transition by introducing core profiles. OpenGL ES also does not have legacy support. Microsoft actually reverted that decision to some extent, and added ‘downlevel functionality’ to the API, so that you could now at least use the new D3D API on most D3D9 hardware, but still without the legacy of the fixed-function pipeline.

Since Direct3D is the only major API that defines hardware levels this way (OpenGL development has been reduced to merely adding extensions based on new functionality that is already in hardware, and supported by Direct3D), these hardware levels have become the de-facto way of describing hardware.

As such, it is common to refer to a PlayStation 3 as being a ‘DirectX 9′ device. Technically it does not use DirectX, but the GPU was obviously designed for DirectX 9, and that is the featureset it implements in hardware. For some reason, many people don’t quite get this. They think you can’t refer to a D3D hardware level for any device that does not use D3D. It’s not like the GPU cares about the API used, no more than that your CPU cares about what OS you use.

Conclusions?

So what to conclude from all this? Well, I’m not sure really. It’s clear that OpenGL is not in quite as mature a state as Direct3D is at the moment, not even on Windows. Will Valve’s efforts make a change in that situation? That is impossible to say at this moment. For OS X it does not seem to be that much of a success. Although the games certainly are playable under OS X, it appears that people still prefer to play these games via Bootcamp. I suppose that linux users will be less likely to dual-boot to Windows, but they are also a smaller market than OS X… So their choices will not be all that decisive in the greater scheme of things. As for Valve… Perhaps their real motive is something different than just bringing their games to linux? They could be looking at their own console, a SteamBox, so to say, based on linux, so they won’t have to deal with third parties such as Microsoft. Or perhaps they are gearing up for another platform altogether, perhaps Android? At the current rate of hardware development, it is not unlikely that Android devices will soon be powerful enough to play DX9-level games and beyond. Imagine just connecting your tablet or smartphone to a TV and some controllers, and go. It doesn’t necessarily have to perform as well as Windows games (the current consoles don’t do that either, and they’re still successful), as long as the games themselves are playable.

About these ads
This entry was posted in Direct3D, OpenGL, Software development, Software news and tagged , , , , , , , , , , , , , , , , , , . Bookmark the permalink.

17 Responses to Valve to bring Steam and Left 4 Dead 2 to linux

  1. haemoo says:

    Hi, you probably found this already but if not:
    http://arstechnica.com/gaming/2012/07/steams-newell-windows-8-catastrophe-driving-valve-to-embrace-linux/

    MS has decided to throw their users out a window to see if they will fly, which is going to hurt a lot of businesses that rely on Windows being, at least, palpable.

    I like steam, but it has to be said that the future of any 3rd party app stores going forward is questionable.

    • Scali says:

      Yes, I’ve seen it… but Newell doesn’t actually make a point. Just because he says Windows 8 is bad, doesn’t make it true. It seems he’s mostly afraid of the Store in Windows 8 competing with Steam sales.
      Other than that, nothing much changes for Windows 8. Steam still runs on it, all Valve games still run on it…
      Trying to make it in some kind of technical debate is just silly. Windows 8 is still going to blow OS X and linux out of the water as a gaming platform, just like Windows 9x, XP and 7 before it.

      Newell just doesn’t make sense. Especially when he talks about the PC being an open platform.
      That has nothing to do with it. Consoles are closed platforms, and they are also very successful, at least as far as Steam/Valve games are concerned.
      Besides, while the PC hardware may be open, Windows is not. Yet Windows is what made the PC such a success as a gaming machine (and made Valve/Steam so successful), mostly with DirectX and hardware abstraction that really made games click-and-go, where in the DOS age it was often very difficult to get games running on your hardware, and home computers or consoles were far more popular. Not to mention the cost of PC hardware in those days.

      Just because Newell *wants* Windows 8 to fail, because he can make more money with Steam on other platforms, doesn’t mean it *will*. If Windows 8 will fail at all, it will ‘fail’ in the way that Vista did: a lot of people will continue to use the older version of Windows, and within a few years, Microsoft will come up with a slightly modified version (Windows 7) which addresses people’s complaints somewhat, and everyone is happy again.
      Back when Vista was released, people would say it will fail, and drive everyone to linux etc… Well no. the three years between Vista and Windows 7 were far too short for any kind of significant OS migration. People didn’t move to linux at all, they just stuck to XP. And then Windows 7 came out, which became the darling of the industry. Just a very solid OS (ironically still retaining some of the Vista features that people initially complained about). If anything, Windows has become MORE popular now (not so much in numbers perhaps, but you see people being more outspoken in their preference for Windows. Previously they used Windows because they had to, with Windows 7 they actually LIKE to use it, because it’s good).
      So if Windows 8 fails, I’m sure we’ll just see Windows 9 in 2-3 years time to fix whatever went wrong.

  2. XP says:

    Windows 8 and NT6.x is crap, even Valve & Blizzard agrees, don’t bother them about it.

    http://arstechnica.com/gaming/2012/07/steams-newell-windows-8-catastrophe-driving-valve-to-embrace-linux/

    Of course, NT6.x retards like Scali and the other NT6.x defenders can’t accept that fact.

  3. Pingback: Apple’s OS X is NOT linux! | Scali's OpenBlog™

  4. Hiram says:

    I thought I should bring this to your attention: http://www.phoronix.com/scan.php?page=news_item&px=MTE1MjI

    • Scali says:

      Seen it… kinda makes me wonder why they picked a 64-bit Windows OS, but not a 64-bit linux OS. Why not both 32-bit or both 64-bit?

      • Hiram says:

        While I’m by no means certain, it could possibly be because of the fact that the 32 bit version of Ubuntu is still the default recommended “flavour”. http://www.ubuntu.com/download/desktop

      • Scali says:

        Well, then why don’t they test it against a 32-bit version of Windows?
        Besides, did they test a 32-bit or a 64-bit version of L4D2 on Windows? The only version available to end-users is the 32-bit version. One could argue that this is less than ideal, since it requires context switches between 32-bit and 64-bit mode for various kernel/display driver operations. Might just be those few fps that Windows is missing out on, compared to Ubuntu.

        Or if it’s the 64-bit version… They used to have 64-bit versions of various HL2-related games. Performance was quite poor however (the Valve guys apparently have a lot of trouble getting their code to run well on any platform http://techgage.com/article/half-life_2_64-bit_-_reason_to_get_excited/2), so they eventually removed them from Steam, and now Steam clients on 64-bit OSes just download the 32-bit versions again.

        Another issue could be with desktop compositing. While game performance in Windows 7 certainly isn’t bad, Windows XP will generally get slightly higher framerates (especially when you get into such high rates, where the rendering time per frame is in the millisecond range), because the display driver gets full exclusive access in OpenGL/D3D9, while for Windows 7 there will always be some slight overhead since the desktop manager also runs under Direct3D.

        So I wonder what framerates they would get in XP/XP x64 for both the D3D9 and OpenGL renderers (I assume they turned everything off in Ubuntu, else they wouldn’t get these scores either, no doubt).

    • Scali says:

      You could download and compile the sample program I’ve included in this project: http://sourceforge.net/projects/bhmfileformat/

      It uses OpenGL and can be compiled for a variety of platforms including Windows, linux, FreeBSD and OS X (in both 32-bit and 64-bit).
      My experience is that Windows XP x64 is fastest, then XP x86, then Windows 7, and then linux, OS X and FreeBSD, at some distance from Windows.
      Also, the Direct3D version of that code (I have support for D3D9, 10 and 11) is faster than the OpenGL one (but I keep the D3D version closed source).

  5. RationalUser says:

    Have to agree with Scali’s post about gaming on various operating systems. I can’t comment too much on Linux, but the Mac really isn’t a great gaming platform if you like to play state-of-the-art games. The best gaming success I’ve had on my Macs is by running games that are a few years old (running them natively) – or by running slightly older titles such as the Prince of Persia Trilogy, Red Baron 3D, etc., in WINE for Mac (known as WineBottler) since Mac native versions are not available. Having said this, people’s mileage may vary with WINE support on a Mac since it’s always a few versions behind. I don’t mind tinkering to some extent, to get these older games to run – but if you enjoy newer gaming and prefer a hassle-free experience, my advice is to stick with Windows.

  6. Marcos Mora says:

    I think is a shame that mainstream desktop distros don’t even try to comply with Linux Standard Base (LSB). It was intended for this problems.
    One of the most often broken is the Filesystem Hierarchy Standard (FHS), you will never find the same layout on Fedora and Ubuntu, period.
    That said, i think the FHS is a convoluted and ambiguous mess, and the best layout out there is the one in GoboLinux, which also offers the possibility of running any 2 versions of the same lib or app concurrently, helping solve the problem with regressions on common culprits like GCC et al. Plus its portage-like functionality and vainilla app flavors. It implements the FHS on the side by using soft-links, and “this ironically, makes us more compatible with applications that other distros”
    I hope you check it out. GoboLinux.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s