The problem with free/open source software

I have recently obtained two Apple Mac Mini G4 machines. They date from 2005, but I did not immediately associate them with retro-programming (I would think they’d have to be quite a bit older for that). They both were unable to boot, and it was not clear whether this problem was hardware- or software-related. It was up to me to find out.

Originally these machines came with OS X 10.4 (Tiger). Since it looked like I had to reinstall them anyway, I went looking for the newest version that would run on them. Apparently that would be OS X 10.5.8 (Leopard), released in 2009. Not very retro, you might think.

I figured I would reinstall them, to see if the machines were actually working, and then install some software on them so they could be used for some daily tasks, and perhaps for some development. I had already done some OS X development a few years back, porting my OpenGL code to OS X and later to iOS. But at the time I just used a virtual machine. This time I have actual Apple hardware.

The hardware seemed to work fine. Software was a different story though. As I found out, support for the PowerPC architecture has died out. Software that is distributed in binary form, is generally only available for x86 machines. For PowerPC binaries, you are limited to legacy versions of the software (I don’t think I have mentioned it before, but the site is a very nice place if you’re looking for older versions of software, such as the last version running on a particular version of an OS).

But, for open source software, there is also MacPorts. It is very similar to the ports system on FreeBSD (and fitting, since OS X is also partially based on FreeBSD). For linux users: you know Gentoo? Well, their Portage is a rip-off of BSD ports (however, Gentoo wants to market the compilation/optimization as having more performance, unlike BSD). So that’s more or less what MacPorts is as well: a distribution system based on source, where you compile the binaries on your own machine. In theory that would mean that it doesn’t matter whether you’re running PowerPC or x86.

However, I have come to the conclusion that free software is free and open under the following conditions:
– You can use any compiler as long as it’s gcc (with some tight version requirements)
– You can compile for any OS as long as it’s linux
– You can compile for any architecture, as long as it’s x86

With FreeBSD you already notice some of these issues. Quite a few open source projects will not compile on FreeBSD out-of-the-box (even on x86), and need specific patches. Which means that it generally takes a while longer for new versions of software to become available through FreeBSD ports. FreeBSD also wants to move from gcc to LLVM/Clang as the default compiler (as part of their greater goal of removing GPL’ed software from FreeBSD, since GPL, especially GPLv3 does not align with FreeBSD’s views of free software. On the one hand I am not too happy with that, because I’ve always liked FreeBSD’s non-political approach to licensing: their own license gives you a lot of freedom, but they did not exclude software with other licenses. On the other hand, GPLv3 does seem to go a bit overboard with its restrictions, as we’ve already seen with things like UEFI secure boot) . But it will take a while before everything works with a compiler other than gcc.

MacPorts is trailing even further behind (it has a smaller team, and OS X differs more from linux than FreeBSD does), and even though it targets OS X machines specifically, not everything will work on PowerPC.

Another issue that you find with MacPorts is that it often pulls in a lot of dependencies. For example, OS X already has a GUI, OpenGL, and even built-in X11, but even relatively simple applications will pull in all sorts of specific GUI libraries such as Qt, GTK, the Xorg libraries, even Mesa, to glue things together. The software is not really ported to OS X, it’s more like you’re compiling a whole bunch of packages so that your system is camouflaged into a linux system, after which the software can be compiled for it. I found that rather strange and annoying with things like Cygwin on Windows already. But OS X is actually a certified UNIX out-of-the-box, so it is rather sad that you still have to rely on so much redundant software.

Another problem that is very annoying, is that different ports can be out-of-sync. I ran into two problems with MacPorts already. The first was mercurial and TortoiseHg. TortoiseHg was an older version, and it could not work together with the newer version of mercurial. Which is quite strange anyway… it would have made slightly more sense the other way around: a new GUI tool which may have options not yet supported in the older commandline tools… but in this case apparently the commandline tools have changed so much that the GUI tool just breaks (there is a version check in thg, but after I disabled it to see if it would work anyway, it would just get into endless recursion and crash on a stack overflow). Luckily TortoiseHg is mainly a set of python scripts, so it was easy to just grab the newer scripts from the repository and overwrite the installed ones. That was enough to make it work with the installed version of mercurial.

The second problem was with mono and monodevelop. Although it is possible to build mono for OS X, I ran into a lot of problems with building monodevelop, since the sources required a newer version of mono, which, much like the above example, had some changes in crucial places of the interface, so it was not compatible.

And even though I eventually managed to get the right versions of mono and monodevelop sources together, I could build it, but I could not run it. Mono can only run GUI-based applications via X11 anyway, and only GTK seems to actually work on OS X. WinForms applications are missing some crucial libraries or functions, so they will never work anyway. And monodevelop apparently also depends on various GUI things that are not available on my OS X build.

These kind of dependencies are becoming a big issue with open source software. Where closed source software generally has a very stable interface between libraries, utilities and whatnot, open source software seems to be in a constant state of flux, and not much attention is paid to forward and backward compatibility. Ingo Molnar, a linux kernel developer working for Red Hat, has also written a two-part story on that on Google+ a while ago.

In short, he is arguing that open source should become more stable. While unlike closed source, you have the ability to modify and recompile applications, this should not be used as a crutch. But that is exactly what is happening: application interfaces are changed ad-hoc to introduce new features or to solve some design issues. But no attention is paid to how this would affect other software depending on it. They just assume that other people can just patch those sources to work with newer versions, and recompile.

This is also common inside the linux kernel itself, which causes problems with nVidia and AMD, who want to keep the sources of their display drivers closed, since their sources would contain a lot of optimizations, patented algorithms and whatnot, which may not end up in the hands of their competitors.

Anyway, I am guilty of some of these things myself. After I set up one of the Mac Mini’s for development, I tried to get the BHM 3D sample running on it. I ran into three problems with my code:

1) My code made use of __sync_add_and_fetch(), which was introduced in gcc 4.1. The default compiler for Xcode on my system was 4.0, so it would not compile. Luckily 4.2 is also available, and after switching to that, it worked. However, it still does not work with LLVM gcc (or on older versions of gcc for that matter). Then again, at the time of writing the code, I had accepted that it would only work with MSVC and gcc, which should cover most systems. The code was written as portable as possible, and only the __sync_add_and_fetch() would need to be replaced for other systems. A proper replacement would require architecture/compiler-specific inline assembly routines however, so I figured it would be beyond the scope of my little example project to include support for a lot of compilers/architectures.

2) My code assumes a Little Endian architecture. The funny thing is, I had totally forgotten about that. I recall that we had a discussion on how to handle Endianness when we first designed the format. Should we allow both Endiannesses, and use some flag to indicate which one is being used? Or do we just support one Endianness for all data? In the end we chose to support only Little Endian, since it is the most common (which pains me somewhat, as a 68k-man, who has always liked Big Endian). If we were to allow both types in our files, it would make parsing more complex, since you had to be able to support both types on all systems. We figured that it was still possible for developers to store Big Endian data inside chunks, by simply defining a new ID for the Big Endian variation of that datatype.

However, since we only used x86 at the time, our implementation simply assumed that all bytes always came in the correct order. Even though I ported it to various systems over time, they were always Little Endian (even ARM, which can run in both modes). So I never gave the Endianness issue another thought. Until I tried to compile it on the G4 that is. Although it can run in both modes, Apple always runs their PowerPCs in Big Endian mode, because the previous 68k-based Macs were Big Endian as well.

So now my code did not work at all. It could not parse the headers of the BHM file correctly. So, I have made some updates to the code to make it work on Big Endian machines as well, so that after all these years, BHM is finally truly portable to other architectures.

3) I had cleaned up the legacy code from the OpenGL framework, making it require OpenGL 2.0 as a minimum. Normally that is quite a reasonable requirement. However, my Mac Mini has a Radeon 9200 GPU, which does not support OpenGL 2.0 (it falls exactly inside that dreaded black hole in the OpenGL API where SM1.x should have been. Under DirectX 8.1 the hardware would be a lot more capable than under OpenGL). So I had to revert back to an older version of the OpenGL code, which still used the legacy pipeline as a fallback when no shaders were available. I am not going to revert the publicly available source code back to this legacy version though. I still think OpenGL 2.0 is a reasonable minimum, and not having legacy support makes the code simpler and cleaner. This platform should be considered ‘retro’, and should not affect the main code. If people want/need a legacy version, they can still get an older version of the code from the repository.

All in all, the Mac did help me to clean up the code somewhat, and address some long-forgotten issues. After I made the OpenGL code work, I also managed to port my QuickTime code from Windows to OS X (which is more work than you might think), and now I can use video textures in OpenGL. I also want to support live capture devices soon.

This entry was posted in Oldskool/retro programming and tagged , , , , , , , , , , , , , , . Bookmark the permalink.

4 Responses to The problem with free/open source software

  1. T. says:

    What do you think of Intel compiler for Linux?

  2. Marcos Mora says:

    It has always been odd to me, that there are tons of papers that get into the trouble of specifying a standard for a language (like the recent ISO/IEC 9899:2011 AKA C11) yet not 2 compilers bother with it.
    I think the whole LLVM C implementation is in part a reaction against this issue and the perceived “only for GCC specific extensions” ecosystem.
    I have always loved the idea of portable code, but is just not easy to find suitable common denominator.

  3. Pingback: When old and new meet: lessons from the past | Scali's OpenBlog™

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s