Porting BHM3DSample to Android: Some… well… a lot of… stressful development

As you may know, I have ported my OpenGL rendering framework to iPhone a few months ago. Originally I was not all that interested in Android, since it apparently uses Java for its apps. While it has OpenGL ES support in Java, it would require me to rewrite the entire framework from C++ to Java. With the iPhone I had a similar situation where Apple wants you to write things in Objective-C. However, as it turned out, there was a way to use regular C/C++ on the iPhone, so in the end I could just use most of my OpenGL framework as-is.

Looking a bit further into Android, I found that Android not only has the SDK for Java-based applications, but also an NDK, for more low-level things. The NDK allows you to use C/C++ and even assembly. Now that’s more like it! So I checked out the NDK, and spotted an example using OpenGL ES from C++. Exactly what the doctor ordered!

Things quickly went downhill from there though… Where I was used to having some a proper IDE for the iPhone and for J2ME, the Android SDK and NDK had a distinct 1970s feel to them: just a set of headers, libraries and some commandline tools slapped together. There was not too much in the way of documentation. It’s mostly learn-by-example, apparently. It didn’t look good. What’s worse, there are a few snafu’s in the SDK and NDK. For example, the Windows version of the SDK installs itself in your Program Files by default. However, writing to Program Files requires administrator rights. But the SDK manager is not installed to have these rights by default. So if you start it, and try to download and install any packages, it will fail to write them. And the NDK has some build/make scripts, which can’t handle spaces in pathnames. So if you put the NDK in Program Files together with the SDK, it won’t work.

Google does offer an Android plugin for Eclipse though, so I figured I’d give that a try. I never liked Eclipse, as I’ve always found it to be slow, cumbersome and generally unstable. For my Java development (including J2ME) I generally use Netbeans. But well, this time I had no choice, so Eclipse it is…

Eclipse did not disappoint… It was still as slow and cumbersome as I remembered it… and from time to time it would also crash… One time it even corrupted the workspace, so it would hang the next time it tried to open it. All I could do was delete the metadata in the workspace and start over.

However, at least the Android plugin was reasonably nice. At least I had a simple tool to build and deploy Android apps in the emulator. Oh yes… the emulator. Another part of the success-story that is the Android SDK. The emulator is extremely slow, and takes up lots of memory. Even on a modern high-end machine like an Intel Core i7, it still takes ages to boot the emulator up. And it actually runs considerably slower than a real phone.

Right, well… now that I had familiarized myself with the SDK somewhat, it was time to actually develop something. It quickly became apparent that you could not get rid of Java completely: the NDK can produce libraries, but they will have to be called from a Java app through JNI. I’m no stranger to JNI myself, I have used it before to add a fullscreen option to the Java demo system used for Croissant 9 on Windows. It involved a simple DLL that set up DirectDraw, and the Java application could pass its pixelbuffer to it through a JNI call.

But JNI has always been rather obscure, and quite archaic in nature. It’s difficult to find good documentation on it, especially now that Oracle has taken over Java, and the old Java.sun.com webpages are gone. A lot of dead links. After some digging, I managed to find a reasonable overview of the JNI calls though. Looks like nothing changed in the roughly 10 years since I wrote my JNI DLL. The Java side is quite simple: just declare a function with ‘native’, and call System.loadLibrary() in the static constructor to load your native library. The system works entirely via reflection, so all linking is done dynamically at runtime.

The C++ side is not so pretty. Firstly you have to give your functions very specific long names in order to make the reflection work in the JVM. Secondly, you also need to use reflection yourself, when you want to call methods on Java objects. What makes it even more cumbersome to use is that you cannot use any of the data directly. You need to perform marshaling via the JNIEnv object in order to access strings, arrays and such in a form that C++ can use. And since the NDK has only limited functionality, you will need use Java objects and functions in C++ from time to time.

To make matters worse, it does not seem to be possible to debug the native code in the emulator. The only thing I managed to do was to output log messages in Android’s logCat, so I at least had some idea of what was going on.

And no, the horror does not stop there… No… Once I got some basic C++ code running inside a Java app, I tried to compile the actual C++ framework for OpenGL ES, which I had used on the iPhone earlier. Problem #1: My code was for OpenGL ES 2.0, but the emulator only supports 1.0. So I could not use shaders and things, and had to go back to fixed function, and using CPU-emulation for the skinning. And for some reason, the GPU emulation is disabled by default… So I first had to reconfigure my emulator to get it to work… Problem #2: There was no STL support in the NDK.

Or at least, that’s what it looked like originally. Googling for some information turned out to be a wild goose chase. People trying to port uSTL or STLPort to Android. That’s what I tried initially, but that didn’t quite work. Luckily I later found out that Google has since added its own limited STLPort to the SDK. It is just disabled by default. But apparently if you add an Application.mk to your project, you can put the following statement in there to enable STL:

APP_STL := stlport_static

Now I could finally get my code compiled. Next step was to actually make it work. The biggest problem here is that you can’t just access files. I wanted to bundle the data files in the Application Package (.apk) itself, much like I did with the iPhone version. But this was not easy… like pretty much everything else in Android development. Eventually I figured out that I had to put it in the assets-folder. And then you can open it from Java, and get an inputstream. But that is still quite useless, since I needed to access the file in C++, not in Java. So I again had to pull some nasty JNI trickery to finally get things to work.

There were a lot of other minor things that delayed the port further… but in the end I did more or less get it to work:

Okay, so the textures still don’t work… But that should just be a formality at this point. I already managed to get the BHM file loaded from the assets, so getting a JPG to load shouldn’t be much harder. There are some Java helper functions for that, which I will be using.

And here it is, running on an actual phone:

All in all, it wasn’t a very pleasant experience. Google may have this geeky/nerdy/linux/developer-like image, but the Android development tools are horribly archaic, immature and just generally unprofessional. Apple has done a much better job on the development tools for iPhone and iPad. And this coming from a guy who likes to code classic Amiga and 16-bit DOS with Hercules/CGA/EGA for fun! This was an absolute nightmare compared to that. I’d almost be inclined to say that the extra money you spend on the iPhone and getting an iPhone developer account is worth the extra money. Decent tools make your life so much easier!

Update: Texturing has been fixed in the meantime. The problem was that the texture filtering was still set to mipmapping, while the mipmaps were not being generated by the texture loading code. For some reason the emulator showed the texture correctly, while they did not appear on a real device, making it difficult to debug with just the emulator. Here it is running on an actual phone:

And here are the binaries (for Android 2.3 or higher):

OpenGL ES 1.0 (should run on virtually any device): https://www.dropbox.com/s/0wrpjdc36bbwcxq/OpenGLES.apk?dl=0

OpenGL ES 2.0 (faster, because it uses shaders for skinning rather than the CPU): https://www.dropbox.com/s/bl8bygku1ppacla/OpenGLES2.apk?dl=0

This entry was posted in OpenGL, Software development and tagged , , , , , , , , , , , . Bookmark the permalink.

2 Responses to Porting BHM3DSample to Android: Some… well… a lot of… stressful development

  1. I know what you mean! W VisualStudio and Amiga with it’s Almighty DevPac!


  2. Pingback: What is a platform anyway? | Scali's OpenBlog™

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s