Source code is NOT documentation

It is time to address another long-running pet-peeve of mine: the notion that source code documents a piece of software accurately. On the surface, this might seem to be a plausible notion, but it is in fact a fallacy, if you look beyond that surface.

I think the best way to approach this is to point out that there are multiple phases when writing software. In this case, we can determine the design-phase and the implementation-phase. I will start with the latter.

Implementation-phase

The implementation-phase is the phase where the actual source code is written. By default, the source code is an accurate documentation of the implementation-phase. As a result, the source code is also a good way to find and fix any errors that were introduced during the implementation. However, there is more to writing software than just implementing it…

Design-phase

The design-phase is where the REAL programming happens. Programming is not just about writing code. It is about solving a problem. Writing the code (the implementation-phase) is trivial once the problem has been solved. If writing the code turns out not to be trivial, then apparently the problem was not solved completely yet, and as such the design was not detailed enough.

The design-phase can be taken quite broadly, including such details as what the problem is that is to be solved, and even why that problem is chosen to be solved. You will generally find such details in software documentation, explaining what the software is supposed to do, and who or what it is aimed at. However, you will rarely find such information in the source code, and it is often impossible to derive the intended use or uses from the source code, unless you are already familiar with the problem (in which case you implicitly already know about the design ‘documentation’).

At a more technical level, the design-phase will also specify details of what solution was taken, and why this solution was taken. It might also touch upon other possible solutions, and explain why these have been passed up. Looking at the source code will only show you the solution that was implemented, but nothing about any possible alternatives, or the reasons behind the choice for this particular solution.

Getting even more technical, the implementation may make a number of implicit assumptions. Think about the size of certain buffers, or upper and lower bounds for certain variables, and other ‘constants’ like that. When looking at the source code, you may be able to tell what these ‘constants’ are, but it will generally not tell you why they were designed that way. Were they chosen to match a certain protocol? Do they have to do with the total memory assumed to be available on the machine? Perhaps the CPU’s caches? Again, you need more background information, more documentation to really understand what the source code is doing there, and why. If you understand these things simply by reading the source code, then you apparently already have some prior knowledge about this. You may have read some documentation from similar software at an earlier stage.

Source code is the ‘what’, not the ‘why’

To summarize this, I can put it in a single sentence:

Source code is the ‘what’, not the ‘why’

In other words: it tells you WHAT the software is doing, but not WHY it is doing it this way. If there are errors in the design, they will be hard, if not impossible to find by just studying the source code. It’s similar to semantic errors: the code appears to be doing exactly what it is supposed to be doing, it’s just not what you really wanted to do.

I can give a few examples that I think will demonstrate quite clearly why source code is not a good means of documentation (in the sense that it does not teach you what you really need to know about the software).

Video compression/decompression

An interesting area of software is that of video compression and decompression. There are many popular compression schemes for video, which rely on lossy compression. They ‘leave out’ unimportant details. A lot of study has gone into determining the different levels of detail and encoding them as efficiently as possible. You could take the source code of an MPEG codec, but you’re not likely to understand WHY it works the way it does, without any external documentation. For example, in the case of MPEG, the Huffman algorithm is used for entropy encoding. However, the Huffman codes are pre-determined by the standard, and encoded in a pre-optimized table. As a result, the code looks nothing like the ‘textbook’ Huffman, where you use a tree to look up a code bit for bit (not that any proper implementation for Huffman does that anyway: often such an optimized table is constructed at runtime). It is just a simple lookup in a table that is hardcoded into the source. There is no way of telling where this table comes from, and it may even be hard to understand what the table is supposed to do, if you only look at the source code, and have no further knowledge of the MPEG standard.

Marching Cubes

Another classic example I like to use is that of Marching Cubes: an algorithm to polygonize an isosurface for a 3D scalar field. Even more than with MPEG’s Huffman example, most of the design of the Marching Cubes algorithm is pre-encoded into tables.

The basic idea is that you divide the scalar field into an even 3D grid. This grid can be seen as a set of cubes, through which we will be ‘marching’ while building the isosurface. The idea is that if you can approximate the surface within a cube with a few polygons, you can do this for all cubes, and you have your polygonized surface (divide-and-conquer).

The problem is now reduced to approximating the surface within a single cube. This is done by looking at the 8 vertices of the cube, and classifying them: is the field value at the vertex above or below the isovalue (respectively outside or inside the isosurface volume)? This leads to 8 binary values, leading to 256 different cases for which a surface has to be constructed. By exploiting symmetry (both rotational and inverting), this can be reduced to a limited set of 15 base cases. A transition from outside to inside means that a polygon vertex will be created at the center of the cube’s edge between an outside vertex and an inside vertex. This way the polygons approximating the isosurface are formed.

The implementation will generally do little more than classifying the cube’s vertices and doing a table lookup. The table itself is hard-coded into the source code (generally just the base-cases, expanded to the full 256 cases at runtime, to save space). What this table means exactly, or how it was designed and constructed, that is impossible to determine from the source alone (and also quite impossible to try and document with source code comments. You’ll want images of the base cases to make sense of it all).

The joke is then, that there is a problem with the original algorithm. Most popular implementations will not just place the polygons at the center of each edge, but rather use linear interpolation along the edge to better approximate the surface. This doesn’t work properly, as the inverse symmetry affects the ‘bias’. If a cube was ‘mostly inside’ the isosurface, inverting the classification of its vertices will result in a case that is ‘mostly outside’ the isosurface. When you move the vertices around by interpolation, this will create holes in the surface.

Instead, the inverse symmetry needs to be dropped from the table, and instead an additional 8 base cases with proper bias need to be added. This problem is impossible to solve if you only have the source code for an implementation with the original base cases. And yes, these flawed implementations DO exist. Marching Cubes is not that hard to implement if you understand the algorithms. But it is virtually impossible to derive the algorithm from the source code. Let alone fixing a common bug that creates holes in the surface. You need to know the ‘why’.

Conclusion

I would go as far as saying that source code is only required for two reasons:

  1. To find and fix bugs in the implementation, as explained above.
  2. As a last resort when proper documentation does not exist.

With proper documentation for the design-phase, the implementation should be trivial, and therefore the actual source code provides no extra informational value about the inner workings of the software (aside from flawed implementations, as mentioned in 1. above). More often than not, hundreds of lines of source code can be explained with just a few sentences and/or a few simple diagrams. A picture says more than a thousand words. Naturally you should not take this the wrong way though: you should still aim to make the source code as easy to read and understand as possible. Use proper variable names, structure your code nicely for readability (use indentation and blank spaces to group/separate code). And use comments for anything that is not immediately obvious. After all, you may have to maintain the implementation at a later stage.

Advertisements
This entry was posted in Software development. Bookmark the permalink.

2 Responses to Source code is NOT documentation

  1. snemarch says:

    I hope nobody in their right mind would argue that source code is THE documentation… but IMHO it’s still an important part of the bigger picture. Yes, more complicated algorithms are best explained in external documentation, referenced in the source code – but the source can still provide comments on specific construction optimizations that would muddy up the bigger algorithmic picture (especially if it’s optimizations tied to compiler or machine architecture, and thus likely to change in the future).

    Requirements, architectural decisions, algorithm descriptions et cetera do belong in external documentation. I don’t agree that *all* “whys” belong outside of the source code, though – especially not considering that in real life, external documentation tends not to be kept in sync with the codebase. If I need to work around bugs in frameworks (like, manually specifying Basic Auth HTTP request headers instead of using HttpWebRequest Credentials), I document this *why* in the source code instead of external docs.

    Generally, I try (and sometimes even succeed :P) to reduce the need for comments through code design – small and well-named functions, splitting complex conditionals into a bunch of well-named booleans, et cetera. Comments can then be used for explaining the WHY rather than the WHAT – if you need to describe the WHAT in your source code, you’re probably trying to be too clever for your own good, or are just writing useless comments.

    • Scali says:

      True, at the lowest level, the why’s of some code (eg optimizations etc) can be written into the comments. That is exactly what they’re for. Likewise, I am also a fan of using comment-based documentation like Javadoc/Doxygen to describe classes, methods etc, so you can easily generate an API reference and class diagram based on the actual implementation.
      But as I say, that will generally be at the implementation level, not the design level (I would say that your example of HTTP headers is also an implementation detail, not a design detail).

      I think my point is mainly that a lot of open source projects don’t really do ANYTHING about documentation, because “you have the source”. I would argue that the opposite is much more preferable… If you don’t have the source code, but you have an API documentation, you will have little trouble understanding and using the functionality. But if there is no decent documentation, and all you have is the source, it might actually be very hard to use it.

      If you look at the open source projects that I maintain, you will find that I have comments in source code, and also some extra documentation to explain the code at a higher level.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s