What is software development? An art? Craft? Trade? Occupation?… Part 1

From very early on, I noticed that although some of my friends in school had computers as well, they didn’t all use them for the same things. Some just liked to game on them. Others also liked to do their homework with them. Some liked to play around with a bit of programming as well. And in fact, some had different computers altogether. Some computers couldn’t even do what other computers could, even if you wanted to. So I started to realize that there was quite a bit of variation both between the personalities of computer users, as well as the ‘personalities’ of the computer systems and their software libraries.

As time went on, computers kept changing and improving, and new uses were invented for computers all the time. Although I knew very early on that I was “good with computers”, and I wanted to do “something with computers” when I grew up, it wasn’t entirely clear what that ‘something’ was going to be. At the time I wasn’t sure if that question was even relevant at all. And in fact, there was no way to predict that really.

I had some ‘online’ experience in the early 90s, because I got an old 2400 baud modem from my cousin, and I had dialed some BBSes and downloaded some things. But shortly before I went to university, the internet (mainly in the form of the World Wide Web) started taking off. This quite literally opened up a whole new world, and as I went through university, the internet was busy changing the world of computing altogether. But the education I was receiving was not able to change as quickly, so I was learning many ‘older’ technologies and skills, such as a lot of mathematics (calculus, linear algebra, discrete, numerical etc.), logic, computer organization, compiler construction, databases, algorithms and datastructures, etc. But not much in the sense of web technology. Not that it really mattered to me, it was not the sort of stuff that interested me in computers and engineering.

So when I set out to find my first job, there was demand for a lot of things that I had no prior experience with. Things that barely even existed only a few years ago. And since then, this pattern repeated itself. About a decade ago, smartphones started to emerge. I had no prior experience with developing apps, because the concept didn’t exist yet, when I went to university. Likewise, new programming languages and tools have arrived in the meantime, such as C#, Go, Swift and json. And things started moving to ‘the cloud’.

On the other end of the spectrum, there were things that I have taught myself as a hobby, things that were no longer relevant for everyday work. Like the C64, the Amiga, and MS-DOS. Using assembly had also gone out of style, so to say.

So, conclusion: there are a lot of different technologies out there. It is impossible to keep up with everything, so every software developer will have to focus on the technologies that are relevant to their specific situation and interests. On top of that, there are of course different levels of education for software developers these days. In the old days, software developers would have studied computer science at a university. In the really old days, they may even ‘just’ have studied mathematics, physics or such, and have come into contact with computers because of their need for and/or interest in automated computation.

Apparently there is lots of variation in the field of ‘software engineering’, both in the ways in which it is applied, and the people working in these fields, calling themselves ‘software engineers’. Many different cultures. Far more different than in other types of ‘engineering’ I would say, where the education is mostly the same, there is a certain specific culture, and the people attracted to that field are more similar types of people, I would say.

So what exactly is ‘software engineering’ anyway?

Software engineering has become something of a ‘household name’. In the old days, it was ‘computer programming’. Take for example Donald Knuth’s work “The Art of computer programming”. But at some point, it became ‘software engineering’. Where did this term come from, and why did we stop talking about just ‘programming’, and started talking about ‘developing’ and ‘engineering’ instead? Let us try to retrace history.

It would seem that the meaning of ‘computer programming’ is the process of developing software in the narrowest sense: converting a problem into a ‘computer program’. So, translating the problem into algorithms and source code.

‘Software development’ is developing software in the broad term: not just the computer programming itself, but also including related tasks such as documenting, bug fixing, testing and maintenance.

But how did ‘software development’ turn into ‘software engineering’? This was mainly a result of the ever-increasing complexity of software systems, and the poor control over this complexity, leading to poor quality and efficiency in software, and software projects failing to meet deadlines and stay within budget. By the late 1960s, the situation had become so bad that people were speaking of a “Software Crisis“.

Edsger Dijkstra explained it as such, in his article “The Humble Programmer“:

The major cause of the software crisis is that the machines have become several orders of magnitude more powerful! To put it quite bluntly: as long as there were no machines, programming was no problem at all; when we had a few weak computers, programming became a mild problem, and now we have gigantic computers, programming has become an equally gigantic problem.

(Note also how his article also makes a point very similar to what I wrote above: Dijkstra didn’t know anything about vacuum tubes, the technology that was used for early computers, but was no longer relevant by the time Dijkstra got into programming, because transistor technology had taken over)

So, people started looking for ways to tackle these problems in software development. New tools and methodologies were developed. They saw analogies in various fields of engineering, where mathematics and science were being applied to solve problems, develop machines, infrastructure and such, also within large and complex projects with difficult deadlines and requirements. And so, Software Engineering was born: treat software development as a form of engineering, and apply systematic scientific and engineering principles to software development.

Oh really?

In the next part, we will see if they succeeded.