After the series of articles I did on software development a while ago, I figured that the term ‘Software Architect’ is worth some additional discussion. I have argued that Software Engineering may mean different things to different people, and the same goes for Software Architecture. There does not appear to be a single definition of what Software Architecture is, or what a software architect does and does not do, what kind of abilities, experience and responsibilities a software architect should have.
In my earlier article, I described it as follows:
Software Engineering seemed like a good idea at the time, and the analogy was further extended to Software Architecture around the 1990s, by first designing a high-level abstraction of the complex system, trying to reduce the complexity and risks, and improving quality by dealing with the most important design choices and requirements first.
I stand by that description, but it already shows that it’s quite difficult to determine where Software Engineering ends and Software Architecture starts, as I described architecture ans an extension of engineering. So where exactly does engineering stop, and does architecture start?
I think that in practice, it is impossible to tell. Also, I think it is completely irrelevant. I will try to explain why.
Two types of architects
If you take my above description literally, you might think that software architects and software engineers are different people, and their workload is mutually exclusive. That is not the way I meant it though. I would argue that the architecture is indeed an abstraction of the system, so it is not the actual code. Instead it is a set of documents, diagrams etc, describing the various aspects of the system, and the philosophies and choices behind it (and importantly, also an evaluation of possible alternatives and motivation why these were not chosen).
So I would say there is a clear separation between architecture and engineering: architecture stops where the implementation starts. In the ideal case at least. In practice you may need to re-think the architecture at a certain phase during implementation, or even afterwards (refactoring).
That however does not necessarily imply that there is a clear separation between personnel. That is, the people specifying the architecture are not necessarily mutually exclusive with the people implementing it.
The way I see it, there are two types of architects:
- The architect who designs the system upfront, documents it, and then passes on the designs to the team of engineers that build it.
- The architect who works as an active member of the team of engineers building the system
Architect type 1
The first type of architect is a big risk in various ways, in my opinion. When you design a system upfront, can you really think of everything, and get it right the first time? I don’t think you can. Inevitably, you will find things during the implementation phase that may have to be done differently.
When the architect is not part of the actual team, it is more difficult to give feedback on the architecture. There is the risk of the architect appearing to be in an ‘ivory tower’.
Another thing is that when the architect only creates designs, and never actually writes code, how good can that architect actually be at writing code? It could well be that his or her knowledge of actual software engineering is quite superficial, and not based on a lot of hands-on experience. This might result in the architect reading about the latest-and-greatest technologies, but only having superficial understanding, and wanting to apply these technologies without fully understanding the implications. This is especially dangerous since usually new technologies are launched with a ‘marketing campaign’, mainly focusing on all the new possibilities, and not looking at any risks or drawbacks.
Therefore it is important for an architect to be critical of any technology, and to be able to take any information with a healthy helping of salt, cutting through all the overly positive and biased blurb, and understanding what this new technology really means in the real world, and more importantly, what it doesn’t.
The superficial knowledge in general might also lead to inferior designs, because the architect cannot think through problems down to the implementation detail. They may have superficial knowledge of architectural and design patterns, and they may be able to ‘connect the dots’ to create an abstraction of a complete system, but it may not necessarily be very good.
Architect type 2
The second type is the one I have always associated with the term ‘Software Architect’. I more or less see an architect as a level above ‘Senior Software Engineer’. So not just a very good, very experienced engineer, but one of those engineers with ‘guru’ status: the ones you always go to when you have difficult questions, and they always come up with the answers.
I don’t think just any experienced software engineer can be a good architect. You do need to be able to think at an abstract level, which is not something you can just develop by experience. I think it is more the other way around: if you have that capability of abstract thought, you will develop yourself as a software engineer to that ‘guru’ level automatically, because you see the abstractions, generalizations, connections, patterns and such, in the code you write, and the documentation you study.
I think it is important that the architect works with the team on implementing the system. This makes it easier for team members to approach him with questions or doubts about the design. It also makes it easier for the architect to see how the design is working in practice, and new insights might arise during development, to modify and improve the design.
Once an architect stops working hands-on with the team, he will get out of touch with the real world and eventually turn into architect type 1.
I suppose this type of architect brings us back to the earlier ’10x programmers’ as I mentioned in an earlier blog. I think good architects are reasonably rare, just like 10x programmers. I am not entirely sure to what extent 10x programmers are also architects, but I do think there’s somewhat of an overlap. I don’t think that overlap is 100% though. That is, as mentioned earlier, there is more to being a software architect than just being a good software engineer. There are also various other skills involved.
The role of an architect
If it is not even clear what an architect really is or isn’t, then it is often even less clear what his role is in an organization. I believe this is something that many organizations struggle with, especially ones that try to work according to the Scrum methodology. Allow me to give my view on how an architect can function best within an organization.
In my view, an architect should not work on a project-basis. He should have a more ‘holistic’ view. On the technical side, most organizations will have various projects that share similar technology. An architect can function as the linking pin between these projects, and make sure that knowledge is shared throughout the organization, and that code and designs can be re-used where possible.
At the management level, it is also beneficial to have a ‘linking pin’ from the technical side, someone who oversees the various projects from a technical level. Someone who knows what kind of technologies there are available in-house, and who has knowledge of or experience in certain fields.
Namely, one of the first things in a new projects is (or should be) to assess the risks, cost and duration. The architect will be able to give a good idea of technology that is already available for re-use, as well as the people best suited for the project. Building the right team is essential for the success of a project. There is also a big correlation between the team and the risks, cost and duration. A more experienced team will need less time to complete the same tasks, and may be able to do it better, therefore with less risk, than a team of people who are inexperienced with the specific technology.
Since architects are scarce resources, it would not be very effective to make architects work on one project full-time. Their unique skills and knowledge will not be required on a single project all the time. At the same time, their unique skills and knowledge can also not be applied to other projects where they may be required. So it is a waste to make an architect work as a ‘regular’ software engineer full-time.
With project managers it seems more common that they only work on a project for a part of their time, and that they run multiple projects at a time. They balance their time between projects on an as-needed basis. For example, at the start of a new project, there may be times where 100% of their time is spent on a single project, but once the project is underway, sometimes a weekly meeting can be enough to remain up-to-date. I think an architect should be employed in much the same way. Their role inside a project is very similar, the main difference being that a project manager focuses on the business side of things, where the architect focuses on the technical side of things.
This is where the lead developer comes in. The lead developer will normally be the most experienced developer in a team. It is the task of the lead developer to make sure the team implements the design as specified by the architect. So in a way the lead developer is the right-hand man (or woman) of the architect in the project, taking care of day-to-day business, in the technical realm.
The project manager will need to work both with the lead developer and the architect. As long as the project is on track, it will be mainly the project manager being informed by the lead developer of day-to-day progress. But whenever problems arise, the current project planning may no longer be valid or viable. In that case, the architect should be an ‘inhouse consultant’, where the project manager and lead developer ask the architect to analyse the problems, and to determine the next course of action. Was the planning too optimistic, and should milestones be moved further into the future, at a more realistic pace? Did the team get stuck on a programming problem that the architect or perhaps some other expert can assist them with? Does the design not work as intended in practice, and does the architect need to work with the team to modify the design and rework the existing code?
The holistic position of the architect also allows him to look at other development-related areas, such as coding standards, build tools etc. And the architect can keep up with the state of technology from research or competing companies, and help plan the strategy of a product line. Likewise, the architect can always be consulted during the sales process. Firstly to answer technical questions about existing products. Secondly, when a potential customer wants certain functionality that is not yet available, the architect can give an expert opinion on how feasible/costly it will be to implement that functionality, and what kind of resources/personnel it would take. In the more complex/risky cases, the architect might spend a few weeks on a feasibility study, possibly developing a proof-of-concept/prototype in the process.
Finally, I would like to reference a few resources that are somewhat related. Firstly, here is an interesting article discussing strategy: https://www.mindtheproduct.com/2018/03/growing-up-lean/
As I argued in my earlier blogs on Agile/Scrum… these methods are often not used correctly in practice. One of the reasons is because people try to predict things too far into the future. This article describes a very nice approach to planning that is not focused on timelines/deadlines/milestones as much, but on current, near term and future time horizons, where the scope is different for each term. The further things are away, the less detail you have. Which is good, because you can’t predict in detail anyway.
The second one is the topic of Software Product Lines: https://en.wikipedia.org/wiki/Software_product_line
It is a great way to look at software development. This goes well beyond just ‘design for change’, and the idea and analogy to product lines such as in (car) manufacturing may make the way of thinking more concrete than just ‘design for change’.