Scali on Agile Development

I recently wrote The Pessimist Pamphlet. I said there:

I agreed with most of the Agile principles and the underlying reasoning

‘Most of’? So not everything? Well, perhaps I should clarify this. So let’s first walk down the 4 main values of the Agile Manifesto:

Individuals and interactions over processes and tools

Certainly! Tap into the knowledge and experience of people, those are the most valuable assets in your team, or even better, the entire company. Borrow from other teams if you must. Apply critical thinking to find the best solutions, rather than following processes and tools like a Cargo Cult.

Working software over comprehensive documentation

‘Comprehensive’ being the operative word here. I have argued before that source code is not documentation. For trivial stuff, yes, having the source code will be enough to understand what is going on, and why. So you will be able to maintain and extend the code.

But there is another class of code, where there is more to it than meets the eye. I had already given the example of Marching Cubes in the aforementioned article. In that case, the code mainly does some table lookups. The actual cleverness of the algorithm is in how these tables are constructed. And you cannot explain that with just the code and a few lines of comments. You will want some pictures of the base cases to really make sense of it.

I could give our demo 8088 MPH as another example of code that is more than meets the eye. Some parts are cycle-exact. If you were to change even a single instruction, the entire effect may fall apart, because the timing is thrown off ever so slightly. You really need to understand the timing of the instructions and how they interact with the other hardware to understand why certain things work the way they do.

So you certainly want, or even need, to have documentation in these cases. Of course, you will want the documentation to be as clear and concise as possible. But you do want good documentation.

Customer collaboration over contract negotiation

This one seems obvious enough. If you want to be agile in handling changes and unforeseen problems, then you don’t want everything set in stone in a contract.

Responding to change over following a plan

This one is also quite obvious. If there are indications that your current plan is not working properly, you try to make changes for the better.

And now, let’s look at the 12 principles of Agile Software:

Our highest priority is to satisfy the customer
through early and continuous delivery
of valuable software.

This does not only satisfy the customer, it also is more satisfying for the team to see the software coming to life. It also helps to weed out problems early, especially those related to the delivery. You are forced to build, test and distribute your software early-on, so it doesn’t just have to work on the developer’s machine, it has to work everywhere.

The ‘highest priority’-part also seems to imply that delivering valuable software is more important than any kind of management overhead or other red tape.

Welcome changing requirements, even late in
development. Agile processes harness change for
the customer’s competitive advantage.

Changing requirements… They are something like Murphy’s Law in software development, the client never really knows what they want. I think of this as a form of Defensive programming. Expect changes to happen at any point, and try to develop your software in a way that you can incorporate changes wherever and whenever.

Deliver working software frequently, from a
couple of weeks to a couple of months, with a
preference to the shorter timescale.

This seems to be very similar to the first principle.

Business people and developers must work
together daily throughout the project.

Very important. Business people need to understand that software development is very much a creative and unpredictable process. It does not let itself be managed that easily. Perhaps a case of “Keep your friends close, keep your enemies closer”?

Build projects around motivated individuals.
Give them the environment and support they need,
and trust them to get the job done.

Again, this seems to be a hint to the managers. They shouldn’t just try to push the developers, but work with them to find solutions. Especially for managers with no background in software development it is important to trust that developers are not there to frustrate you. If they signal problems, there are problems. If they try to advise you to change course, they have good reasons to do so, because it will benefit their work, and ultimately the end product.

The most efficient and effective method of
conveying information to and within a development
team is face-to-face conversation.

Yes, face-to-face conversation allows direct dialogue. Discussions over email generally do not work that well, because it is mostly an exchange of monologues.

Working software is the primary measure of progress.

I don’t think ‘working software’ is a strong enough criterion. I can think of plenty examples of software that ‘works’, but which is in no way properly designed and maintainable software. You are fooling yourself if you think that the fact that it ‘works’ is a measure of ‘progress’ in this case.

You may have to end up doing huge refactorings or big rewrites to fix the code later in the process, because you have painted yourself in a corner. So all the progress you thought you had made is undone.

Agile processes promote sustainable development.
The sponsors, developers, and users should be able
to maintain a constant pace indefinitely.

This is a very important one. People can only be in ‘crunch mode’ for so long, and fatigue will mean that their progress will slow down, and the quality of the work will degrade. You should avoid any ‘crunch mode’ or overtime or whatnot, and try to keep everyone in the team as fresh as possible, so that they can continue to deliver their best.

Remember, developing software is a creative process. Make sure to keep those creative juices flowing. You can’t manage inspiration.

Continuous attention to technical excellence
and good design enhances agility.

This is similar to what I said earlier about defensive programming. Also, as I said in The Pessimist Pamphlet, this is a huge catch-22: how does the team know what ‘technical excellence’ and ‘good design’ are?

Simplicity–the art of maximizing the amount
of work not done–is essential.

This one I am not sure about. Sometimes it pays off to invest some extra time in developing a more elaborate solution, so that you can more easily extend the functionality at a later point. So this is somewhat in conflict with the earlier points about defensive programming. You will want the simplest solution that does NOT affect technical excellence and good design. But that again is a very vague, subjective thing.

Also, there are cases where other characteristics, such as performance or scalability, are more important than simplicity. A well-optimized, parallelized, scalable solution is inherently more complex than the simplest, most naïve solution that just gets the job done. But sometimes, getting the job done isn’t the requirement. Getting the job done in the shortest amount of time is.

So, the simplest solution that does NOT affect technical excellence, good design, or any requirements (functional or non-functional)?

I could also argue that sometimes finding the simplest solution is the hardest. That is, you first have to create a solution and analyze and study it thoroughly before you can ‘connect the dots’ and start seeing ways to generalize, abstract and simplify the solution. It is a case of iterative refinement.

The best architectures, requirements, and designs
emerge from self-organizing teams.

Again, this one I am not sure of. As I argued in The Pessimist Pamphlet, this assumes that the people who can build the best architectures, requirements and designs are already present inside the team. This is not necessarily the case.

Another objection I have is that we are still talking about science, computer science. In the history of science, breakthroughs were more often than not the result of a brilliant leap of mind of a single individual, rather than some team working towards a solution. Which brings me back to the earlier objection about working software and documentation: In the trivial case it may be true, but not all problems are trivial.

Sometimes a single person can make all the difference, by viewing the problem in a unique new way, and opening the door to solutions that the others were blind to until then. This person might not necessarily be on your team.

This is also what is argued in the No Silver Bullet article: great designers are very rare. You can’t expect every software team to always have just the right ‘great designer’ on board for the specific problems they run into.

At regular intervals, the team reflects on how
to become more effective, then tunes and adjusts
its behavior accordingly.

I think this last one is obvious enough. Apply the Agile principles not only to software development, but also to the structure and workflow of the team itself: don’t just follow a plan, welcome changes for the better.

So, in conclusion, I suppose the trick (or catch-22 if you will) is to build a team that has all the relevant skills and insights to be able to properly reflect on their own work, know what technical excellence and good design are, and how to get there. This calls for a heterogeneous team buildup. A team consisting entirely of younger, less experienced programmers, especially if they all have similar education and interests, is a recipe for disaster. As they say: in the land of the blind, the one-eyed man is king.

Also, you should see the Agile Manifesto (or any other kind of process/methodology) as a set of guidelines, not strict rules. The key is to use common sense in when to follow the rules, and when to pick a better option in a specific situation.

This entry was posted in Software development and tagged , , , , , , , , , , . Bookmark the permalink.

4 Responses to Scali on Agile Development

  1. axeldominator says:

    “In the history of science, breakthroughs were more often than not the result of a brilliant leap of mind of a single individual, rather than some team working towards a solution.”

    I work in the video game modding industry and this is usually exactly what happens: mods are very simple and limited at the beginning, until a very gifted individual unlocks the true potential of the game. That’s usually a modding library of some sort or a new method of accessing internal functions… something like that. This person comes up with examples on how to implement advanced features and immediately after that the modding scene flourishes with lots of other people using those examples to add the most diverse things into the game.

    In general, every time I’ve been involved in a creative project, either the entire thing or a big section revolves around a talented individual. I can’t stress how important it is for these creative minds to be in an environment where they can express themself at their fullest.

    • Scali says:

      Exactly, I suppose the phrase “Standing on the shoulders of giants” is appropriate here.
      Every now and then, you need that ‘giant’ to take things to the next level, then the rest of the team can build on that.

      Software development is one of those areas where there can be huge differences between capabilities of developers, like the phenomenon of the ’10x programmer’. Some developers really are 10x ‘better’ than others, or even more. Orders of magnitude.

      That is something you simply cannot ignore. The No Silver Bullet article clearly stresses this phenomenon, but the Agile Manifesto or Scrum methodology certainly do not explicitly mention it. In my interpretation, you can read it between the lines, but I don’t get the feeling that this is how most people see it, and as such how they implement it in their teams.

  2. Pingback: What is software development? An art? Craft? Trade? Occupation?… Part 2 | Scali's OpenBlog™

  3. Pingback: Read any good books lately? | Scali's OpenBlog™

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s