The Skeptical Methodologist

Software, Rants and Management

What is design?

I work in a field where people are paid awfully large sums of money to ‘design’ software.  Yet there’s a rather large argument to be made that software is nothing more than a design.  How can you design a design?  People will start religious wars over design techniques, but do any of us really have any clue what it is?

I have two theories, one I’ll call the Compass theory of design and the other I’ll call the Seed Crystal theory of design.

Compass Theory of Design

The Compass theory of design posits that basically after we have a reasonable ideas of our requirements, design is a direction we want to move the implementation in.  Let’s assume all software starts as nothing but a “Hello World”.  Design in this case is a direction we want to move our “Hello World” in, or a function which maps current code to another proposed refactor. We take the specs as is (not assuming they won’t change, of course), our experience and our talents, and we come up with a general direction to move the code and call this a design.

An important part of thinking of design in this way is that as soon as you take the first step, your design is already out of date.  Your crudely drawn UML diagrams or rough English sketch of what you want to do is nothing but a general feeling at a certain point in time – the direction the needle pointed at time T.  That does not mean the needle will continue to point there – or, more importantly, that when you head in that direction you won’t run into impassable cliffs or tigers or something.  Sometimes we head in one direction and then realize it’s a dead end and we actually need to head in quite another.  This kind of design works best with the idea of “patterns”, or Fowler’s “refactoring to patterns”.  Patterns in this case are things that have worked in the past to build software, and as such they may be a direction we head in again – they are a well worn path and familiar.

When thinking of design as a compass, we frequently find ourselves returning to the drawing board and redesigning.  This is OK, and it means we’re making progress.  If you think of it this way, 99.999% of our designs will not solve the problem.  Chances are that if we have a design and we aren’t actively finding out it’s wrong by attempting to implement it and finding it’s faults, we’re not making progress.  Too often we think this lack of redesign means we’ve stumbled upon that .001% of solution early but the odds are against us.  If things are going smoothly, be worried.  Contrarily, if things are being redesigned every day, relax.

Compass driven designs work well against open problems – problems where it’s best to constantly re-evaluate where we are and where we need to go.  Hence the emphasis on patterns: in Terra Incognita, any bit of familiarity is sacred and should be protected.  In open problems where the overall architecture is as yet unknown and unsolved, any bits and pieces we can move towards well understood solutions we should take.

Notice that this theory of design assumes iterative development.  In fact, both theories of design will assume iterative development.  The fact of the matter is that Waterfall style design doesn’t make the damnedest bit of sense in the world, since it conflicts with the idea of Software as a design.  What is often called ‘design’ in Waterfall circles is frequently referred to by some overpaid, old fogey architect as ‘high level’ or ‘abstract‘ view of things, which leaves only the ‘implementation details‘ to be filled in by some code monkey they pulled off the street.  Designs of this sort are not abstract!  Software abstraction means something very specific(ironically): the generalization of a repeated pattern.  If you have three objects that behave similarly, you abstract out a common interface.  In some particularly skilled ‘architect’ hands, we do find actual anticipated abstraction, with well defined interfaces and all.  But in most cases, what you really have is an underspecified piece of napkin art.  The big decisions, much to our constant surprise, are easy in software.  It’s the small decisions that kill us.

The Seed Crystal Theory of Design

This is another view of design – sometimes these two views can occur on the same project.  This view is a little more compatible with Waterfall, even though it too assumes rapid, iterative development.  In this case, we assume software is an evolutionary process.  It is the process of taking one piece of working code and changing it into another, slightly different piece of working code.  The fitness function in this evolutionary process is how well the software fits the requirements and customer needs.  This implies some meta-evolution is taking place since the fitness function itself is always changing but that’s really beyond the scope of this article.  In the case of software that evolves, it has to start somewhere.  Now, again we can start with a simple “Hello World”, but in this case we take a different path.

The “seed crystal” of design is a good first guess of what the final implementation ought to look like.  It probably will have x,y and z interfaces, it might use a, b and c algorithms.  Compared to starting at a “Hello World” and Test-driving development our way to a substantial piece of software, an initial sketch or design can do wonders to jump start your efforts.  The drawback is that you may very well end up with things in your design that the requirements end up not requiring.  And as your software crystallizes around the initial seed of design, expanding ever outward, you may find out only later that substantial portions of your original thought need to be revamped.  Seed crystals are great when you have substantial amounts of prior art – you’ve built a system like this before.  Against open problems, you’re just as likely to move away whatever seed crystal you start with as you are to move towards it, and any seed crystal you come up with against an open problem will be so sorely underspecified as to be a hindrance rather than a helper.


Ultimately, I believe we need both forms of design.  Try as agilistas might, there’s always that first initial step that’s going to be bigger than all the others.  Once we get a good running start, we can iteratively develop ourselves into something really great.  But the dictum of “think then develop” will always have some authority.  We need to have a good idea of where to start from, and that takes some thought.  On the flip side, agile style design is like a Compass.  Once you get going, you rely on your spidey sense, experience, collaboration and creativity to give you a good idea of where to move the implementation after each refactor.  This too requires the same skills as traditional ‘big up front’ design, just spread out throughout the entire project.

In other words, when you find yourself with a lot of momentum, make sure to stop every now and then and ensure your refactors are pushing your implementation in the right direction: check your compass.  And when you find yourself about to plunge into something totally new, give that something an initial seed to crystallize around and give it some cohesion.  These two design activities should go hand in hand with traditional agile methods like TDD and iterative development, working hand in hand to ensure you don’t try and test-drive yourself into a Sodoku solver.

May 2, 2010 Posted by | Uncategorized | Leave a comment