The Skeptical Methodologist

Software, Rants and Management

Risk and Incremental Development

In an earlier post, I talked about D’Souza’s idea of Real Options in project planning, explaining what an option was and applying it quite literally to a software project between a vendor and customer. This time I want to talk about how the idea of options and the actual planning steps that must be taken to incorporate them, as well as the financial concepts of liquidity and volatility imply that Incremental Development is inherently less risky than so-called formal methods.

Let’s go back to real options. In the most basic interpretation, real options in a project are decisions you build into your schedule to hedge risks. They can be as basic as go / no-go at predetermined moments, or enumerate many different paths forward from any particular point in the future. Additionally, these decisions can be structured to behave like put and call options from finance, limiting risk on any particular project moreso than hands-on control which is subject to behavioral biases like sunk costs.

If you have started to incorporate real options into project planning you may notice something – options more or less can only be incorporated at milestones. We must first learn something before we can make any decisions about it. Like control theory, there must be some sort of input before there can be feedback.

Recall that I argued in my earlier post a trader simply promising to himself he’d sell at a certain price is still more risky than options, and one of the reasons for this risk was due to the fact that prices in stocks can suddenly move very quickly, especially during overnight periods (called ‘gaps’). A trader might watch a ticker all day, getting one update a second, and then see two prices differ wildly because one was the closing price yesterday and the other the opening price today. Value changes even though much less trading is taking place.

A quickly moving value is called volatile, and one measure of volatility is basically the statistical variance in price movements over time. Volatility changes in time as well, leading to periods of relative calm as well as stormy weather. Projects can be volatile too – especially at the beginning. As estimates and schedules continue to change and churn, the value and cost of the project changes drastically from day to day.

The stock itself is not actually more volatile – if we only look at second to second trading, it looks nothing like close to open gaps. What we’re actually witnessing is illiquidity. A liquid asset is one that can be bought and sold easily. During trading hours, most stocks are incredibly liquid. But after trading hours liquidity dries up. There’s private over the counter markets, but no public markets to trade on. In control theory, liquidity can be thought of in terms of how often input comes in, and how often feedback is allowed. During times of illiquidity, input is sparse and opportunity for feedback is rare.

In an illiquid market, volatility appears to rise simply because we more or less aren’t allowed to observe the trades in between. For instance, in the housing market, which is relatively illiquid right now, it may seem volatile because my house price can drop by $30,000 in a day or two. This is because, since houses aren’t moving, the actual time between measurements – i.e., the selling of comparable houses – spreads out.

If we want to reduce our exposure to this liquidity risk, we want to stay in liquid assets. We want to change the rules of the market such that they’re open all night, so we are no longer exposed to these large price gaps.

Additionally, when planning projects, milestones are input and decisions, or real options, are feedback. The time between milestones exposes the project to illiquidity risk. If you want to get out between two milestones, you’re out of luck (or at least, its much harder) than if you simply exercise an option built into the schedule from the beginning at the appropriate time. How can we reduce this risk? The same way we might reduce the same type of risk in finance – we add liquidity. In the case of project management, we have to add more opportunities for feedback and input.

This more or less means more milestones. A lot more. As many as you can think of, with options spread throughout.

There’s another major issue in project planning, though, laid bare through the use of Real Options. As described in his paper, D’Souza uses decision trees and options to better estimate Net Present Value, or the value of any particular project. When we add in options that allow us to walk out, we don’t necessarily have to take the project as a total loss, indeed, we can lower the cost of failure paths if we arrange our project to best take advantage of residual value.

In other words – when I say we’re going to scrap a project, I really mean we’re going to scrap it.

At each milestone in a project, we have something, a tangible something that we use to evaluate whether we should go further or stop. This something almost always has some sort of value. When planning projects with real options, one strategy to raise the value of the project is to maximize this residual value. This implies that even if a project fails, losses are limited.

Usually, when planning with Real Options, we look to prioritize tasks which remove the most risk first for the least amount of cost. We also want to arrange tasks such that residual value is maximized in failure paths. It is this second point to which really speaks to Incremental Development (by which I mean any iterative method which focuses primarily on prototypes or results rather than designs).

The main argument for formal methods is that they try to eliminate the most risk up front, in the design phase, before any real cost is laid down in code. I believe that’s a pretty hard sale nowadays given that actually getting started on a product and incrementally improving it has shown itself to be equal in reducing future risk. Some might claim incremental development is even superior at reducing future risk, but I’m not going to say that. Instead, I’m going to say it’s incremental development’s ability to maximize residual value that makes it a superior method, as well at it’s ability to inherently be a more liquid form of project management.

What is the value of a review? What is the value of a design that is not implemented? UML diagrams that don’t execute, or documentation for code that isn’t written yet? Some might argue these things do have some limited value – they can be used to build the things they design. I believe this is false. All these things are simply flags, or signals, of progress but aren’t inherently progress themselves. A well documented design is evidence that designers have been thinking about the problem at hand – but make no mistake, the designs themselves are practically worthless compared to the knowledge gained by the designers in making them. A design is valuable insofar as it has helped the designer, at least when it comes to code.

Conversely, what is the value of working code? Particularly, what is the value of modular code, which results from iterative development of subsystems? It can be quite valuable, and certainly good project planning can make it very valuable indeed. For instance, an iteration in developing a completely new game might be developing a new game engine. Perhaps after the engine is complete, we decide the game is no longer worth pursuing. But we still have the engine. This code can be reused. It has a high residual value.

Formal methods that emphasize design certainly produce value – but only insofar as it has push domain knowledge into the heads of the designers. If the project is cancelled, that work is meaningless. Designers may be furloughed or moved on to other projects where their UML designs don’t apply. But working, tested, modular code does have high residual value. This limits a lot of risk down failure paths of projects. There will be losses for sure – that’s what failure means. But losses will not be total losses. The project will, literally, be scrapped – as in, sold and valued for the scraps. It’s code may be melted down and put into something new, while the mountains of UML diagrams in a more formal program simply sit and take up space.

When you begin to introduce as many milestones as you can (decomposing your project to the finest level, to the point of 1-2 week milestones) and optimizing for residual value, you are doing iterative development. When you get really good at this sort of planning, you begin to take residual value as the only value that matters – treating every decision as ‘ok, now we’ve built X, what options are available to us?’. You move forward along a continually evolving schedule, always adding value and only risking the next step – never the whole project. As these schedules never seem nailed down, some might even call them agile.

But the fact is you’ve pushed out liquidity risk as much as you can. Your schedule may change, but actual cost and deadlines are never that volatile. You are always able to take the next step in the way that best optimizes the customer’s interest and yours. You’re never stuck with work-in-progress like formal methods.

Agile and iterative methods aren’t usually associated with planning out PERT charts with embedded decisions trees, and calculating NPV at each step. But given how automated these tools are becoming, they can no longer really be considered heavy handed. Indeed, the methods these tools allow – making real options to reduce risk, maximizing residual value, eliminating WIP, etc… are all things agile has already been practicing for years.

People see constantly changing schedules and they see risk. But the people for whom risk means the most – people who try and eliminate it any way they can to maximize profit – they’ve come up with ways to eliminate and categorize certain kinds of risk. Liquidity, volatility, and the like. They’ve come up with mitigating steps to lower cost of failure, such as residual value. These methods work to reduce and ameliorate risk. Formal methods, for all their pomp and circumstance, can – at best – only approach iterative methods in their ability to manage risk. They can add more reviews, sure, to lower liquidity risk. Many updates to formal methods do just this. But reviews are costly and are rare for a reason. They can not, however, optimize for residual value. They cannot take full advantage of Real Options. Thus, they’re inherently risky methods with little upside and should be avoided.

August 20, 2012 - Posted by | Uncategorized

No comments yet.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: