The Skeptical Methodologist

Software, Rants and Management

Pair Programming: Putting a Spy in Every Cube

Apparently, Facebook and Square can’t get enough pair programming. While the rest of us wait out the gloating of the all-grown-up start-up flavor-of-the-year rediscovering techniques known for decades by other firms and practitioners, someone in the counter-counter-culture has to speak up and say the obvious.

No, Pair Programming is not that effective.

First, let’s place Pair Programming in its philosophy and history. Its roots are in extreme programming, whereas the general approach is to take any ‘good’ thing and do it a lot more – take it to the ‘extreme’. So, in the case of unit testing, extreme programming advocates would move to test driven development. Pair programming is the end result of taking peer review to the extreme. “If some peer reviews are good”, so the thinking goes, “maybe we should peer review as much as we can? Maybe we should peer review constantly.

I bring that up to make the point that we should not be comparing Pair Programming with working alone. That’s not what it’s replacing. It’s replacing more traditional forms of peer review. So when you see the benefits of Pair Programming being extolled from on high, ask the question – Are they going on about the benefits of Pair Programming, or the benefits of peer review?

Most of the ‘bug-catching’ arguments in favor of Pair Programming are actually arguments in favor of peer review. Simply having someone else look over your work is not a new idea. Nor is it an ineffective one, as a few studies have pointed to peer review being the most effective means of preventing defects, per dollar spent.

So let’s only focus on the benefits of Pair Programming as compared with traditional means of peer review – in fact, let’s compare Pair Programming with its next closest relative in terms of peer review style. We’ll compare it to a process that advocates light weight peer reviews over small chunks of code done very often.  Moreover, I also want to make it clear I’m focusing on Pair Programming as your primary means of development. In most cases, what we might call Pair Programming is a very common event – one developer simply helps the other out on some sort of problem, or they both work on a particularly tricky part of a design together. This, for some reason, is just considered normal work, not Pair Programming. Pair Programming, it seems, is when management decrees you will work with another person even when you don’t need to.

The Benefits

The argument then becomes that Pair Programming finds defects slightly earlier than normal peer review would – instead of waiting a day or two, or perhaps hours, before you can submit your work for peer review, your pair would point out flaws and you two would fix them. The argument would continue by relying on the much repeated philosophy that a defect caught earlier is cheaper to fix. If you can catch it in a few minutes, isn’t that cheaper than catching it in a few hours?

That may be true, but it is nowhere near the cost differential as catching a defect now to a few months from now. After a few hours of work, I am still very much in the context of the original defect, so it does not take nearly as much ‘spin up’ time to effectively quash the defect.

Does Pair Programming in this way save time? Yes. But not nearly as much as implementing peer reviews do in the first place. This implies that Pair Programming may have a negative balance in a cost benefit analysis. The benefits are there, but they are small. How about the costs?

The Costs

The costs are both implicit and explicit. First, you are tying up two developers. Even in the most generous terms, I’ve heard pair programming might increase productivity by around 150% compared to a single developer. That means, on average, you’re down half a person.

And even those generous terms may be misleading. Let’s say I have two developers, one who’s not too keen on actually programming and is probably in the wrong field (i.e., 80%+ of the developer population), and a superstar. The dullard may code at around 50% productivity by whatever measure. When I compare him to a superstar programmer, that superstar programmer may code at around 150% productivity by the same measure. Seeing a superstar being about 3x more efficient than a dullard is not uncommon. Their arithmetical average productivity is 100% – the average developer.

Now I pair the two, and their joint productivity springs forward to 150%! They’re more productive than the average developer, so one assumes something in the method is working. But let’s casually stroll by their cube and see the magic in action. Lo and behold, the superstar is in the driver’s seat, doing mostly what she was doing before, and the dullard simply looks on and remarks about the occasional missing semi-colon. No one’s more efficient – we’ve simply improved our numbers by taking the keyboard away from the dullards.

Notice one extolled benefit I’m putting squarely in the implicit cost column – wasted time. Some might argue Pair Programming reduces time wastage, on the internetz for example, or using one’s smart phone. This, unfortunately, assumes time wasted is something to minimize. This, in turn, is unfortunately something newly Peter Principal’ed managers in their early 30’s naively believe.

Developers – the good ones – enjoy development. They love what they do. So when they aren’t doing it, one has to assume that they aren’t doing it for a good reason. In many cases, people need the downtime to think about things, or just relax a bit after a few hours in the zone. Pair programming enforces through a cultural mechanism constant work. Even if your devs enjoy it, they’re going to be suffering due to lack of down time.

Peopleware, which ought to be canonized in our field, makes this point very clear in comparing the standard cubicle to the office of old. Offices increased productivity by such a large margin they paid for themselves many times over. Offices, it is to be remembered, reduce interactions, they do not encourage them.

Why?

Given the large costs and relatively small benefits, why is Pair Programming so popular? Well, follow the money – or at least, follow who it’s popular with. Amongst the developer community, you’ll see advocates on both sides. Some really love it, others hate it. The consensus seems to say it’s a useful technique in some situations.

But among managers, especially uneducated, newly minted managers who really don’t seem to understand there’s a whole field and literature devoted to the subject of managing organizations, love Pair Programming. And what they really love isn’t the method itself – sure, they might chime in about reduced defect rates and other supposed benefits. But what they really love about it is how close it is to the ultimate tool of uneducated managers – the bull pen.

Constant supervision

For some crazy ass reason, there’s a huge school of thought amongst managers (again, usually the type that don’t actually read any of the prior art in management and make it up as they go) that the secret to good management is supervising everything. You can’t trust your employees further than you can throw them, so the lower we can make these cubical walls, the more I can overhear everything, and the more time I have to walk around and status the hell out of you, well… that’s bound to increase productivity right? The fundamental assumption of the uneducated manager is all increases in productivity come from reductions in laziness.

Pair Programming cuts down the amount of work a manager has to do to keep an eye on you, to make sure that you’re doing whatever it is programming is (it’s mostly typing, right?). Pair Programming puts a spy in every cubicle. It’s no wonder overall productivity suffers so much. Morale plummets.

If not now, then when?

Pair Programming as practiced as an edict doesn’t seem to justify itself. Its marginal increases over standard iterative peer review are miniscule compared to the costs to culture, morale and productivity. It’s generally put in place as a means to supervise, not as a means to innovate. That being said, there are many cases where Pair Programming, in a looser sense, is very effective.

Particularly Hard Problems

There are times when something is just going to be really tricky to pull off, and you need help. You need constant conversation, constant white boarding, and constant prototyping with someone who can play devil’s advocate. Luckily, for us, we already ‘pair program’ in these cases.

Mentoring and Training

One place where Pair Programming could be used more, though, is in mentoring and training. Pairing a junior dev with a more senior and experienced dev is an incredibly effective and efficient way to get the junior both up to speed on the code base, as well as the idioms and practices used around the shop. Generally this will be a more hierarchical pair – it is clear who the driver will be, and who’s mostly going to just sit back and learn for the first couple of days. This limits some of the social strife pairing might cause when there’s disagreement, limiting the usual downside.

Cross Functional Pairs

A pairing I’d find very effective but don’t hear talked about too often is pairing developers with people from outside of development. This would go a long ways to prevent stove piping, provide for a common culture, and increase the bandwidth between these roles. Pair a developer with a UI designer, or a developer with a tester, or a developer with a business analyst. All of these would allow each of these specialists to learn a lot more about the other’s work habits and keep information flowing between them. I’d encourage this sort of pairing as well as job rotation amongst these roles to completely keep stove piping at bay.

Pair at your own risk

Pairing has a potential upside. This can be emphasized in specific situations when the pair has a lot to learn from each other, or when two heads really are better than one. But it also has costs, costs that get much worse when pairing becomes the default rather than just another tool in the chest.

Like all of our techniques, we need to look at when Pairing is warranted. There’s no rationality in saying ‘never pair’ any more than there is in saying ‘always pair’. Pair when it makes sense, when it saves time, when it builds teams. Otherwise, don’t feel so guilty about putting up an away message and keeping your office door shut.

Advertisements

August 28, 2012 Posted by | Uncategorized | Leave a comment

Risk and Incremental Development

In an earlier post, I talked about D’Souza’s idea of Real Options in project planning, explaining what an option was and applying it quite literally to a software project between a vendor and customer. This time I want to talk about how the idea of options and the actual planning steps that must be taken to incorporate them, as well as the financial concepts of liquidity and volatility imply that Incremental Development is inherently less risky than so-called formal methods.

Let’s go back to real options. In the most basic interpretation, real options in a project are decisions you build into your schedule to hedge risks. They can be as basic as go / no-go at predetermined moments, or enumerate many different paths forward from any particular point in the future. Additionally, these decisions can be structured to behave like put and call options from finance, limiting risk on any particular project moreso than hands-on control which is subject to behavioral biases like sunk costs.

If you have started to incorporate real options into project planning you may notice something – options more or less can only be incorporated at milestones. We must first learn something before we can make any decisions about it. Like control theory, there must be some sort of input before there can be feedback.

Recall that I argued in my earlier post a trader simply promising to himself he’d sell at a certain price is still more risky than options, and one of the reasons for this risk was due to the fact that prices in stocks can suddenly move very quickly, especially during overnight periods (called ‘gaps’). A trader might watch a ticker all day, getting one update a second, and then see two prices differ wildly because one was the closing price yesterday and the other the opening price today. Value changes even though much less trading is taking place.

A quickly moving value is called volatile, and one measure of volatility is basically the statistical variance in price movements over time. Volatility changes in time as well, leading to periods of relative calm as well as stormy weather. Projects can be volatile too – especially at the beginning. As estimates and schedules continue to change and churn, the value and cost of the project changes drastically from day to day.

The stock itself is not actually more volatile – if we only look at second to second trading, it looks nothing like close to open gaps. What we’re actually witnessing is illiquidity. A liquid asset is one that can be bought and sold easily. During trading hours, most stocks are incredibly liquid. But after trading hours liquidity dries up. There’s private over the counter markets, but no public markets to trade on. In control theory, liquidity can be thought of in terms of how often input comes in, and how often feedback is allowed. During times of illiquidity, input is sparse and opportunity for feedback is rare.

In an illiquid market, volatility appears to rise simply because we more or less aren’t allowed to observe the trades in between. For instance, in the housing market, which is relatively illiquid right now, it may seem volatile because my house price can drop by $30,000 in a day or two. This is because, since houses aren’t moving, the actual time between measurements – i.e., the selling of comparable houses – spreads out.

If we want to reduce our exposure to this liquidity risk, we want to stay in liquid assets. We want to change the rules of the market such that they’re open all night, so we are no longer exposed to these large price gaps.

Additionally, when planning projects, milestones are input and decisions, or real options, are feedback. The time between milestones exposes the project to illiquidity risk. If you want to get out between two milestones, you’re out of luck (or at least, its much harder) than if you simply exercise an option built into the schedule from the beginning at the appropriate time. How can we reduce this risk? The same way we might reduce the same type of risk in finance – we add liquidity. In the case of project management, we have to add more opportunities for feedback and input.

This more or less means more milestones. A lot more. As many as you can think of, with options spread throughout.

There’s another major issue in project planning, though, laid bare through the use of Real Options. As described in his paper, D’Souza uses decision trees and options to better estimate Net Present Value, or the value of any particular project. When we add in options that allow us to walk out, we don’t necessarily have to take the project as a total loss, indeed, we can lower the cost of failure paths if we arrange our project to best take advantage of residual value.

In other words – when I say we’re going to scrap a project, I really mean we’re going to scrap it.

At each milestone in a project, we have something, a tangible something that we use to evaluate whether we should go further or stop. This something almost always has some sort of value. When planning projects with real options, one strategy to raise the value of the project is to maximize this residual value. This implies that even if a project fails, losses are limited.

Usually, when planning with Real Options, we look to prioritize tasks which remove the most risk first for the least amount of cost. We also want to arrange tasks such that residual value is maximized in failure paths. It is this second point to which really speaks to Incremental Development (by which I mean any iterative method which focuses primarily on prototypes or results rather than designs).

The main argument for formal methods is that they try to eliminate the most risk up front, in the design phase, before any real cost is laid down in code. I believe that’s a pretty hard sale nowadays given that actually getting started on a product and incrementally improving it has shown itself to be equal in reducing future risk. Some might claim incremental development is even superior at reducing future risk, but I’m not going to say that. Instead, I’m going to say it’s incremental development’s ability to maximize residual value that makes it a superior method, as well at it’s ability to inherently be a more liquid form of project management.

What is the value of a review? What is the value of a design that is not implemented? UML diagrams that don’t execute, or documentation for code that isn’t written yet? Some might argue these things do have some limited value – they can be used to build the things they design. I believe this is false. All these things are simply flags, or signals, of progress but aren’t inherently progress themselves. A well documented design is evidence that designers have been thinking about the problem at hand – but make no mistake, the designs themselves are practically worthless compared to the knowledge gained by the designers in making them. A design is valuable insofar as it has helped the designer, at least when it comes to code.

Conversely, what is the value of working code? Particularly, what is the value of modular code, which results from iterative development of subsystems? It can be quite valuable, and certainly good project planning can make it very valuable indeed. For instance, an iteration in developing a completely new game might be developing a new game engine. Perhaps after the engine is complete, we decide the game is no longer worth pursuing. But we still have the engine. This code can be reused. It has a high residual value.

Formal methods that emphasize design certainly produce value – but only insofar as it has push domain knowledge into the heads of the designers. If the project is cancelled, that work is meaningless. Designers may be furloughed or moved on to other projects where their UML designs don’t apply. But working, tested, modular code does have high residual value. This limits a lot of risk down failure paths of projects. There will be losses for sure – that’s what failure means. But losses will not be total losses. The project will, literally, be scrapped – as in, sold and valued for the scraps. It’s code may be melted down and put into something new, while the mountains of UML diagrams in a more formal program simply sit and take up space.

When you begin to introduce as many milestones as you can (decomposing your project to the finest level, to the point of 1-2 week milestones) and optimizing for residual value, you are doing iterative development. When you get really good at this sort of planning, you begin to take residual value as the only value that matters – treating every decision as ‘ok, now we’ve built X, what options are available to us?’. You move forward along a continually evolving schedule, always adding value and only risking the next step – never the whole project. As these schedules never seem nailed down, some might even call them agile.

But the fact is you’ve pushed out liquidity risk as much as you can. Your schedule may change, but actual cost and deadlines are never that volatile. You are always able to take the next step in the way that best optimizes the customer’s interest and yours. You’re never stuck with work-in-progress like formal methods.

Agile and iterative methods aren’t usually associated with planning out PERT charts with embedded decisions trees, and calculating NPV at each step. But given how automated these tools are becoming, they can no longer really be considered heavy handed. Indeed, the methods these tools allow – making real options to reduce risk, maximizing residual value, eliminating WIP, etc… are all things agile has already been practicing for years.

People see constantly changing schedules and they see risk. But the people for whom risk means the most – people who try and eliminate it any way they can to maximize profit – they’ve come up with ways to eliminate and categorize certain kinds of risk. Liquidity, volatility, and the like. They’ve come up with mitigating steps to lower cost of failure, such as residual value. These methods work to reduce and ameliorate risk. Formal methods, for all their pomp and circumstance, can – at best – only approach iterative methods in their ability to manage risk. They can add more reviews, sure, to lower liquidity risk. Many updates to formal methods do just this. But reviews are costly and are rare for a reason. They can not, however, optimize for residual value. They cannot take full advantage of Real Options. Thus, they’re inherently risky methods with little upside and should be avoided.

August 20, 2012 Posted by | Uncategorized | Leave a comment