The Skeptical Methodologist

Software, Rants and Management

Where Innovation Comes From

Innovation is not a well understood concept. This is sad because our economy is driven by it. Businesses seek it out for profit. Whoever can crack the innovation puzzle drastically increases their chances of success. We know it when we see it – three guys in a garage, tinkering away. But we can’t seem to recreate it at the office.

This leads to a second source of melancholy – the very methods many enterprises use to spur innovation end up back firing. This is because innovation ends up being the very opposite of what a large organization usually seeks, and what largess provides. Organizations seek to be efficient, specialized and focused. TLDR: don’t be those things.

Breadth, not Depth

One important facet of innovation is that innovative people and teams invariably are … varied. A group of five software engineers may be only marginally more innovative than any single software engineer. But a group consisting of multiple types of engineers, as well as marketing and finance specialties is many orders of magnitude more innovative.

This occurs for two reasons – one, the obvious one, is that the team can supplement any particular individuals weaknesses with another’s strengths. Since innovation always consists of unknowns and discoveries, there are no efficiencies to be reaped by simply adding more of the same specialization to the team. Adding another electrical guy may do no good whatsoever if the team eventually decides to go after a problem that doesn’t benefit from that sort of engineering talent.

The other more subtle reason is that innovation is 95% realizing a problem and 5% realizing a solution. Many people just are not familiar with problems they can solve. This, too, results from specialization. Any particular analyst or associate could give you a list of open problems in their field. They don’t even attempt to solve these as usually such things happen in university labs. But they have no idea what the problems are in other people’s fields – some of which may be easily solved using techniques native to their own field. This opening up of the problem space to easy exploration allows a multifaceted team quick access to low hanging fruit in the innovation space – the things that seem ‘obvious’ in retrospect, but no one really goes after because it’s thought to be unsolvable or not important.


Slack not only in time, but money. Of course, most successful businesses do not optimize for slack – slack looks bad on ROI calculations as simply wasted resources. But we need to change our views of slack to something more positive.

The fact is, if you hire creative and passionate people, and give them free time, they won’t stop producing. They’ll very much continue to produce things because they enjoy it. Key example here would be the open source software movement.  Open source software represents many collective man millennia of work, all done for free. Wikipedia is another good example. When you give creative people free time, they continue to work.

Moreover, slack gives people time to relax. To open their minds and work on something where the company isn’t at stake. Ideas that formerly seemed too risky or out there suddenly become costless in terms of experimentation.

You might have heard of the term ‘cognitive surplus’. These are simply extra brain cycles not dedicated to anything due to built in slack. Brain cycles will compute something – they won’t be wasted.

So let’s summarize so far – we need broad teams given lots of slack. If I were to lock a team of brilliant and diverse people in a hotel  room and give them limitless food, drinks and any tools they required, would I get ideas out? Not necessarily. A final ingredient is in order.

Dogmas at the Door

An interesting question comes to mind when thinking about slack and the cognitive surplus it provides. During the middle ages, due to taxes and tithing, the Church had a great deal of surplus. And, as expected, we did get a great deal of innovation from that period in architecture, art, and music. But oddly these innovations seemed to miss obvious points. Moreover, what about technological or scientific innovation?

The problem then was that all the surplus was too centralized. Hence, centralized dogmas could take over and direct where and when that surplus was to be used. Arbitrary decisions and politics ended up creating roadblocks along many scientific and cultural paths, despite the surplus.

The same thing is happening in China, which struggles with innovation. Until potentially dangerous ideas have a chance to be nurtured and grown, no real innovation will happen. Guided innovation in certain sectors may continue, but no revolutions will spring up.

Enterprises tend to jealously guard what little slack they are willing to give away. Perhaps an hour here or hour there, but damnit, we better get our moneys worth and see some results! Ironically, they are willing to pour thousands of man hours and millions of dollars into inefficient processes and never complain of a lack of results, yet the minute someone asks for slack, they suddenly care about ROI.

Alternatively, enterprises attempt to ‘guide’ innovation, setting up certain committees, shutting down projects early to ensure no precious slack is wasted on bad ideas. The problem is, we don’t know what’s a bad idea at first. More importantly though, from an ROI perspective, these mechanisms to ensure no slack is wasted often end up costing far more than the wasted slack they save in terms of opportunities lost by innovation not taking root in a bureaucracy.

Innovation is an artificial Ecosystem

We are attempting to grow a small ecosystem, to take advantage of mutation and natural selection and produce an entirely new species of product, process or other work. Ecosystems need diversity, they need spare resources to allow innovation to thrive unabated and not allow efficiency concerns to crush ideas too early, and they need protection from outside stewards to ensure special interests and politics don’t clear cut certain ‘worthless’ endeavors.

Like growing any thing else, the best thing for businesses worried about the risks of innovation to do is to embrace it whole heartedly – don’t just innovate once, innovate dozens of times. Don’t simply put one cross functional team in a hotel room with limitless supplies and a promise to not interfere – lock many teams in various rooms. The fact is even with the above three environmental conditions in place, we can be assured innovation will take hold but we can not be assured it’s fruits will be useful. Don’t be quick to attempt to ‘prune’ innovation: leave your dogmas at the door and let it grow. But, to ensure a good crop, batch process. If you have even one stellar idea, it will fund many dozens of teams with bad ideas.




March 7, 2013 Posted by | Uncategorized | 1 Comment