Convergent and Divergent Thinking
I read this article in the Times today, and began thinking of how apt a metaphor it was for software.
Not to put words in the author’s mouth, but to sum up, human thinking seems to fall into two discrete camps, what they call divergent and convergent thinking.
Convergent thinking is basically attentioned, focused thinking, where the mind is applies reduction rules to a problem. Doing a math problem is a good example of convergent thinking, the brain is applying a mechanized system to some problem, where the answer is basically just a number of steps from the problem, and we know to go to B from A, and C from B, and so on, to the answer, it’s just a matter of doing the work.
Divergent thinking is what we do when we don’t know the answer, when we don’t know the next step, when the problem is completely open ended. Another common problem that you solve using divergent thinking would be a word puzzle like Wheel Of Fortune. How do you solve those puzzles? Do you methodically plug in every letter of the alphabet into the puzzle until one fits? Obviously, even a trivial puzzle could have you calculating for days using that method as it explodes in complexity. Instead, you stare at it, relax a little, and the answer occurs to you. It’s a eureka moment.
We’ve all had stories of spending all day working tirelessly, and fruitlessly, on a project, only to have the answer occur to us right as we drift off to sleep.
As software developers, we’ve basically come to believe the false notion that what we do is convergent thinking. We’re lead to believe that, because ultimately what we’re working on is algorithms and numbers and bits and bytes, we need to use the same strategies we’ve always used for those technical problems. Indeed, the vain hope with different software methodologies is that the ‘one-true-process’ will emerge and allow anyone to produce what they want given some specified requirements.
We don’t just need experience to tell us the folly of this expedition, now we can look into human psychology.
I would posit that, if anything, the arts, like sculpture, painting and drawing are fundamentally divergent activities. Certainly an artist must have a measure of ‘technical’ skill, she ought to be able to draw well. But just drawing well does not a work of art make – she needs to also have a flash of insight, to be inspired. Otherwise what she will produce will be drab and lack quality. It will be unmotivated. This is not specifically tied to divergent thinking, but quality is something I’d like to go on more about, but at a later time.
Anyways, to produce a work of art, one must have the technical skill to do so, but also the flash of insight to know what to produce in the first place.
Software, while having some ‘engineering’ aspects to it, is fundamentally a creative pursuit. This is not any sort of wishy-washy statement, but a simple conclusion from logical deduction. Yes, it’s ironic that we’re using logic to show that straight out logic won’t suffice in the end, but that’s beside the point.
The fundamental problem solving technique we developers use is abstraction. Upon finding a solution to a problem, we identify what can vary, abstract out the solution, and reuse it. This is the basis for libraries and all other code ‘reuse’. The point being, if there is ever an ‘algorithm’, or actual pattern of work that is done, some sort of mechanical process, it is within our power to abstract that pattern out and reuse it as a primitive. Despite what the GoF will tell you, patterns CAN be generalized and pushed into languages and libraries. Hence, in the ideal development environment, you won’t be faced with reimplementing some already known technique or solution, but instead will have those solutions already at hand. What you will be doing is using those already known solutions in a completely unique way that’s never been tried exactly like this before. You now have an open ended problem, for the sheer sake that all the problems that convergent thinking could do have already been solved for you by people before you. And once your solution is complete, it too will fall into the hall of convergent solutions, and be reusable. Hence, the only real work in idealized software is creative work of taking the solutions we already have at hand and putting them together in a completely new and unique way.
That ‘proof’ required a lot of hand waving, but I think you get the point. Common sayings in software like DRY and KISS exist as words of wisdom because we should not be doing mechanical work as software developers – that’s what the computer is for. We should focus on creative work, and let the computer do the rote stuff. Most problems in software, I suspect, come from either the false belief that creative work can be shifted into mechanical work (and our endless supply of methodologies is evidence of that) or that mechanical work should be redone as creative work (and reinventing the wheel, or not-invented-here syndromes are evidence of that).
As the article at the top implies, creative work is best done divergently. And creative work is best solved by not trying to solve it at all. Know the problem, understand it in detail and research it. Investigate other attempts at solutions and have discussions, but never think you can just sit down and mechanically come up with some sort of solution. Inevitably, the ‘ideal’ solution that ought to be a one-liner will blow up into thousands of lines of case statements as you learn more and more about the problem. As we know, the best predictor of number of bugs is number of lines of code. Less is definitely more.
Back to my post on new hires being driven by company culture to simply throw as many hours at a problem as possible, we can now see how mere hours won’t amount to much. Psychology has shown that while our ability to solve problems requiring convergent thinking decreases as fatigue increases, it does so at a much slower pace than our ability to solve divergent problems. Sleep deprivation saps creativity, gumption and quality in one’s work.
In fact, any sort of ‘drive’ to find solutions mechanically will cause work to suffer. Many times, when trying to figure out how to design some sort of extensible framework, the BEST thing we can do for ourselves is to stop working on it. Sure, you can do the ‘moral’ breaks like taking walks, or a corporate-approved power nap. But ‘immoral’ things would work too – surfing the web, reading blogs, chatting, playing video games. All of these things improve morale and give your divergent mind a chance to churn away at the real solution.
I know it’s against our protestant culture and mindset to ever believe there’s virtue in laziness, but I maintain that all great insights occur when we’re not even looking for them. So to increase your chances of finding those insights, you best stop looking for them as soon as possible and let them creep up on you. You may end up ‘working’ less, but getting a lot more done.
2 Comments »
I try my best to seperate the diamonds from the rough in the software world. I’m convinced no one really knows how to produce software: including me. Be skeptical.
- March 2013
- February 2013
- December 2012
- November 2012
- October 2012
- August 2012
- July 2012
- June 2011
- May 2011
- March 2011
- December 2010
- August 2010
- May 2010
- December 2009
- November 2009
- October 2009
- September 2009
- August 2009
- July 2009
- May 2009
- April 2009
- March 2009
- January 2009
- December 2008
- November 2008
- October 2008
- September 2008
- August 2008
- July 2008
- June 2008
- April 2008
Site infoThe Skeptical Methodologist
The Andreas04 Theme.