Today’s leaders are expected to deal with ambiguous situations. We’ve all heard that, right? In a way, it’s trivially obvious but sounds like something you should jot down from your 10 day MBA book. It’s not like a CEO or Entrepreneur ever has their job responsibilities really nailed down for them. They are expected to define the position. Moreover, what’s the bit about ‘today’? Was it ever the case that leadership wasn’t ambiguous? Since when did we ever have things really nailed down for us?
This tautological line, however, is being used increasingly not so much as unhelpful advice to actual leaders, but instead as a cliche response from middle managers explaining away their inadequacy. Think about it – since when does a subordinate ever respond to their lead with “I understand the work I’ve done for you is ambiguous in value, but today’s leaders are expected to deal with ambiguous situations.” That is career suicide. Instead, the statement is more often used in the reverse – when a leader fails his subordinate, he might punt to the ambiguity of the situation. Indeed, some leaders may bath themselves in ambiguity, using ambiguous milestones such that they can claim progress where none has been made, or ambiguous job requirements so they can criticize a subordinate for not living up to ‘expectations’ when they’re having a bad day.
I claim we should be taking a different tact with this statement. Indeed, when a leader punts to a subordinate justifying herself with ambiguity, shouldn’t we ask the question – who’s responsibility is it to deal with that ambiguity?
Why, today’s leaders, of course.
When a leader cloaks their milestones, expectations or other requirements in ambiguity, they are literally not doing their jobs. They are not dealing with ambiguity, and instead having their subordinates deal with ambiguity. They have left their subordinates holding the bag when it comes to who to blame when ambiguous situations get out of hand, even though it’s their job to resolve ambiguity insofar as possible. This often seems like a political move – ambiguity’s metaphorical cloak becomes a bit more literal when it comes to masking intentions and actions to potential political rivals. Some smoke filled room horse trading is always to be expected – we can’t be naive. But allowing this ambiguity to leak through to your team is failing to do your job.
Misaligned strategies, the 8 different bosses situation, or even allowing the innate ambiguity of the actual markets to leak through are all examples of the sources of ambiguity. But they are all examples of horribly failed organizations or leadership, and it’d be unjust to instead claim that they are just what ‘we have to deal with today’.
Good leaders should seek out ambiguity and banish it insofar as possible. Are you not sure whether we need to satisfy customer A or B? Do as best analysis as you can in the due amount of time and just go with it. Don’t punt to someone else. Make a fucking decision, for Christ’s sake. It’s what you are paid to do. Good leaders unite their team behind a single vision even though the real world is very ambiguous. This is how you show up on game day and dominate because every other team still doesn’t even have a plan. Good leaders inspire their teams by removing the uncomfortable ambiguity of business – “I know both customer A and B are important. But this team is going to focus on A, you’re bonus will be tied to A, and your promotion will be tied to A.” This is how you motivate people, by giving them clear requirements even though you may yourself have not received any. If you give a subordinate any inkling that they should somehow do both customer A and B even though there’s only time for one, you’re going to get half assed work and a lot of hours worked for nothing. You will have failed to do your job to properly direct your subordinates, and you may be able to horse trade awhile long clouded in the mystique of ambiguity, but you will forever be an imposter and eventually, they will find out.
“Hey there, did you know everyone else is lying to you about diet? It’s all due to their corrupt corporate interests! I have no agenda, and will explain to you why everyone else is lying. Also, avoid processed food, as everyone knows that’s bad for you.”
Before we get started, there are a lot of great gems in the article. It does go into why diet is so confusing. But unfortunately, it breaks its own rule. It fails to cite evidence for its claims, and even misuses the word ‘natural’ in the middle of an article dismissing the word as meaningless.
Let’s dive into a few assumptions here. First, they are right in stating that there’s a lot of mixed messages out in diet. I’d come away saying that we actually know little beyond people who eat a lot of fruits and vegetables seem to be healthier. But we don’t necessarily even know why since there’s so many confounding variables there.
But this ‘corporate interest’ bullshit that should be the first sign that it’s not one person selling you truth and another falsehood, its just two bullshit vendors providing different varieties. Corporations are not evil. Repeat that with me. Corporations are almost completely amoral, in fact. They, for the most part, exist to maximize profits assuming human management with bounded rationality is making the day to day decisions. What does that mean? Does it mean Corporations are trying to get you fat, to die early, to lie to you? No, it means that established corporations biggest motivator is to protect the status quo. The status quo makes them money.
Let’s think about that for a bit. Does that mean that General Mills might try to convince you Fruity Pebbles is completely healthy for you? Yes, it does. They currently sell Fruity Pebbles, its in their interest to try and convince you to keep buying it. But what if they sold Grandpa John’s brand dried vegetable slaw (now with more wholesome!)? Well then they’d try to convince you that vegetables are healthy for you too! They aren’t trying to sell you packaged shit, they just happen to already sell packaged shit, so they want you to keep buying.
This same logic works for so-called ‘natural’ foods, which appears to just be foods closer to their original state. Organic farmers can charge nice hefty margins because they’ve convinced you that organic is healthier, even though there’s little evidence for that. Basically, everyone who sells something is motivated to keep you buying. It’s really not that hard to figure out, and it doesn’t make them evil. If science finds the best diet, just wait twenty years and these same evil corporate interests will have built up supply chains, warehouses, and marketing campaigns to make sure you buy that best diet, whatever it is. But until then, expect them to come kicking and screaming. That’s just their nature.
The second assumption they cite without evidence is this ‘processed food is bad for you’ clap trap. First of all, what the hell is processed food? Apparently Chetoes. They really seem to hate Chetoes. But beer, cheese, bread, pickles, deli meats, milk and yogurt are all processed too. Fermentation, cooking, pickling, and the like. Those are all processes. Canned vegetables, frozen berries, these are processed too. Are all these things bad for you? Not at all. You can tell the author’s motivations when they compare so called ‘processing’ to cooking at home, which is just another process. Apparently, processing is okay if you do it yourself. Do you see the motivations and assumptions too?
I’ll spell it out: eat food rich people eat, and avoid food poor people eat. Cooking at home is a luxury someone who isn’t working two jobs can do, it must be healthier for you. This sort of unprocessed push is just more upper class privilege asking why the poor colored masses aren’t living as long. Clue: it’s because they are poor, not because they aren’t eating what you eat. Just like the obesity epidemic was more or less invented so we can continue to hate blacks and latinos, the rampage against processed food is another way we can all feel better about making fun of that lower class family picking up some more hamburgers from McDonalds and blame them for their own misfortune.
Does this mean Fruity Pebbles is healthy? I have no idea. Fortified cereals have been shown to lower a nationwide deficiency of folic acid, so they’ve got that going for them. But the more important thing to point out is that there are many processed foods that are bonafied healthy – tomato paste being my favorite example as having a more bioavailable form of lycopene than whole tomatoes. Using simple rules like “unprocessed” is a great way to eliminate more than half the population from your recommendation since they can’t afford to eat that way anyway and eliminate a whole lot of good pathways to nutrition.
I’d say this unprocessed fad is no better than all the others, and it really gets to me that someone would go through such trouble to point out why everyone else is lying to you while blatantly doing the same to you themselves. It’s the kind of moral superiority that upper class, out of touch, privilege gets them, I suppose.
This “death of theory” has been argued before, and periodically comes up from various empiricist leaning circles arguing that all you really need is data. The problem is that, as Kant showed when he reconciled rationalism with empricism, theory and data are intractably linked. Contrary to what some die hard empiricists might argue, we do not believe “1 + 1 = 2″ simply because we’ve seen it enough. We invent notions and concepts to think symbolically, and in that realm, we set up definitions which we use to prove “1 + 1 = 2″ inside the system we’re discussing. We then tie our theories to fact on the ground to make predictions.
Indeed, I’d say the reason AI originally failed in the 1980’s is due to its over-reliance on theory to the diminishment of actual data. Purely symbolic systems can only go so far, and only go so fast, especially in discerning very noisy rules like those of spoken language. And now it appears we’re swinging too far the other way – we’ll never have need of symbolic or formal reasoning systems again, it may be argued. We’ll just throw enough data at it. This obviously rubs many of those actually in the machine learning field the wrong way, as its precisely those formal methods that allow them to code (type checking, compilers) and prove (mathematics, theorem provers) their models work within certain parameters.
The argument Cringely seems to be implying is that theory has just been a crutch for we mere humans, and that if we only had enough data, we wouldn’t need it. However, theory does many things, and even in the era of big data, it will continue to do these things:
1. It is our only path to actual truth.
One can prove theorems based on assumptions. All empiricism ultimately falls to Hume’s problem of induction.
2. Theory can succinctly describe in a single equation many terrabytes of data.
Big data means lots of data – but the above fact is still true no matter how cheap data gets. We can always do more when data is married with theory, each unit of processing power will always be more useful when mixing the two rather than simply trying to neural network the whole problem.
3. Theory can communicate ideas.
Linked to the above in terms of compression of data into a single equation, our symbolic language also makes it easy to communicate ideas to one another. This will be true of big data as well – moving around simple equations will always be cheaper and faster than moving around the entirety of data sets on which they are based.
4. Theory can make predictions out of sample.
Big data’s predictions out of sample are always, at best, guesses. Educated guesses, but guesses. And they are, in turn, guesses based on a few fundamental theoretical assumptions of attempting to minimize error. If we ever run into inputs that are not in our data set, or alternatively, if we want to backsolve for our inputs given required outputs, this is always easier when we have theories to supplement our data. Regression analysis, for instance, allows for a lot more theoretical interpretation of results than a cackle of random forests. When we have regression coefficients, we can make many more predictions about our data set using far fewer facts.
Big data is already changing things, and many of Cringely’s predictions are true. But to say that Big data is going to ‘automate science’ away is a large misunderstanding of what the theoretical side of science does, and how theory serves us.
I was recently intrigued by the this video. It walks you through some hand wavy steps on how we can actually assign a number, in this case, -1/12th, to the series 1 + 2 + 3 + 4…. It makes a few appeals to intuition on why this may be correct. It got me really interested in how it worked, sending me off to Wikipedia to learn about the Riemann Zeta function, extensions to calculus, and so on. I think it is a great example of making mathematics fascinating to larger numbers of people, a great thing in our world where so many mathematics educators seem to think it is their duty to pass as few kids as possible.
Little did I know about the shit storm of engineering undergrads who apparently knew more than Euler when it came to this form of analysis. With fingers waving wildly, they tore the video apart, explaining just how wrong it was, some going so far as to talk about the damage it could do to we mere unwashed masses.
The lesson they’d rather teach? Math is boring, hard to understand, and we like it that way. STAY OUT.
I could go into a deconstruction of these types of personalities – how they identify with their ability to excel at increasingly difficult arithmatic with genuine mathematical ability throughout high school. How they most likely chose to pursue an engineering degree (no small feat, mind you) rather than a pure mathematical degree plan because of the unease they met finding out that math is actually quite different than mindlessly following rules. This video is just a reminder that they didn’t really ‘get’ how math could be equally invention, art and science.
When geniuses like Euler invent new rules and terminology to solve the problems that vex them, that shows just how freeing mathematics can be. But its precisely that freedom that scares a lot of arithmaticians shitless because it violates their precious black and white rules.
Truly, these folks are little more than crows pecking at Prometheus’ abdomen – the punishment any good educator receives when they have sufficiently ‘dumbed things down’ enough to convert the unengaged into the engaged from those who were quite happy being the few who understood before. In a way, you can’t blame them – their knowledge is being inflated away, as more and more people grasp the basics. What they saw as precious and theirs is shared with everyone else – not diminished in any way, but certainly no longer an attribute that makes them unique.
So they can but caw and cackle as they point at technicalities, sad that no matter how much liver they tear away, the fire has been lit.
I’ve been playing around with a thought. What if the metaphor of software development as engineering isn’t wrong insofar as software development could never be made as rigorous as engineering, but rather, perhaps its wrong because of the assumptions loaded with engineering? After all, when we say engineering, we generally are talking about applied science and technology of physics.
Perhaps its the case that software developers are engineers, but of a different sort. Perhaps they are practitioners of applied science and technology of metaphysics? This definition itself is a little hairy, since one could argue that the sheer concepts of science and technology are metaphysical ones. Perhaps it would be better said if we simply said software developers were applied meta-physicists?
After all, the debates over immutability of data versus mutation harken all the way back to to the presocratics and the famous quotation attributed to Heraclitus that one never steps into the same river twice. Arguments over substance sound eerily familiar to that over object oriented programming. Finally, the introduction of process philosophy by Whitehead can be tied very easily with the ideas embodied by event driven programming.
The tie of software design to metaphysics seems obvious upon reflection. It is our job to listen to loose descriptions of our clients worlds and try and encode them into a rigorous system of objects, relationships, processes and data – so rigorous, in fact, a computer can follow through with them. One of the hardest problems in computer science, it is said, is naming things, which is very much a hard problem in philosophy. The corollary to metaphysics also explains one reason why software design is so amorphous and hard to nail down, while software itself is ironically very rigorous once written. Meta-physicists do not yet have the tools of science, mathematics or empiricism. It is in fact the very same philosophers who invent those tools. Much like C style object orientation, they have to use tools that their language doesn’t necessarily give them, thus rely on convention over configuration, to a degree.
Software engineering as a psychological pursuit surely will come under the knife of science in time – we can measure whether or not teams that pair program, test-drive design and object orientate their work are more or less productive than others. But the source of those ideas and processes itself will always, it seems, live in the realm of metaphysics which can only loosely be attacked through rational debate (notwithstanding the assumptions one must make to even get to the point where you’re allowed rationality).