The Skeptical Methodologist

Software, Rants and Management

Protected: Project Buffett Lives!

This content is password protected. To view it please enter your password below:

June 28, 2011 Posted by | Uncategorized | | Enter your password to view comments.

Testing the Untestable

TDD is a great way to develop certain types of code, but you frequently run into situation where you either run into the question of “How do I test that?” or more appropriately, “Is it worth testing that?”

We need to remember that testing is just one tool we have towards a particular goal – verifying and validating software. In many cases, other tools and techniques that do the same thing can very much supplement or even replace testing. The best example of a formal system we all know and love is the type system. In a statically typed language, I don’t have to test for type errors. They are formally proven out of the code (unless you’re the tool who passes void* everywhere). That cuts down on the amount of testing I have to do for V&V, as well as supplements testing as a design tool. While TDD gives us clean, modular designs that are very decoupled from each other, the type system gives us help with design-by-contract and cleaner mechanisms of dispatch. They both have their strengths and weaknesses.

Does it work the other way? In other words, I’ve given an example of where we have another tool that limits what we have to test. But can we find a mapping from the hard-to-test to a tool? Not always, but in many cases, yes. In particular, I want to talk about the testing of GUI’s.

First, following a MVC design pattern helps dramatically with testability. Fact is, testing the controller and model leverages far more traditional techniques. The view has always been the problem – how do we test GUI code? Certainly we can just stick to our guns on TDD, mocking out nearly all the library calls we make to our GUI framework, possibly writing drivers that control mouse and keyboard input to pick and poke the interface. These sometimes have their place, especially if you can leverage pre-existing resources. Remember, though, testing is done primarily for V&V. Is there another way to validate and verify our GUI software?

Enter XAML. Or QML, or HTML for all that matters. Each of these fits the same slot – it is a language for specifying the layout of GUI’s. The most important aspect of each of these is that none of them are Turing complete. This drastically limits the amount of damage we can do to ourselves, and also dramatically expands what we can prove about our code automatically. Specifying a layout in one of these markup languages rather than in a universal Turing complete language like C++ is both easier and far less bug ridden. It’s simply impossible to make certain mistakes. Knowing that an XAML document is well-formed and parseable gives much greater guarantees than knowing a program is well-formed and compilable. And we know how important keeping a program in a compilable state is!

Moving from these restricted domain specific languages into graphical forms is also much easier than trying to graphically specify a C++ program. In fact, a designer working with a tool generating XAML makes the V&V loop so tight it nearly disappears.

I think the fact that strict GUI code like the above doesn’t really require testing isn’t all that new. Most corporations doing ‘enterprise-y’ work already know this subconsciously, using languages like C++ or C# with rich tool support for their markups. It’s in the younger dynamic languages without as much support that we fall back into thinking we need to test GUI’s. We do not need to test GUI’s, at least not in the traditional way. GUI’s are, quite frankly, easy today, and incredibly well specified. Leverage these specifications in a DSL designed for GUI’s rather than poking and prodding all your slots and signals to make sure you didn’t screw up in building your GUI. Stop testing and learn to love proof-by-construction that these type of DSL’s give us.

June 16, 2011 Posted by | Uncategorized | Leave a comment

Standardized Testing is the Worst Means of Qualifying People…

…except for all the others. This morning there was an interesting blurb on NPR about the LSAT, namely how it discriminates against the blind by including questions that are best solved with a diagram. The real meat I’m going after here is almost a foot note in the story, that is that some schools would like to drop the requirement for the LSAT altogether.

For many professional schools, like Law or Medicine, the LSAT or MCAT is a great denominator. It makes up the burden of your application materials, and doing well on either almost guarantees you a made man/woman. Compare this to getting a job, where most of the decision process is made during interviews and based on a resume. Getting into law school is tough, and may be the hardest thing some people do, and it isn’t exactly fair, with well-known biases. But getting a job is unfair in an entirely different way. First, there’s no known hurdles to jump through, unlike law school. The LSAT is a tough, but concrete, hurdle. Getting a job requires a bunch of weird arbitrary strokes of luck for the most part, rendering it a crap shoot for all practical purposes. Furthermore, you’re up against completely unknown and unfamiliar biases. Did you miss that last interview because you were black and the interviewer was white? He certainly seemed put off when he first saw you.

Getting into law school requires a tough, biased but well understood measure to be levied against you. Getting a job is based on no uniform measure at all, and a great many problems can come with that.

Law schools would very much like to drop the weighting on the LSAT (currently it’s required to be a legitimate law school) and more heavily weight other parts of the application, which to remind you, are mostly essays, a resume and references. The most concrete thing I can gather from an essay about you is how well you can write – lucky for us the LSAT already tests for that. Far less concretely, I can get a vibe for the culture you come from both by how you write and what you chose to write about. It’s like having a one-sided conversation, where you’ll get no feedback from me, yet I’m asking you to pour out your soul. Even worse are the resume, which is used primarily to see if you’ve worked anywhere with a name brand and went to a name brand school*, and the references which amount to whether you know anyone who’s already been to that school.

By putting less emphasis on the standardized test, schools will be given far more leeway to simply chose people like them, rather than people who are qualified. You already see this in many business schools, who bemoan fit as the most important thing in an application. Law schools will start to look more and more for fit, rather than qualification, and use it as an excuse to reject whomever doesn’t fit their stereotype of the day of the successful lawyer. Fit in the past kept out Blacks from higher education, Jews from the Ivy Leagues, and is being used right now to cut down on the number of first generation Asian Americans in business schools. It also provides justification for legacy applications – after all, if your parents fit, then you most likely do too.

Standardized testing isn’t the problem, standardized tests are. Don’t lessen the impact of a universal measure that is well-known and hard to corrupt, instead focus on eliminating biases in the tests as we find them. To say a standardized test is biased in favor of current students reading application essays and former students performing interviews is a farce.

(* The trick here with business schools, at least, is to require a cover letter and resume up front. They don’t use them as genuine application materials per say, but instead use them to see how well you can write a cover letter and resume. Remember, business school rankings are determined in a large party by who gets hired and what their salary is on graduation. People coming in to schools who already are good at applying for jobs have a leg up here, and thus are less risky than other students. In this way, many business schools tend to work like a self-reinforcing filter system, vouching for those who are already vouched for, and getting jobs for people who can already get them, than a true education system which improves upon any base ability.)

June 15, 2011 Posted by | Uncategorized | Leave a comment

Why are unhealthy people so stupid?

That ought to have been the alternative title to this article. The LA Times asks the following question: Why do unhealthy people not adopt more healthy lifestyles? Implied in the question is the whispered do they just not know or care that they’re unhealthy?

Thus, our crusaders real agenda has arrived. We’ll regurgitate truisms and scary facts about how you need to change your sinful ways, oh gluttons, because we believe you simply haven’t understood us yet. You are, after all, well, kinda stupid.

Why don’t fatties lose weight? Especially after a heart attack? Surely they know the health risks they undertake by… oh wait, maybe they don’t. Maybe we should tell them. Maybe the next time I’m at the grocery store and some single mom is buying ice cream for her kids’ birthday party, I’ll make sure she knows that ice cream is unhealthy, and that she’s going to kill herself. After all, it’s obvious she’d only buy ice cream because she were ignorant.

Enough of that. While I’ll commit a mortal sin here and argue against this article using anecdotal evidence, I guarantee if someone were to look for the statistics to back this up, they’d find it. Why don’t fatties diet? What an absurd question. Everyone I know who is overweight or believes themselves to be is either dieting right now or has in the recent past. From fad diets like the Zone and Ornish, to weight loss clinics, to Jennie Craig, to paleo-nonsense and organic foods. Overweight people are trying their damndest. This doesn’t jive with the collected sermons that fatties are both lazy and stupid, and that’s why they’re in the straits they are. It allows otherwise nice people to look down with disgust upon you, and justifies one of the few remaining bigotries left. You see, obesity isn’t a health condition, it’s a moral condition.

That underlying theme permeates all we hear, and we make implicit little jumps from it whenever we try and analyze this issue. For instance, did you know that people who’ve had a heart attack fail to lose weight afterwards? Let’s de-obesitize this. People who’ve suffered from a condition fail to follow up on a treatment that helps with that condition. “Hrm,” we all say, “the treatment must be pretty harsh for people to risk their lives like that.” A reasonable conclusion. Adding obesity back in, we suddenly change our minds: “Aha! It’s the obese! It’s not a harsh treatment, it’s the fact that as we know, fatties are lazy and stupid, that’s why they don’t stick to their treatment.” Our conclusions should be based on the data, not the type of people we’re concluding about.

Why don’t obese people lose weight after a heart attack? The hell if I know. I can tell you that most of them probably attempted to lose weight before the heart attack, and the majority probably tried after the heart attack. But do we measure effort? No, we instead measure results – no weight lost – and then blame effort. We turn a blind eye again and again to how hard many people are trying to fit into the cookie cutter figures we expect in society, so that we can blame them again and again for their lack of discipline and moderation.

Convenient for us, but bigotry is sadly too familiar to us to not recognize it. Why, indeed, could we be so stupid to fall for this again.

June 13, 2011 Posted by | Uncategorized | Leave a comment


The Four Hour Workweek blog interviews Rob Mee on ‘classic entrepreneurial myths’.  There are a few things I agree with, and a few I don’t. It’s not that controversial to agree with someone, so I’ll focus on where we part ways. You’ll notice a pattern.

Rob says that interruption is unavoidable in the modern work place, and then somehow follows with a non-sequiter about pair programming. I have nothing against pair programming, but it is not a golden hammer. It is just another tool in the chest. Some people liken it to a continuous peer review, which it can be if used successfully. But it can also very much be likened to a continuous meeting.

It’s a slippery slope to go from pair programming back to the bull pen. As Peopleware very successfully argued years ago, measurable progress, in terms of features completed and defects prevented or fixed, increases when distractions decrease. I could write a whole ranting blog post about why manager’s like pair programming (hint, it’s the same reason managers like meetings and bull pens) but that’s not really the point here. The point is that sometimes you need to collaborate, sometimes you need to be alone. And when I say alone, I mean alone alone. Private offices, chat/phones/twitter off, and allowed to concentrate.

“But SkepticalMethodologist!”, you might respond, “That implies management is harder than simply taking one philosophy (collaborate/work alone) and taking it to its extreme!” Yes, and I’m sorry I’ve scared you so. Management… nay, Leadership requires reacting to the problem at hand in its context and solving it with the best tool available for the job. It cannot be outsourced to some guru or methodology. It also implies that if you want to succeed, you need to have ways to easily switch between collaborating and working alone. This implies some excess in capability – if we don’t focus on one thing the whole time (collaboration/working alone), we can no long afford to neglect alternative methods. That means, yes, you should have private offices. But yes, you should have open spaces with white boards a plenty! And no, they will not always be in use!

The second point I disagree with Rob on is his argument against specialization. The argument goes that if you have a specialist, then you have a risk, and risks can sink you. There are a few things wrong with this – the most glaring is that successful companies did not succeed because they were so insured against risk that nothing could sink them. Instead, they succeeded because they took risks, rolled the die, and won. Which I guess gives Rob less consulting work to do, as entrepreneurs who understand that a great deal of success is luck probably aren’t willing to shell out as much in ‘guaranteed success’ consulting fees. Another is deconstructing this argument to a classic management laziness that would rather hire and fire a commodity product (generalists) than something hand crafted and unique (specialists).

Most importantly, though, is that frequently a business must do one thing well. That’s more or less what a business in a free market – it’s a specialization, an optimization of the local economic space. It does one thing better than it’s competitors, well enough, in fact, to make an economic profit.

Focusing on generalists won’t get you that.

The main risk of having specialists is losing them. The response should not be to only hire people who are easily replaced, but go to the human root of the problem and decrease turnover that way. Why is your specialist leaving? If you can’t afford to compensate your specialists or you treat them like shit, well then, hiring generalists just make it easier for you to be a cheap asshole. They don’t increase the chances of your success at all. Certainly there are reasons outside of your control in people leaving – perhaps their spouse got a job elsewhere and they aren’t willing to telecommute, or they want to shift careers altogether, or their hit with health problems, etc…

Engineers deal with these kinds of risks with redundancy. After all, a team of general practitioners is simply not as qualified to deal with my brain fog as a single neurologist. And if that single neurologist fails, I get a second opinion. Having a specialist is not a risk. Having only one specialist and not having plans to get another is. Having two specialists, or more, is expensive. It’s hard to either pay them, as a start up providing a salary, or get their buy in, if they want a piece of the company. Too bad! That’s your job as a leader. Making your job easier doesn’t help the company grow. Solving high level problems like finding expertise in the domain you’re trying to break into is. And I’m sorry it’s so hard. Really, I’m fighting back tears here.

The theme of both of these points is slack. You need slack to react, to stay agile, and to turn your company from a cupcake bakery into a hedge fund in the blink of an eye. Companies that can change and adapt thrive. Adaptation takes slack, and yes, slack can be expensive. But worth it.

June 9, 2011 Posted by | Uncategorized | Leave a comment