The Skeptical Methodologist

Software, Rants and Management

Testing the Untestable

TDD is a great way to develop certain types of code, but you frequently run into situation where you either run into the question of “How do I test that?” or more appropriately, “Is it worth testing that?”

We need to remember that testing is just one tool we have towards a particular goal – verifying and validating software. In many cases, other tools and techniques that do the same thing can very much supplement or even replace testing. The best example of a formal system we all know and love is the type system. In a statically typed language, I don’t have to test for type errors. They are formally proven out of the code (unless you’re the tool who passes void* everywhere). That cuts down on the amount of testing I have to do for V&V, as well as supplements testing as a design tool. While TDD gives us clean, modular designs that are very decoupled from each other, the type system gives us help with design-by-contract and cleaner mechanisms of dispatch. They both have their strengths and weaknesses.

Does it work the other way? In other words, I’ve given an example of where we have another tool that limits what we have to test. But can we find a mapping from the hard-to-test to a tool? Not always, but in many cases, yes. In particular, I want to talk about the testing of GUI’s.

First, following a MVC design pattern helps dramatically with testability. Fact is, testing the controller and model leverages far more traditional techniques. The view has always been the problem – how do we test GUI code? Certainly we can just stick to our guns on TDD, mocking out nearly all the library calls we make to our GUI framework, possibly writing drivers that control mouse and keyboard input to pick and poke the interface. These sometimes have their place, especially if you can leverage pre-existing resources. Remember, though, testing is done primarily for V&V. Is there another way to validate and verify our GUI software?

Enter XAML. Or QML, or HTML for all that matters. Each of these fits the same slot – it is a language for specifying the layout of GUI’s. The most important aspect of each of these is that none of them are Turing complete. This drastically limits the amount of damage we can do to ourselves, and also dramatically expands what we can prove about our code automatically. Specifying a layout in one of these markup languages rather than in a universal Turing complete language like C++ is both easier and far less bug ridden. It’s simply impossible to make certain mistakes. Knowing that an XAML document is well-formed and parseable gives much greater guarantees than knowing a program is well-formed and compilable. And we know how important keeping a program in a compilable state is!

Moving from these restricted domain specific languages into graphical forms is also much easier than trying to graphically specify a C++ program. In fact, a designer working with a tool generating XAML makes the V&V loop so tight it nearly disappears.

I think the fact that strict GUI code like the above doesn’t really require testing isn’t all that new. Most corporations doing ‘enterprise-y’ work already know this subconsciously, using languages like C++ or C# with rich tool support for their markups. It’s in the younger dynamic languages without as much support that we fall back into thinking we need to test GUI’s. We do not need to test GUI’s, at least not in the traditional way. GUI’s are, quite frankly, easy today, and incredibly well specified. Leverage these specifications in a DSL designed for GUI’s rather than poking and prodding all your slots and signals to make sure you didn’t screw up in building your GUI. Stop testing and learn to love proof-by-construction that these type of DSL’s give us.

June 16, 2011 - Posted by | Uncategorized

No comments yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: