The Skeptical Methodologist

Software, Rants and Management

SYWTLTC: (AB) Chapter 3.1 Quality: Test Driven Development

Go here if you want the prolog and table of contents to the SYWTLTC series!

In this and future chapters, treat all links as required reading.

Five Pillars of Quality

I claim that there are five pillars of quality in software.

  1. Testing and Dynamic Analysis
  2. Static Analysis
  3. Peer Review
  4. Contracts and Types
  5. Design

You can find people who will swear by one, or even two. Some folks will say all you need is tests, others will say that types are the only way to prove there are no bugs in your code.

Each of these pillars has strengths and weaknesses, but they all tend to be very complimentary. That means they work best as a team.

But first…

Why Quality? And Why so Early?

We all want to be ‘good’ at what we do, we all want to produce ‘quality’ work, sure. But I’ve got to get this script out by tomorrow, so we can keep all the nice and pretty stuff like testing for tomorrow, I have real work to do today!

This is very myopic thinking!

Ultimately, we are looking how to be productive coders. Quality is part of productivity, it’s part of being fast – it is not the opposite of being fast! The fastest coders out there code quality, and the reason is that the number one thing that’s going to slow you down is rework. After all, we never seem to have time to do things ‘right’, but we always seem to have time to ‘do them over’. The previous sentence should bother your logic center, as clearly – we have time to do things right if we have time to do them over.

And if we do them right in the first place, we don’t have to redo them later, and we go much faster.

Software maintenance costs us about 60% of our total effort we put into software, the rest going to requirements, design and coding. That means that our coding time – the part you think you’re ‘speeding up’ by not doing quality work – accounts for a very small part of our overall efforts. You may be penny wise but pound foolish.

What is maintenance? It’s anything you do to code after it’s already written. That’s going to be the lion’s share of your work – and you probably have already noticed that you’ve done a lot of maintenance. You attempt to write out some code to solve a problem, and it doesn’t work exactly right. Everything you do after that point is maintenance. When you hack on your program to attempt to make it work, that’s maintenance.

It’s most of what you do as a developer.

When we forego quality, we rack up what’s called ‘Technical Debt‘. We call it debt like credit card debt because, from the time we take it on to the time we pay it off, we have to pay ‘interest’ in terms of effort. We go slower and slower in future projects, spending more and more time hacking through our low-quality code to get things done.

Debt is generally a bad thing until you know how to deal with it. So for now, it’s best to learn how to never go into debt, as well as if you find yourself in debt, how to dig yourself out.

Why so early, though? Why do you need to learn about quality, now? You barely know how to do string manipulation or arithmetic in Python. Why are you having to worry about getting things perfect now?

There are two main reasons.

What did I JUST SAY about productivity?

Do you want to learn how to code faster? I just said that quality is the same thing as productivity because it prevents rework. Why wouldn’t you learn quality as soon as possible so that you can blow through the rest of these lessons as fast as possible?

Learn Good Habits Early

You’re going to run into a lot of coders who refuse to test. Who review peer review. Who think static analysis is a waste of time. They never really bit the bullet to learn the right way to do things, and they don’t like how testing, peer review, and other processes make them feel dumb.

We end up justifying a lot of stuff to ourselves to avoid psychological pain. Tests are painful – they’re going to tell you where you screwed up. But they don’t have to be – if you learn good testing discipline early, you’ll realize that tests are just a part of the process, not a tool designed to make you feel dumb. You’ll realize that everyone makes mistakes – a lot of mistakes – and the earlier we catch them, the better for everyone. Macho coders who refuse tests, reviews and other help aren’t all that good, they just don’t want to be ‘exposed’ as an imposter.

Learn these things now so you never really learn how to ‘code’ without them – an early realization of their value means you never have to think “I can either implement this thing my Boss wants by tomorrow, or learn to test.” You’ll already know how to test.

You’ve Already Been Doing It

Code Combat is a test driven system – you had to keep trying out your code on the right to see if you passed the challenge on the left. Each state is a test, and it had certain requirements to move on to the next stage. That’s the exact same thing as a test.

Of course, Code Combat also gave you a very nice visual debugger too – you got to see where your character was. So keep in mind, testing and using the debugger go hand in hand.

On to Testing!

Early on, I advised learning how to do what’s traditionally called unit testing. Traditional unit testing is when you write some code, and then write tests that exercise that code. We’re going to be doing the opposite, though, and focus on Test-Driven-Development, or TDD.

What is TDD? What are its benefits?

TDD is when you write the tests first, then you write code that passes that test.

The benefits of TDD are as follows:

  • In the traditional approach, you can’t guarantee that you’ll write code that’s easily testable. As you get more experience coding and testing, you’ll realize that some code is hard to test, while another code seems easier to test. If you start with the tests first, you’ll almost always, instinctively, write code that’s easy to test.
  • You reduce pressure from management. If you write code first, bad managers might be tempted to ask you to deploy what you have, and explain “we can always test later.” If you start with the tests first, you never can be pressured to deliver before things are quality and get into technical debt.
  • TDD is also sometimes called ‘Test Driven Design’. This is because sometimes starting with the tests helps us think about our code as already complete and well designed – how would you like to interact with the module you’re writing? If you test against that design, then you’ll be forced to code to that design. Too often, if we try and build prototypes first, we end up testing whatever design we get that works. We don’t think about how we want our code to look from the outside and make sure it looks like that. With TDD, we get those benefits.

How do I do TDD in Python?

I’m glad you asked!

First, watch this video.

Then, read this article.

We’ll be using py.test from here on out, but it’s useful to see other testing frameworks like nosetest and unittest to see similarities.

How do I know when I’m done, or if I’ve done a good job?

Testing and TDD have a great initial metric of ‘goodness of testing’ called coverage. Coverage is a measure of how many lines of your program were executed by your tests.

Generally speaking, higher coverage is better, and if you can get 100% that’s great, although some functions are intrinsically harder to test. There are many kinds of bugs that can sneak through 100% test coverage. Coverage is a good first metric to watch, though.

Coverage has been shown to improve perceptions of quality by nearly 10%.

You can get a plugin for py.test that adds coverage reporting here. Read through the overview to see how to use it, specifically how to generate an HTML report! When you generate an HTML report, you should be able to open the index.html file it generates (in a directory it makes) in your browser to see a very nice, colorful coverage report built for you automatically.

Once you’re done with the videos, reading and tools above, you’re ready for this module’s code challenge.

Code Challenge

You need to fork this repo on GitHub.

I’ve started a simple calculator module and tests, with the “calculator_add” function, and “test_calculator_add” test.

I want you to fully implement the simple arithmetic for a calculator:

  • Add “calculator_subtract”, “calculator_multiply” and “calculator_divide” functions.
  • Add appropriate tests.
  • Do it all TDD style

For the purposes of this exercise, you should be able to reach 100% code coverage easily.

To ensure you’re following TDD, please use the following Github workflow:

  1. Fork the source repo
  2. Create a branch where you’ll do your work
  3. Add a test THAT FAILS, commit your work.
  4. Add code that makes the test pass, commit your work.
  5. Go to 3 until all functions are added.
  6. When you’re done, open a pull request on your branch (how to open a pull request)

What you are doing, and what TDD emphasizes, is also known as unit testing. There are many other forms of testing, but no one seems to agree on hard and fast definitions so we’re gonna skip over them for now.

The important thing to note is that unit tests and tests you write need to be reasonably fast. This is so you can run them again and again and again – even after every file change. A good habit to get into is every time you save your file, you should run the test.

There’s actually a helper function in py.test that will automatically rerun your tests for you as you make changes to a file. It’s a great idea to use this plugin and keep at least two windows open, one for your test runs constantly and one for your text editing. (You may even want three, one text editor, one test window, and finally, a window with IPython running where you can interactively prototype your program.)

Finally, there is a py.test option to drop into the debugger on test failure. Try it out.


For this challenge, please confirm that your mentee has built the above-mentioned functionality, that they can generate an HTML coverage report and that they’ve reached 100% test coverage. You can confirm this in person.

You’ll also need to confirm they know how to use the looponfail feature and the debug on fail feature of py.test.

You need to also review the commit history in their repo to make sure they’re following TDD.

Onward and Upward!

First, the requirements imposed above – that you’ll need 100% test coverage, or as high as you can get it, and that you’ll commit after every test and after every passing of a test, are requirements of all future challenges. Your mentor will double check that you are keeping test coverage high and working in a TDD fashion!


October 21, 2016 - Posted by | Uncategorized


  1. […] the 5 pillars of quality, up next is static analysis. As always, treat all links as required reading unless stated […]

    Pingback by SYWTLTC: (AB) Chapter 3.2 : Static Analysis « The Skeptical Methodologist | November 11, 2016 | Reply

  2. […] SYWTLTC: (AB) Chapter 3.1 Quality: Test Driven Design […]

    Pingback by So You Want To Learn to Code: Prologue « The Skeptical Methodologist | November 14, 2016 | Reply

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: