The Skeptical Methodologist

Software, Rants and Management

SYWTLTC: (AB) Chapter 3.1 Quality: Test Driven Development

Go here if you want the prolog and table of contents to the SYWTLTC series!

In this and future chapters, treat all links as required reading.

Five Pillars of Quality

I claim that there are five pillars of quality in software.

  1. Testing and Dynamic Analysis
  2. Static Analysis
  3. Peer Review
  4. Contracts and Types
  5. Design

You can find people who will swear by one, or even two. Some folks will say all you need is tests, others will say that types are the only way to prove there are no bugs in your code.

Each of these pillars has strengths and weaknesses, but they all tend to be very complimentary. That means they work best as a team.

But first…

Why Quality? And Why so Early?

We all want to be ‘good’ at what we do, we all want to produce ‘quality’ work, sure. But I’ve got to get this script out by tomorrow, so we can keep all the nice and pretty stuff like testing for tomorrow, I have real work to do today!

This is very myopic thinking!

Ultimately, we are looking how to be productive coders. Quality is part of productivity, it’s part of being fast – it is not the opposite of being fast! The fastest coders out there code quality, and the reason is that the number one thing that’s going to slow you down is rework. After all, we never seem to have time to do things ‘right’, but we always seem to have time to ‘do them over’. The previous sentence should bother your logic center, as clearly – we have time to do things right if we have time to do them over.

And if we do them right in the first place, we don’t have to redo them later, and we go much faster.

Software maintenance costs us about 60% of our total effort we put into software, the rest going to requirements, design and coding. That means that our coding time – the part you think you’re ‘speeding up’ by not doing quality work – accounts for a very small part of our overall efforts. You may be penny wise but pound foolish.

What is maintenance? It’s anything you do to code after it’s already written. That’s going to be the lion’s share of your work – and you probably have already noticed that you’ve done a lot of maintenance. You attempt to write out some code to solve a problem, and it doesn’t work exactly right. Everything you do after that point is maintenance. When you hack on your program to attempt to make it work, that’s maintenance.

It’s most of what you do as a developer.

When we forego quality, we rack up what’s called ‘Technical Debt‘. We call it debt like credit card debt because, from the time we take it on to the time we pay it off, we have to pay ‘interest’ in terms of effort. We go slower and slower in future projects, spending more and more time hacking through our low-quality code to get things done.

Debt is generally a bad thing until you know how to deal with it. So for now, it’s best to learn how to never go into debt, as well as if you find yourself in debt, how to dig yourself out.

Why so early, though? Why do you need to learn about quality, now? You barely know how to do string manipulation or arithmetic in Python. Why are you having to worry about getting things perfect now?

There are two main reasons.

What did I JUST SAY about productivity?

Do you want to learn how to code faster? I just said that quality is the same thing as productivity because it prevents rework. Why wouldn’t you learn quality as soon as possible so that you can blow through the rest of these lessons as fast as possible?

Learn Good Habits Early

You’re going to run into a lot of coders who refuse to test. Who review peer review. Who think static analysis is a waste of time. They never really bit the bullet to learn the right way to do things, and they don’t like how testing, peer review, and other processes make them feel dumb.

We end up justifying a lot of stuff to ourselves to avoid psychological pain. Tests are painful – they’re going to tell you where you screwed up. But they don’t have to be – if you learn good testing discipline early, you’ll realize that tests are just a part of the process, not a tool designed to make you feel dumb. You’ll realize that everyone makes mistakes – a lot of mistakes – and the earlier we catch them, the better for everyone. Macho coders who refuse tests, reviews and other help aren’t all that good, they just don’t want to be ‘exposed’ as an imposter.

Learn these things now so you never really learn how to ‘code’ without them – an early realization of their value means you never have to think “I can either implement this thing my Boss wants by tomorrow, or learn to test.” You’ll already know how to test.

You’ve Already Been Doing It

Code Combat is a test driven system – you had to keep trying out your code on the right to see if you passed the challenge on the left. Each state is a test, and it had certain requirements to move on to the next stage. That’s the exact same thing as a test.

Of course, Code Combat also gave you a very nice visual debugger too – you got to see where your character was. So keep in mind, testing and using the debugger go hand in hand.

On to Testing!

Early on, I advised learning how to do what’s traditionally called unit testing. Traditional unit testing is when you write some code, and then write tests that exercise that code. We’re going to be doing the opposite, though, and focus on Test-Driven-Development, or TDD.

What is TDD? What are its benefits?

TDD is when you write the tests first, then you write code that passes that test.

The benefits of TDD are as follows:

  • In the traditional approach, you can’t guarantee that you’ll write code that’s easily testable. As you get more experience coding and testing, you’ll realize that some code is hard to test, while another code seems easier to test. If you start with the tests first, you’ll almost always, instinctively, write code that’s easy to test.
  • You reduce pressure from management. If you write code first, bad managers might be tempted to ask you to deploy what you have, and explain “we can always test later.” If you start with the tests first, you never can be pressured to deliver before things are quality and get into technical debt.
  • TDD is also sometimes called ‘Test Driven Design’. This is because sometimes starting with the tests helps us think about our code as already complete and well designed – how would you like to interact with the module you’re writing? If you test against that design, then you’ll be forced to code to that design. Too often, if we try and build prototypes first, we end up testing whatever design we get that works. We don’t think about how we want our code to look from the outside and make sure it looks like that. With TDD, we get those benefits.

How do I do TDD in Python?

I’m glad you asked!

First, watch this video.

Then, read this article.

We’ll be using py.test from here on out, but it’s useful to see other testing frameworks like nosetest and unittest to see similarities.

How do I know when I’m done, or if I’ve done a good job?

Testing and TDD have a great initial metric of ‘goodness of testing’ called coverage. Coverage is a measure of how many lines of your program were executed by your tests.

Generally speaking, higher coverage is better, and if you can get 100% that’s great, although some functions are intrinsically harder to test. There are many kinds of bugs that can sneak through 100% test coverage. Coverage is a good first metric to watch, though.

Coverage has been shown to improve perceptions of quality by nearly 10%.

You can get a plugin for py.test that adds coverage reporting here. Read through the overview to see how to use it, specifically how to generate an HTML report! When you generate an HTML report, you should be able to open the index.html file it generates (in a directory it makes) in your browser to see a very nice, colorful coverage report built for you automatically.

Once you’re done with the videos, reading and tools above, you’re ready for this module’s code challenge.

Code Challenge

You need to fork this repo on GitHub.

I’ve started a simple calculator module and tests, with the “calculator_add” function, and “test_calculator_add” test.

I want you to fully implement the simple arithmetic for a calculator:

  • Add “calculator_subtract”, “calculator_multiply” and “calculator_divide” functions.
  • Add appropriate tests.
  • Do it all TDD style

For the purposes of this exercise, you should be able to reach 100% code coverage easily.

To ensure you’re following TDD, please use the following Github workflow:

  1. Fork the source repo
  2. Create a branch where you’ll do your work
  3. Add a test THAT FAILS, commit your work.
  4. Add code that makes the test pass, commit your work.
  5. Go to 3 until all functions are added.
  6. When you’re done, open a pull request on your branch (how to open a pull request)

What you are doing, and what TDD emphasizes, is also known as unit testing. There are many other forms of testing, but no one seems to agree on hard and fast definitions so we’re gonna skip over them for now.

The important thing to note is that unit tests and tests you write need to be reasonably fast. This is so you can run them again and again and again – even after every file change. A good habit to get into is every time you save your file, you should run the test.

There’s actually a helper function in py.test that will automatically rerun your tests for you as you make changes to a file. It’s a great idea to use this plugin and keep at least two windows open, one for your test runs constantly and one for your text editing. (You may even want three, one text editor, one test window, and finally, a window with IPython running where you can interactively prototype your program.)

Finally, there is a py.test option to drop into the debugger on test failure. Try it out.


For this challenge, please confirm that your mentee has built the above-mentioned functionality, that they can generate an HTML coverage report and that they’ve reached 100% test coverage. You can confirm this in person.

You’ll also need to confirm they know how to use the looponfail feature and the debug on fail feature of py.test.

You need to also review the commit history in their repo to make sure they’re following TDD.

Onward and Upward!

First, the requirements imposed above – that you’ll need 100% test coverage, or as high as you can get it, and that you’ll commit after every test and after every passing of a test, are requirements of all future challenges. Your mentor will double check that you are keeping test coverage high and working in a TDD fashion!


October 21, 2016 Posted by | Uncategorized | 2 Comments

Don’t Mistake Engagement for Passion

Passion is used as a code word nowadays by many hiring managers. It means “will work a lot of extra hours for free”. Rightfully so, many people are turned off by this idea about passion, about passionate developers, about looking for passionate hires.

But you have to give the mislead hiring managers a little slack. After all, they’ve been asked to hire good talent, so they look around at the company and they ask “well, who’s good here, and how do we hire more of them?”

They see some of your best people putting in hours on the weekend, staying late, and taking their work very seriously. And they – without much critical thought – think, “Ah, yes, it’s those type of people we want. I’ll just hire more of them!”

But here’s the problem – those people aren’t passionate they’re engaged. And there’s a huge difference.

Passion is hired. Engagement is earned.

In fact, even the above is a simplification, since ‘passion’ is so hard to find. You really have to find boy-geniuses (and they’re almost always males) fresh out of school who seem to know the latest technology and be willing to work all hours of the night because they don’t know any better.

Then they churn out some hacked together crap as fast as possible, and seem to provide evidence that you have to hire passionate people who are “smart and get things done”. People who are worried about “work-life balance” or “working smarter, not harder” aren’t going to fit into your “workplace culture” – you hire passionate people and expect the best!

But you have no diversity, which means your innovation sucks. And you have no senior developers, which means your quality and repeatability sucks. You have software that sometimes works and is a few months away from crumbling under its own weight. You wrote facebook in PHP and have no idea how to add new features to stay competitive – everything seems to take forever to change.

What gives?

Engagement is frequently confused for passion. Engagement is “I believe in this company and this project.” Can you hire engaged people? No, how would you? You can’t truly believe in a group of people or what they work on until you’ve… worked with them. Usually for awhile.

There’s no question you can ask in the interview process that’s going to determine whether someone will be engaged or not. That’s because everyone is capable of it. Everyone can become engaged – they can believe in the people they work with, and the project they work on, and when they do that they become internally motivated to push hard for those two things.

So instead of hiring for ‘passion’, try hiring good all around talent and build a team that people are proud to be a part of, and a project that they’re proud to work on.

October 17, 2016 Posted by | Uncategorized | Leave a comment