How to Test

I was talking to my very good friend about testing today and wanted to throw together a list of the heuristics that I’ve “discovered” which I found, personally, very difficult nuggets to discover through my career. Dan North in his article, “Introducing BDD,” describes the story arch in his career which sounds very much like my own:

The deeper I got into TDD, the more I felt that my own journey had been less of a wax-on, wax-off process of gradual mastery than a series of blind alleys. I remember thinking “If only someone had told me that!” far more often than I thought “Wow, a door has opened.” I decided it must be possible to present TDD in a way that gets straight to the good stuff and avoids all the pitfalls.

Example Code

Before we begin, I’m going to use a mutable Stack implementation in Scala as an example of code that should be easy enough to read and understand, regardless of the languages you’re familiar with.

Please note that I did not use a pure functional implementation with the intention of the code being more easily read by people coming from an imperative background!

A stack is a “last-in-first-out” data-structure that lets you place an item on the top of the stack (“push”) or retrieve and remove the item on the top of the stack (“pull.”) There is generally an operation called “peek” as well which will retrieve the item on the top of the stack without removing the value from the stack.

You can find an example of a Java Stack here if you’d like further implementation details but it’s not necessary for comprehending the content in this article.

Naming Tests is Describing Behaviour

I’ve seen a lot of code in different environments in my career. One of the patterns I’ve seen in less experienced development teams is code like the Java JUnit example below.

public class StackTests {
    @Test
    public void testPeek() { ... }
}

The test name seems reasonable at first glance – it’s clear that the stack is the class under test by the test suite name, we’re trying to test the Stack’s “peek” method in the peekTest method.

Well, the problem is that we know what noun we’re testing, but we don’t know which specific behaviour is being exercised. Is it when it’s empty? Is it when it’s loaded with data? Should it do something different in these two cases?

One of the qualities of good tests is that they act as documentation to someone trying to read and understand the code. By describing the behaviour we are trying to test, we are also creating documentation that is forced to stay up to date with the code.

A good heuristic is to start a test name with “it should.” This forces the writer of the tests to focus on the behaviour instead of the noun. And this is the foundation of Behaviour Driven Development (BDD) – a focus on behaviour!

Let’s have another look at that JUnit test applying this practice.

public class StackTests {

    @Test
    public void itShouldReturnNothingWhenPeekingEmptyStack() { ... }
}

Most modern testing tools have been influenced by BDD so they will generally try to guide you toward describing behaviour. You can see in our JUnit example above that the test name is very long. Modern tools will help you organize your code by allowing you to describe different scenarios, and then describe the behaviour. Below is an example from ScalaTest which gives several different semantic options for describing your tests.

class StackSpec extends FlatSpec with Matchers {
  "An empty Stack" should "return None when peeking" in { ... }
  
  it should "return None when popping" in { ... }
}

How to TDD

Knowing the word TDD and understanding it in practice are two different things. TDD does not simply mean writing tests with your code, nor does it mean writing all of the tests before the code is written, rather, it is a practice of allowing the writing of code to be guided by the addition of tests, one at a time. The best way to highlight this is to demonstrate the addition of tests and code for our stack implementation.

The heuristic is: write the most general test that you can, and do the minimal work to implement the behaviours described so far in a test. This might mean hard coding a return value or stubbing features that aren’t yet under test as you go.

For example, let’s say we start with the test described above again:

class StackSpec extends FlatSpec with Matchers {
  "A an empty Stack" should "return None when peeking" in { ... }
}

The simplest thing to do here is to just hardcode the return value.

class Stack {
  def peek = { None }
}

Now, this obviously isn’t correct, but we chose the most general test, and then added the behaviour, and the test now passes. We can now add another test, and write the code to make it pass.

class StackSpec extends FlatSpec with Matchers {
  "An empty Stack" should "return None when peeking" in {
    val stack = new Stack(List())
    assert(stack.peek == None)
  }
  "A non-empty Stack" should "peek last value" in {  
    val stack = new Stack(List(1, 2, 3))
    stack.peek should equal(Some(3)) 
  }
}

class Stack(initialData: List[Int] = List.empty[Int]) {
  private var data: List[Int] = initialData.reverse
  def peek = { 
    if(data.nonEmpty) Some(data.head) else None 
  }
}

You can continue to add behaviour in this manner, and just continue to change the code to make the tests pass. Adding our first push behaviour, for example:

class StackSpec extends FlatSpec with Matchers {
  "An empty Stack" should "return None when peeking" in { ... }
  "A non-empty Stack" should "peek last value" in { ... }

  it should "push a value to the front of the queue" in { 
    val stack = new Stack(List(1, 2, 3))
    stack.push(4)
    stack.peek should equal(Some(4))
  }
}

class Stack(initialData: List[Int] = List.empty[Int]) {
  var data: List[Int] = initialData.reverse
  def peek(): Option[Int] = { 
    if(data.nonEmpty) Some(data.head) else return None 
  }
  def push(newValue: Int) {
    data = newValue :: data
  }
}

And so on, like this. There are two scenarios where TDD is very good:

• finding and fixing bugs: add the tests to the behaviour to show its broken and to cover the expected behaviour (because there is obviously no test for it!) Then you have a repeatable mechanism for demonstrating the bug so you can easily do analysis to find the root cause.
• building new features: this is really where TDD shines – adding new features is a joy with TDD.

Ping Pong – TDD With Other People

This is also a very fun thing to do with another person! The game of “Ping Pong” translates to TDD very well. With yourself and another person, you separate roles of test writer and implementor and you both take turns in a round of writing the behaviour and then writing the next test. So the activities above would be done like so:

1. Person A:
   1. Writes the test: “An empty Stack” should “return None when peeking”
2. Person B:
   1. Implements the minimum code to make the test suite pass.
   2. Writes the test: “A non-empty Stack” should “peek last value”
3. Person A:
   1. Implements the minimum code to make the test suite pass.
   2. Writes the test: “A non-empty Stack” should “push a value to the front of the queue” 

You continue like this until the feature is done. It’s really a lot of fun if you both understand the “rules.”

• When writing tests: Write the most general test to the most specific.
•  When writing code: Implement the smallest amount of code to get the tests passing.

It’s not screw your neighbour per se… But if you can find a “creative” way to make the tests pass by writing less code on your turn… the option is there!

At some point a test that is introduced may force the hand of the pair to write a lot of code before the next test – this is okay and you can still pair as you normally would through that period of pairing, trading off the driver/navigator roles as desired.

Test Bottom Up: From Unit to Integration

I’m starting a role at Elastic and their Developer Constitution sounds like it could come from my own mouth. I deeply resonated with this article when I bumped into it, and the heuristic laid out here come from sometimes painful experiences so avoid those errors and follow the guiding principals. You’ll still get to learn from mistakes, you just don’t have to make them yourself.

This section on testing is a good heuristic.

Test bottom up. If you write code, write unit tests first. Write many of them. Write code so you can write many of them. Integration testing is the last step. Focus on adding more tests that execute fast and are easy to debug, like unit tests. This is crucial for developer velocity.

So the idea is to Unit test class/modules in isolation. And then to test them integrated together, and integrated with whatever dependencies it has.