A Best Practices Guide for Unit Testing

As more and more developers embrace the joys of unit testing, I'm starting to see a lot more tests in code reviews, which is great to see! I am, however, seeing a lot of the same mistakes pop up.

To help with this, I wanted to use this blog post as a means to showcase a document I've been working on that outlines some best practices when writing unit tests in C#.

UPDATE: This document has been added to the Microsoft docs website (https://docs.microsoft.com/en-us/dotnet/core/testing/unit-testing-best-practices)

If you would like to contribute, the GitHub repo can be found here.

Happy unit testing!

Fakes, Mocks, Stubs, and Spies.. Oh My..

As I do more and more work with the exercism.io community, I've noticed a reoccurring theme. There seems to be a lot of confusion around the various types of "test doubles" and what each of them actually do.

Now, if you've had some exposure to test driven development already, you may already be familiar with test doubles such as fakes, stubs, spies, mocks, and dummies (did I miss any)? I feel like a lot of these terms are very similar and do not add much value, but rather just further complicate the discussion.

I prefer to simplify by classifying every test double as either a Fake, Stub, or Mock. I would even categorize a test double as a Fake. But what are Fakes, Mocks, and Stubs? I've taken a leaf out of Roy Osherove's book The Art of Unit Testing to define them as I believe they are the easiest to understand.

Fake - A fake is a generic term which can be used to describe either a Stub or a Mock object. Whether it is a Stub or a Mock depends on the context in which it's used. So in other words, a Fake can be a Stub or a Mock.

Mock - A mock object is a fake object in the system that decides whether or not a unit test has passed or failed. A Mock starts out as a Fake until it is asserted against.

Stub - A stub is a controllable replacement for an existing dependency (or collaborator) in the system. By using a stub, you can test your code without dealing with the dependency directly. By default, a fake starts out as a stub.

Now that we have a formal definition of each, there's a few key points I want to highlight.

When you say mock, you probably mean stub.

This is probably the most common mistake I see when performing code reviews of test suites and discussing tests with other developers.

Consider the following code snippet:

var mockOrder = new MockOrder();  
var sut = new Purchase(mockOrder);

sut.ValidateOrders();

Assert.True(sut.CanBeShipped); 

This would be an example of Mock being used improperly. In this case, it is a stub. We're just passing in the Order as a means to be able to instantiate Purchase (the system under test). The name MockOrder is also very misleading because again, the order is not a mock.

A better approach would be

var fakeOrder = new FakeOrder();  
var sut = new Purchase(fakeOrder);

sut.ValidateOrders();

Assert.True(sut.CanBeShipped); 

By renaming the class to FakeOrder, we've made the class a lot more generic, the class can be used as a mock or a stub. Whichever is better for the test case. In the above example, FakeOrder is used as a stub. We're not using the FakeOrder in any shape or form during the assert. We just passed it into the Purchase class to satisfy the requirements of the constructor.

To use it as a Mock, we could do something like this

var fakeOrder = new FakeOrder();  
var sut = new Purchase(fakeOrder);

sut.ValidateOrders();

Assert.True(fakeOrder.Validated);  

In this case, we are checking a property on the Fake (asserting against it), so in the above code snippet the fakeOrder is a Mock.

It's important to get this terminology correct. If you call your stubs "mocks", other developers are going to make false assumptions about your intent.

The main thing to remember about mocks versus stubs is that mocks are just like stubs, but you assert against the mock object, whereas you do not assert against a stub. Which means that only mocks can break your tests, not stubs.

When you say mocking framework, you probably mean isolation framework

In the same vein as "stubs are not mocks" I wanted to touch a little bit on the idea of mocking frameworks.

Unfortunately the term "Mocking framework", is confusing and ultimately incorrect. Take any framework you may consider a mocking framework (Moq, NSubstitute, etc), sure they can mock, but they can do so much more (e.g. they're also capable of stubbing).

The goal of these frameworks are to isolate your code. We should be calling them isolation frameworks. This even confused me for quite some time, having had some experience with Moq.

Consider the example

var mock = new Mock<IOrder>();  
var purchase = new Purchase(mock.Object);

Assert.True(purchase.Pending); 

Even though we called the variable mock and Moq provides a class called Mock, in this context, it's actually a stub.

So unfortunately, mock is an overloaded word. It's confusing. Avoid if it all possible, unless you actually are referring to mocking. Explaining to a developer that you can "isolate" something from its dependencies rather than "mock" its dependencies, feels more natural.

Key Takeaways

1) If you must roll your own class for isolation purposes, consider naming it a Fake. This gives you the freedom of using it as either a Mock or a Stub and does not give off the wrong intent.

2) Mocks are not Stubs. If you do not assert against it, it's a Stub.

3) Prefer the terminology Isolation framework. Mocking framework can be confusing.

DEVintersection 2017 Shenanigans

Kathleen Dollard in the house!

Well, that's a wrap! DEVintersection is over after a long week of sessions, technical difficulties, and an intense all-day workshop. I'm back in my hotel room at the MGM Grand, but we need to talk about everything that happened this week before I inevitably forget.

Docker all the things!

For starters, one thing is clear, Docker isn't going anywhere. Every speaker was leveraging Docker in some form in all of their sessions, even if the talk had nothing to do with it. Scott Hanselman gave one of the keynotes, Microsoft's open source journey, and he somehow managed to segway into demoing a raspberry pi cluster leveraging Kubernetes and Docker that hosted a .NET Core web application.

Coolness factor aside, it was actually pretty interesting to learn that .NET was actually "open sourced" a long time ago, the early 2000s I believe he said. Though it was just shared as a zip file and was intended for academia. The source code was sanitized as to not leak IP, and they refused to accept any contributions. Not that there were really any channels that would allow it. So in reality, it was more... "source opened".

So.. what exactly is DevOps?

After the keynote I attended a session called How DevOps Practices Can Make You a Better Developer and while the subject matter was pretty straightforward: defining what CI (continuous integration), CD (continuous delivery), etc were.. the biggest takeaway for me was Microsoft's definition of DevOps.

DevOps is the union of people, process, and products to enable continuous delivery of value to our end users.  Donovan Brown, Microsoft DevOps PM

It made a lot of sense. DevOps is about so much more than just delivery. The above definition really highlights how so many companies can get DevOps "teams" wrong. It isn't a team! It's combining process and people together in order to deliver software to our users. That means that DevOps is not only delivery, but infrastructure, testing, monitoring, and so much more.

Another common theme? Microservices!

Yep, on top of containerization, microservices was another big discussion point of the conference. Which in all honesty, makes sense. The two go pretty well together.

I attended a few sessions relating to microservices, and though they were a little introductory, there were definitely some highlights I felt jotting down. One of the sessions I went to was hosted by Dan Whalin, a pretty big name in the JavaScript world, and he outlined a quick list of four questions to ask yourself before taking the plunge into microservices.

  1. Are multiple teams working on different business areas of the application?
  2. Is it hard for new team members to understand the application and contribute quickly?
  3. Are there frequently changing business rules?
  4. Can subsystems within the application be changed with a minimal amount of impact?

I believe there's a lot of value in taking a step back and questioning whether microservices are right for you and your project. Like everything else in our field, there are no silver bullets. Microservices are a good solution to an array of problems, but it's not one size fits all. The additional complexity, especially in infrastructure, may not be worth the investment if you don't answer "yes" to all of the above, or at least some.

To top off my microservices immersion, I also attended a session on implementing a microservices messaging architecture, specifically with Azure Event Hubs. Now, the session was cut pretty short. The speaker's presentation apparently relied heavily on the Internet for some demos, which.. did not get up and running for probably a good 30 minutes. They even attempted to use their cell phone to tether a connection. A little unfortunate, but I think they made their point.

Their main points? When using messaging and events, events are facts. They happened! And, you shouldn't necessarily immediately jump into breaking up your database architecture into consistent and eventually consistent. Start with everything going to the same destination (consistent store), and if/when you need to scale, you can take an eventually consistent approach.

I promise there were other session topics..

As of late, I've been reading and writing more functional code. It's a completely different way of thinking, and there are a lot of great benefits to doing so. I saw a session entitled Functional Techniques for C# and had to give it a go.

There were a lot of defining of terms, which I am completely OK with. I seem to always forget the definitions of words, so getting refreshers on currying and high order functions was a nice introduction to the talk.

The biggest takeaway I had from the session was the idea of Functional Refactoring. Now, we've all seen Imperative Refactoring. It's the type of refactoring that most of us are familiar with. It's where we pull out some code and shove it into a method, then call that method instead of the code.

Here's a quick refresher..

public void PrintResponse(int choice)  
{
  var output = string.Empty;
  switch(choice)
  {
    case 1:
      output = "Hello";
      break;
    case 2:
      output = "Goodbye";
      break;
  }
  Console.WriteLine(output);
}

Now in this .. case .. PrintResponse has two responsibilities: it has to figure out what the response is and then it has to print it. Applying some inside-out refactoring, we get the following solution:

public void PrintResponse(int choice)  
{
  var output = GetResponse(choice);
  Console.WriteLine(input);
}

private string GetResponse(int choice)  
{
  var output = string.Empty;
  switch(choice)
  {
    case 1:
      output = "Hello";
      break;
    case 2:
      output = "Goodbye";
      break;
  }

  return output;
}

Now PrintResponse only has one job, to print the response. The job of getting the response has been abstracted away into a new private method called GetResponse

Functional refactoring is a little different, again, it's outside-in rather than inside-out as I just showed. Here's a simple example:

public void Save()  
{
  try
  {
    context.Save();
  }
  catch(Exception ex)
  {
    Logger.Log(ex);
  }
}

Applying functional refactoring..

public void Save()  
{
  Handling.TryWithLogging(() => context.Save());
}

public static void TryWithLogging(Func<DbContext> operation)  
{
  try
  {
    operation();
  }
  catch(Exception ex)
  {
    Logger.Log(ex);
  }
}

So as you can hopefully see, we've taken the outside code, the using block, and replaced it with a TryWithLogging higher-order function. With this approach, you can wrap using statements for calls to your database, wrap try and catch blocks to automatically log to the database on failure instead of copy pasting Logger.Log(ex); at the end of every catch.

Lastly an all day grind.. with C#!

I was excited about this workshop. I have honestly never done a workshop before at a conference, so I didn't really have any idea what to expect. All I knew was that it was going to be an all-day event, in a room, with the one and only Kathleen Dollard.

There was so much information in this workshop, it might even need to be a blog post all by itself, but I wanted to highlight some of the key points that I felt stood out and that I resonate with.

The first part of the workshop really focused on teams and learning. Improving yourself and others. She spoke to that research has shown that within a team, the who didn't actually matter, that is, the team's diversity. But rather, teams performed better if everyone spoke around the same amount and that there was psychological safety (no fear of belittlement during a review or just for speaking up). It's interesting to think about anyway, as I feel most of us are conditioned to believe that diversity plays a big role in the success of a given team.

She also spoke about cognitive dissonance. Which essentially states that you will take new input and associate it with your current perception. Seems obvious when it's put like that, but it makes a lot of sense and really shows why things such as code reviews are so important. We really need to be challenging what we know, day to day, so we can take that new information and apply it correctly.

For example, I once had this preconceived notion a long time ago that several meant seven in mathematics. This perception caused me to get stuck on a particular mathematics problem for quite some time because I just couldn't reason about how the problem could even be solved. The question didn't make any sense in the context of several being seven. The world was much more clear to me when I was corrected. This is why code reviews are so beneficial. If our perception is wrong, and we stay in our bubbles, we will continue to be wrong.

Now, all of that really has little to do with C#, but I found it immensely helpful as a software professional. The tips and tricks were great but mostly focused on functional techniques that I discussed previously, just in greater detail.

All good things must come to an end...

And with that, the conference was over. I got to spend a good 30 minutes talking with Kathleen after everyone else had left the workshop room about her new position at Microsoft and what they were up to. Most others had to catch flights or had other arrangements. I for some reason thought the red eye was a good idea and wouldn't be leaving until 10:00 that night..

Until the next conference!

Hacktoberfest is Coming!

photo from hacktoberfest.digitalocean.com

It's that time again! For those who don't know, Hacktoberfest is put on by DigitalOcean and GitHub as a means to celebrate open source software and to get more people involved in contributing to open source projects.

Getting Involved

So how can you get involved? Well, I'm going to be a little bias and suggest the Exercism C# repository. For this event, we have gone through and added a hacktoberfest label to issues that we felt would be perfect for Hacktoberfest. If you don't feel like working on any of those issues, no problem! We're always welcoming new ideas and/or things that we've simply missed. Just create an issue, and let's have a discussion about it.

Not interested in programming? Contributions do not have to be programming related. There are quite a few open issues that relate to documentation or DevOps. Whatever fits your skill sets, we'll find something for you.

Of course, we are just one repository out of many. Exercism itself has a variety of repositories to choose from, but any project that you can find on GitHub is free game.

If you're struggling to find projects to work on, there's an application which you can find here that lists projects that are looking for assistance during Hacktoberfest.

Did I Mention Free Stuff?

That's right! If you submit four pull requests during the event, Digital Ocean will hook you up with a free T-Shirt (the request does not even need to be merged). As long as the request was made between Oct 1 and Oct 31, it counts.

So make sure you sign up at their website and start making some contributions!

The Transformation Priority Premise

Recently I stumbled across a test driven development article that mentioned something I had not heard before. It's a premise that Uncle Bob came up with as a means to order the priority of the transformations you should apply when practicing test driven development. He called it the Transformation Priority Premise

I wrote a couple small programs using the premise, and really liked the concept he was trying to convey. Though in order to fully explain the premise, we should probably talk about test driven development itself.

So.. what is TDD?

Test Driven Development

TDD is a software development methodology that has three "laws" set forth by none other than Uncle Bob. They are as follows:

  1. You are not allowed to write any production code unless it is to make a failing unit test pass.
  2. You are not allowed to write any more of a unit test than is sufficient to fail, and compilation failures are failures.
  3. You are not allowed to write any more production code than is sufficient to pass the one failing unit test.

These three laws, if adhered to, force you into a cycle commonly referred to as red, green, refactor. Let's demonstrate cycle by writing our own program. First, we'll need some requirements.

This program will return four different responses, and the response will be based on what kind of sentence is used as input.

  1. If the input ends with a period, return "Ok."
  2. If the input ends with a question mark, return "No."
  3. If the input ends with an exclamation mark, return "Quiet!"
  4. If no punctuation is found, return "What?"

This program is based on an exercism exercise called Bob, which is actually based off of another exercise, Deaf Grandma.

So where to start? The test.

Before we write any production code (the code that will ultimately end up into the compiled binary) we need to first stand up a unit test. To start, we'll need to create our System Under Test (SUT).

[TestMethod]
public void Input_has_no_punctuation_response_says_what()  
{
    var sut = new Responder();
}

And not all that surprising, the compiler is already yelling at us.

The type or namespace name 'Responder' could not be found (are you missing a using directive or an assembly reference?)  

But that's ok! We're already abiding by the first law since we started with a unit test. The compilation error is also expected; the second law states that we can't write any more of the unit test than is sufficient to fail (and compilation errors are failures).

So let's switch context a little bit and start writing some production code.

public class Responder  
{
}

We're done! The unit test compiles and passes. The third law forbids us from writing any more production code.

At this point of our development cycle we have gone through red (unit test compilation error), green (adding the Responder class to make the test pass), and now we're onto refactoring. Heh well, in this case, there's not really anything we can refactor, so we can move on.

With one cycle completed, we start from the beginning again with red. Just like last time, we need to write some more code in our test case so that it fails.

We'll want a method on the Responder that can take an input of type string, and we know our first requirement is that if no punctuation is found the result of the method is "What?"

[TestMethod]
public void Input_has_no_punctuation_response_says_what()  
{
    var _sut = new Responder();
    Assert.AreEqual("What?", _sut.Response("Hello"));
}

Now we can go ahead and compile that...

'Responder' does not contain a definition for 'Response' and no extension method 'Response' accepting a first argument of type 'Responder' could be found (are you missing a using directive or an assembly reference?)  

Another compiler error. Let's go ahead and fix that up.

We know the compiler error stems from the fact that we never implemented a Response method on the Responder class, so that's pretty easy to implement. But what do we write inside of the method body? The answer may seem a little surprising.

public string Response(string input)  
{
    return "What?";
}

That's right. A constant string value of "What?". Once again, this is because of the third law. We cannot write any more production code than is sufficient to pass the one failing unit test. It may seem a little silly at first, but bear with me, it'll hopefully make a little more sense as we continue writing our program.

Alright, so we've tested the case of no punctuation. Let's move onto a case that includes punctuation, the period. Testing for that gives us a unit test that looks like this:

[TestMethod]
public void Input_is_statement_response_says_ok()  
{
    Assert.AreEqual("Ok.", _sut.Response("Do it now."));
}

Continuing with the red, green, refactor cycle, we now have a failing test. Let's go ahead and write the bare minimum implementation.

public string Response(string input)  
{
    if(input.EndsWith("."))
    {
        return "Ok.";
    }

    return "What?";
    }
}

Easy enough, time for another test.

[TestMethod]
public void Input_is_question_response_says_no()  
{
    Assert.AreEqual("No.", _sut.Response("Please?"));
}

Next up? You've got it. Let's make this test pass.

public string Response(string input)  
{
    if(input.EndsWith("."))
    {
        return "Ok.";
    }

    if (input.EndsWith("?"))
    {
        return "No.";
    }

    return "What?";
}

Now, when we make this test pass, we can see that there is some code duplication going on that we should probably refactor. After all, after making a test pass, we are given the opportunity to refactor the code. Unfortunately, it may not always be clear on how to refactor it. There is hope, however!

The Transformation Priority Premise

As stated in the introduction, The Transformation Priority Premise (TPP) is a premise that was put together as a means to prioritize the transformations that occur when getting unit tests to pass.

When you're practicing TDD you may ask: "How doesn't all code produced by using TDD, just result in code that is specifically tailored to pass the tests?"

You might notice that we're starting to see that a little in our current program. As it stands right now, we have one conditional per unit test. There's really nothing to stop this trend from occurring. There is, however, another little mantra that goes with TDD that pushes developers away from this practice.

“As the tests get more specific, the code gets more generic.”

Put another way: As we add more tests to our system (become more specific), our code becomes more generic (agnostic to the input).

With this in mind, it should be a little clearer to see that our current approach may not be the best one that we can take to solve this problem. We're just introducing more and more if statements to make the tests pass. Let's take a stab at refactoring our code and get away from our potential mountain of conditionals.

To start, the root of the TPP is its list of transformations and their priority. Here is the full list:

  1. ({}–>nil) no code at all->code that employs nil
  2. (nil->constant)
  3. (constant->constant+) a simple constant to a more complex constant
  4. (constant->scalar) replacing a constant with a variable or an argument
  5. (statement->statements) adding more unconditional statements.
  6. (unconditional->if) splitting the execution path
  7. (scalar->array)
  8. (array->container)
  9. (statement->recursion)
  10. (if->while)
  11. (expression->function) replacing an expression with a function or algorithm
  12. (variable->assignment) replacing the value of a variable.

.. and In case you've forgotten, this is the code we're trying to refactor.

public string Response(string input)  
{
    if(input.EndsWith("."))
    {
        return "Ok.";
    }

    if (input.EndsWith("?"))
    {
        return "No.";
    }

    return "What?";
}

Now we want to refactor this in order to get rid of the duplication. We started with a single constant, "What?" which was priority #3 and moved onto splitting the execution path, #6. It's time to consult the list and see what transformations we can make in order to clean up the if statements.

Being at #6 currently, the next logical step would be to take a look at #7, scalar to array. That could probably work, but given the context of this problem, we know it's a mapping issue. We're mapping punctuation to results. So let's take it one step further and leverage #8, array to container.

Note: The difference between an array and container is that an array is generally going to be a primitive array (think int[], string[], etc). Whereas a container is going to be something like a List, Set, or Dictionary.

Using scalar to array, and then array to container, we get a refactored method that looks like this:

public string Response(string input)  
{
    var inputResponses = new Dictionary<char, string>()
    {
        { '.', "Ok." },
        { '?', "No." }
    };

    if (inputResponses.ContainsKey(input.Last()))
    {
        return inputResponses[input.Last()];
    }

    return "What?";
}

That's pretty neat. No more repeating if statements. Recompile, ensure the tests still pass.. and they do! Now, there's only one punctuation mark that remains in our requirements, and that's the exclamation mark. We just finished refactoring, so we start again from red and introduce our last test:

[TestMethod]
public void Input_is_yelling_response_says_quiet()  
{
    Assert.AreEqual("Quiet!", _sut.Response("Woot!"));
}

Going back to our production code, it should be pretty straight forward as to how we can get this test to pass.

public string Response(string input)  
{
    var inputResponses = new Dictionary<char, string>()
    {
        { '.', "Ok." },
        { '?', "No." },
        { '!', "Quiet!" }
    };

    if (inputResponses.ContainsKey(input.Last()))
    {
        return inputResponses[input.Last()];
    }

    return "What?";
}

That's all there is to it! All of our tests pass, and we've met all of the requirements that we set out to meet.

The gain from leveraging the TPP is that it keeps us grounded, and forces us to continue to take baby steps when developing code. We generally do not want to take giant leaps forward. Start with repeating if statements over and over until something like a dictionary or a loop pops out at you.

If you're interested in learning more about all of the nuances of the Transformation Priority Premise, I highly recommend checking out Uncle Bob's initial blog post on the subject which can be found here.