Test doubles

“Test Double is a generic term for any case where you replace a production object for testing purposes”. This is how Martin Fowler describes it. They are mainly used in unit tests, when the unit must be tested in isolation.

A little background

If we are not testing the component in isolation, it’s hard to cover everything. The dependencies may throw exceptions and it’s difficult/impossible to control them. The indirect inputs that are coming from the dependencies can be unpredictable or may not even exists in the test environment. On the other side we must make sure that the indirect outputs of the test component have been verified and have produced the desired effect. Indirect inputs must be controlled and indirect outputs must be verified. That’s where test doubles come to play.

Test doubles types

Depending of its purpose the term test double can mean different things:

1. Dummy Object is an object that is used as an method argument, but has no impact on the actual testing process.

2. Test Stub replaces the real component so it’s possible to test the indirect inputs of the component that we are testing. The Spy is an extensions of the stub, in the sense that it can also verify the indirect outputs of the component under test.

3. Mock Object replaces the real component in order the test can verify it’s indirect outputs.

4. Fake is an object that replaces the functionality of a real object with an alternative implementation.

Dummy objects don’t do anything. Their sole purpose is to match the argument list of a method. They must be type compatible with the argument that is being passed. An important distinction between dummy and null objects is that dummy object is not being used whereas null object is being used but does nothing.

Stubs are objects that control the indirect input when its methods are being called. A stub can be a Responder used to inject indirect inputs via normal returns from method calls, or it can be a Saboteur used for raising errors or exceptions for injecting abnormal indirect inputs. Stubs use STATE verification.

A spy can be looked as a stub with added capabilities to also observe the indirect outputs using silent records of its method calls. A series of assertions can be made to check the expected calls. Spies add BEHAVIOR verification.

On the other hand, the mock can also check the indirect outputs but it differs from the spy through the fact that it compares the actual calls with an expected behavior which was previously defined. It may return information in response to method calls like the stub. Mocks use BEHAVIOR verification.

The difference between mocks and stubs is the fact that mocks always use behavior verification, while the stub can go either way. They both replace the real object right? Yes, but in a different way as you will see in the examples below.

So what’s the difference between mocks and spies? In the testing frameworks world a spy is seen as a partial mock (it uses a real object, but can control the response of its methods) while mock replaces the whole object. But wait, I can use a spy and replace all methods of the existing object? Well in that case the object is not “real” anymore and a mock is more appropriate.

A fake object differs from a stub or mock in the sense that it’s not controller nor observed by the test. There is no verification on it, usually it simplifies the real object due to reasons like it’s not available in the test environment or it’s too expensive to build.

The test doubles can be hand-build, or there can be dynamically generated(using an existing framework). In this post I will use Mockito, one of the most popular mocking framework.

Ok. Some examples will shed some light on this.

public class QuoteCentralTestWithoutMocks {

    private final QuotesCentral quotesCentral = new QuotesCentral();

    //This is a spy and also a stub
    private class SpyMessageServer extends MessageServer {
        int methodCalls = 0;

        public int getNumberOfCalls() {
            return methodCalls;
        }

        @Override
        public boolean publish(List quotes) {
            System.out.println("stubbed publish");
            methodCalls++;
            return true;
        }

        @Override
        public void connect() {
            System.out.println("stubbed connect");
            methodCalls++;
        }

        @Override
        public void disconnect() {
            System.out.println("stubbed disconnect");
            methodCalls++;
        }
    }

    //This is a Fake
    private class FakeQuoteProvider extends QuoteProvider {

        @Override
        public List generateQuotes() {
            return QuoteGenerator.QUOTES;
        }
    }

    private SpyMessageServer messageServer = new SpyMessageServer();
    private FakeQuoteProvider quoteProvider = new FakeQuoteProvider();

    @Before
    public void setup() {
        quotesCentral.setMessageServer(messageServer);
        quotesCentral.setQuoteProvider(quoteProvider);
    }

    @Test
    public void testFindByAuthor() {
        String author = "Mark Twain";
        List result = quotesCentral.findByAuthor(author);
        assertThat(result).as("There should be 2 quotes").hasSize(2);
        assertThat(result).as("The author should be " + author)
                  .extracting("author")
                  .allMatch(s -> s.equals(author));
    }

    @Test
    public void testPublishQuotesByAuthor() {
        String author = "Mark Twain";
        boolean result = quotesCentral.publishQuotesByAuthor(author);
        assertThat(result).as("Quotes should be published").isTrue();
        assertThat(messageServer.getNumberOfCalls()).as("Should be 3 calls").isEqualTo(3);
    }
}

SpyMessageServer is a stub and a spy. It’s a stub because it overrides the default behavior of the original class and the response of the of the publish method. It uses state verification

assertThat(result).as("Quotes should be published").isTrue();

It’s a spy because it verifies the indirect outputs using silent calls

assertThat(messageServer.getNumberOfCalls()).as("Should be 3 calls").isEqualTo(3);

FakeQuoteProvider it’s a fake. It simplifies the functionality of the original class (which reads the quotes from a file)  and it’s nor used or verified in the test.

Let’s see the same thing with mocks.

public class QuoteCentralTest {

    private final QuotesCentral quotesCentral = new QuotesCentral();

    @Mock
    private MessageServer messageServer;
    @Mock
    private QuoteProvider quoteProvider;

    @Rule
    public MockitoRule mockito = MockitoJUnit.rule();

    @Before
    public void setup() {
        quotesCentral.setMessageServer(messageServer);
        quotesCentral.setQuoteProvider(quoteProvider);
        //stub method calls
        Mockito.when(quoteProvider.generateQuotes()).thenReturn(QuoteGenerator.QUOTES);
    }

    @Test
    public void testFindByAuthor() {
        String author = "Mark Twain";
        List result = quotesCentral.findByAuthor(author);
        assertThat(result).as("There should be 2 quotes").hasSize(2);
        assertThat(result).as("The author should be " + author)
                  .extracting("author")
                  .allMatch(s -> s.equals(author));
        //verify interactions
        Mockito.verify(quoteProvider).generateQuotes();
    }

    @Test
    public void testPublishQuotesByAuthor() {
        ArgumentCaptor quotesCaptor = ArgumentCaptor.forClass(List.class);
        String author = "Mark Twain";

        //stub method calls
        Mockito.doNothing().when(messageServer).connect();
        Mockito.doNothing().when(messageServer).disconnect();
        Mockito.when(messageServer.publish(any(List.class))).thenReturn(true);

        quotesCentral.publishQuotesByAuthor(author);

        //verify interactions
        Mockito.verify(messageServer).connect();
        Mockito.verify(messageServer).publish(quotesCaptor.capture());
        assertThat(quotesCaptor.getValue()).as("There should be 2 quotes").hasSize(2);
        assertThat(quotesCaptor.getValue()).as("The author should be " + author)
                  .extracting("author")
                  .allMatch(s -> s.equals(author));
        Mockito.verify(messageServer).disconnect();
    }
}

As you can see there is an expected behavior predefined, and the verification it’s more complex. This is behavior verification.

Tips

Don’t mock everything. It will impact the performance of your tests. It will take more time to do a complete run if you have a high number of mocks. Not only that, but if we are mocking everything then what are we actually testing?

Don’t mock value objects(like models). Are you too lazy to build them? You can create builders in the test package for them. Or even a rule. The tests should be easy to read and verbose methods like buildX() do not help. Don’t be afraid to write helper classes.

Refactor your tests to be simple and easy to understand. Consider creating custom assertions if things become too verbose. Consider using custom conditions, argument matchers.

Don’t mock types you don’t own. There are many opinions on this, as it is described more a suggestion than a rule. The most obvious example here is mocking a 3rd party library. Like any library it gets new versions and if you mock something from it, the tests may pass and the real problem may be revealed in production. That it unless you have integration tests. If you need to mock something like this than you may have a different problem in your code, maybe it isn’t decoupled enough.

SOURCE CODE