Are you verifying your stubs?

What is stubbing? What is verifying? Do we need both?

If we look into the java docs for Mockito.when() (stubbing) and Mockito.verify() (verifying behavior) we see something strange that is common to both:

Although it is possible to verify a stubbed invocation, usually it's just redundant.

Well, we known that the difference between stubs and mocks is that one cares about the state and the other about the behavior. Now in mockito when stubbing you get automatically verify as well by the fact that the stub is called.

So all this time I was duplicating test logic? Apparently yes.  Well, why there is still verify() method available? Examples will shed some light.

Example 1

@Test
void testQuotesShouldBePublished1() {
    ArgumentCaptor quotesCaptor = ArgumentCaptor.forClass(List.class);
    String author = "Mark Twain";
    //stub method calls
    doNothing().when(messageServer).connect();
    doNothing().when(messageServer).disconnect();
    when(messageServer.publish(any(List.class))).thenReturn(true);

    assertThat(quotesCentral.publishQuotesByAuthor(author)).isTrue();

    //verify interactions
    verify(messageServer).connect();
    verify(messageServer).publish(quotesCaptor.capture());
    assertThat(quotesCaptor.getValue()).as("There should be 2 quotes").hasSize(2);
    assertThat(quotesCaptor.getValue()).as("The author should be " + author).extracting("author").allMatch(s -> s.equals(author));
    verify(messageServer).disconnect();
}

In the above example, we are stubbing everything but a clear indicator that we don’t need to do that is doNothing().

 //stub method calls 
 doNothing().when(messageServer).connect();
 doNothing().when(messageServer).disconnect();

Example 2

@Test
void testQuotesShouldBePublished2() {
    ArgumentCaptor quotesCaptor = ArgumentCaptor.forClass(List.class);
    String author = "Mark Twain";

    when(messageServer.publish(any(List.class))).thenReturn(true);

    assertThat(quotesCentral.publishQuotesByAuthor(author)).isTrue();
    verify(messageServer).publish(quotesCaptor.capture());
    assertThat(quotesCaptor.getValue()).as("There should be 2 quotes").hasSize(2);
    assertThat(quotesCaptor.getValue()).as("The author should be " + author).extracting("author").allMatch(s -> s.equals(author));

    //verify interactions
    verify(messageServer).connect();
    verify(messageServer).disconnect();
}

If the mocked method has no impact on the logic do not mock it. Verify is appropriate. We eliminated two lines of code without any impact on the test. Let’s further examine it. If we turn our attention to the verify with argument captor

 verify(messageServer).publish(quotesCaptor.capture());

we can see that this is an extra check as we make sure that the stub was correctly called. We could have skipped this line of code and the test will still have worked. That’s another indicator that we need to do something here. We still want this extra check and that leads to

Example 3

@Test
void testQuotesShouldBePublished3() {
    String author = "Mark Twain";
    List markTwainQuotes = quotesCentral.findByAuthor(author);

    when(messageServer.publish(markTwainQuotes)).thenReturn(true);

    assertThat(quotesCentral.publishQuotesByAuthor(author)).isTrue();
    //verify interactions
    verify(messageServer).connect();
    verify(messageServer).disconnect();
}

We have access to the input parameter (markTwainQuotes ). If we put it as input parameter for the stub then we can get rid of the verify with argument captor. If the stub invocation does not meet this parameter(it uses equals for comparing) it won’t stub and the test could fail. If you stub with Mockito.when() then it’s implicitly verified by the fact that the stubbed value is needed in further processing.  If you don’t have access to the input parameter ArgumentMatcher.any(Class<T> type) can help you, but you will lose some depth in testing. You don’t know if the method is called with the exact parameter we expect.  But that’s ok, sometimes it is not justifiable or not possible to get it. And do you even care? Do you care about the input and/or output? Decide this for yourself.

So no extra verifying needed. We have even less lines of code without impact on the test.

Clear. Then it comes this

Example 4

@Test
void testQuotesShouldBePublished4() {
    when(messageServer.isAuthorPublished("Mark Twain")).thenReturn(true);
    assertThat(quotesCentral.isAuthorPublished("Mark Twein")).isTrue();  
}

This test is passing. All good. We created the stub with input parameter. We assumed that this is called and verified as we are using the stub result in further processing.

public boolean isAuthorPublished(String name) {
    return messageServer.isAuthorPublished(name) ? true : true;
}

Well no. This isn’t good. We have two (obviously) issues here. The test contains an error.

"Mark Twain".equals("Mark Twein")  == false;

That means the stub was not invoked and yet the test is passing. Well, if we have used verify

Example 5

void testQuotesShouldBePublished4() {
    when(messageServer.isAuthorPublished("Mark Twain")).thenReturn(true);
    assertThat(quotesCentral.isAuthorPublished("Mark Twein")).isTrue();

    //verify interactions
    verify(messageServer).isAuthorPublished("Mark Twain");
}

immediately we get notified that the arguments do not match. This means we have a problem in the code we try to test.

return messageServer.isAuthorPublished(name) ? true : true;

Instead of going on the if branch it goes on the else branch. Obviously this is a stretch. Some example of bad code. We could have written

return messageServer.isAuthorPublished(name);

an not have this problem. This is a super edge case, if you want. But we followed the “rules” described above and got burned(ourselves). It’s our fault, but if we have used stubbing with explicit verify we would have seen it immediately and not hours/days later.

So? Which we should use? Mockito.when() or Mockito.verify()?

As a conclusion, do one or another depending of the situation. Verify works when you need to check the number of invocations. Situations like the latter are extreme. If you do both expect extra lines of code. It’s not that bad, when you have few tests(you should not have this). It becomes bad when the number of lines grow significantly. A great article on this here.

SOURCE CODE