Mutation Testing: Make your tests bulletproof - 2

Welcome to the continuation of our last post on mutation testing. In part 1 of this post we saw what mutation testing is all about and understood that a test which covers 100% of your code is actually may not be robust enough. In this part we will run mutation test on an existing test with 100% coverage and track down the missing tests to make it perfect and robust.

An Example

The snippet below is a registration service for Provident Fund for employees.

In part 1 of this article we wrote a test which covers 100% of lines of above code.

Now we will be running mutation tests against our code. For running mutation tests we will be using open source library for Java called PITEST.

Pitest

PIT is a state of the art mutation testing system, providing gold standard test coverage for Java and the jvm. It's fast, scalable and integrates with modern test and build tooling.

pitest Integration

We are using spring boot with maven for our example. In order to integrate pitest in our application we need to add pit plugins in pom.xml.

• pitest-junit5-plugin
• pitest-maven

That's all the configuration is needed. Now when we build our application using command mvn clean install, the pitest plugin modifies (mutates) our compiled code and runs the existing tests against those modified codebase (mutants).

Let's look what all possible mutations can be done here:

An == operator can be converted to !=.

A <= operator can be converted to >.

A statement returning as boolean can simply be replaced by true or false.

A method returning a value will be changed to return void.

... ...

and there are many such mutations. (Full list can be found here)

In our example below can be possible mutations when mvn clean install is run.

Let's run the build using mvn clean install. A pit-report will be generated in target folder.

hmmm... some of the mutants are not killed by our tests!!

What does it mean? It means we need to write more tests or modify existing tests so that these mutants gets killed in our next run and we find a perfect, robust test for our application 😀

Let's write additional tests to cover those missing scenarios.

Looks like we covered all missing test cases. hmmm.. time to run the build again.

Let's run the build using mvn clean install. A pit-report will be generated again in target folder.

viola.. 👏👏

Mutation testing can slowdown you build process. So, configure it to run only for domain. You can easily configure the desired package which you want to test using mutation testing, when you add plugin in pom.xml.

<configuration>
          <targetClasses>
            <param>in.anupbaranwal.mutation.service.*</param>
          </targetClasses>
      <targetTests>
            <param>in.anupbaranwal.mutation.service.*</param>
          </targetTests>
    </configuration>

I hope this article was helpful to you in basic understanding of mutation testing. If you have any doubts, Let me know in the comments below. You can always reach out to me on My Email Id !!! 😀