Mutation Testing: Make your tests bulletproof - 1

Mutation Testing is "testing the tests" by modifying or mutating the actual code to understand if existing tests fail.

Before we understand what Mutation testing is, we need to understand what the code coverage is.

When someone says "I follow minimum 100% of code coverage through test" or "My code is 100% covered through test". What does he really mean?

Does it mean his\her 100% of code is robust to any modification in future?

Unfortunately not.

You can have 100% code coverage through tests and still your tests would not be perfect.

So the question should be, how well that 100% of code is covered?

Tests are our safety nets. We can do bigger refactoring or add new features with more confidence if we have better safety nets. Same as it happens for the performers who are walking on a rope. They can take bigger risks if they know there is a safety net below to protect them.

Let's say you have code which is covered 100% through tests. Now you change some business logic of the code. Two things can happen now:

• Either your existing tests fails. (i.e. Your test is perfect to detect the change) OR
• Your existing tests still passes. (i.e. Your test is not well written to detect this change.)

Mutation testing does the same thing:

Mutation testing modifies the actual piece of code (I meant compiled version of your code) and runs existing tests to see if they detect the change and fails.

So, How does it work?

1. Modify existing code to create different versions to it. (Creating The Mutants)
2. Running existing tests against new versions of code (Killing Mutants)
3. If test fails, means your test was perfect (Mutant Killed)
4. If test passes, means your test was not perfect (Mutants Survived)

So our ideal situation would be to keep modifying tests until all the "Mutants are killed" and then only we can trust our safety nets. 🤩🤩

An Example:

The snippet below is a registration service for Provident Fund for employees.

Let's say you write a test which covers 100% of above code and your coverage report also validates the same.

Below is the code coverage report for above test.

In the next section of this blog I will run this code through Mutation testing and show you why these tests are not perfect and how we can improve it.

Stick with me! See you in next Section >>>>>>>>>>>>>>>>