What is testing? What is JUnit?

JUnit is the most popular testing framework for Java. It is used for writing and running automated and repeatable unit tests.

The fact that they are repeatable is important! You don’t want to run your test cases only once – you will want to run them every time you change your program, to check if the new changes broke anything.

Unit testing means testing the smaller units of your program, like classes and methods.

The goal of unit tests is to validate that each unit of the software performs as expected. Executing a test can have two outcomes:

• The test passes (green 🟩) if the behaviour of the unit under test is the expected one.

• The test fails (red 🟥) if the behaviour of the unit under test is not the expected one.

If a test fails, and the test is correct, it means that we have found a software error (software bug 🐞). We can then use the failing test(s) to debug the program and fix our program. Then, we can re-run all the tests to ensure that all the existing tests are still passing.

How to write JUnit test cases?

Consider the following MathOperations class that defines a max function:

package nz.ac.auckland.se281;

public class MathOperations {
    
    public int max(int a, int b) {
       if(a >= b) {
         return a;
       }
       return b;                             
    }
    
}

Let’s write a test for it.

First of all, we need to create a public Java class that will contain our JUnit test cases. It is a standard practice to separate the Java classes that implement the functionalities of your program from the test classes. For this reason, in Java, the convention is to create a special source folder called test, and put the source code of all the test classes inside. Another common convention is to call this class *Test, where * is the name of the class under test. It is important that this class is under the same package of the class under test so that it can access protected methods (if any).

In our case we will have to create the following class:

package nz.ac.auckland.se281;

public class MathOperationsTest {
    
}

To define a test case, we need to use the JUnit annotation @Test on top of a public void method. In this way, we are telling JUnit that the method is a test case. Test is a class in JUnit that has to be imported like this: import org.junit.Test;. If you don’t put the annotation @Test, the test will not be executed!

Let’s write our first test case:

Most unit tests follow the same template, which is composed of four parts:

• input, which represents the input(s) of the method under test. In our test we have two inputs: int a = 10; int b = 5.

• expected output is the output that we expect from calling the method under test with the specified input. For our method under test max(), the maximum between 10 and 5 should be 10.

• actual output is the output returned by invoking the method under test with the specified input.

• assertion oracle is a special JUnit method that decides if the test has passed or failed. In this case, the assertion oracle checks if the expected output is equal to the actual output. If they are equal the test passes 🟩, otherwise it fails 🟥. In our example, the test case passes because the actual value is 10 (as expected).

We wrote the test case in a verbose way to distinguish the four distinct parts. However, it is better to write test cases in a more compact way:

import org.junit.Test;
import static org.junit.Assert.assertEquals;

public class MathOperationsTest {

    @Test
    public void testMax() {
        MathOperations math = new MathOperations();
        assertEquals(10, math.max(10, 5));
    }
}

Note that we need the import declaration import static org.junit.Assert.assertEquals; to import the static method assertEquals().

The interesting thing about assertEquals(), is that it automatically relies on the equals() method of the objects being compared.

Another important thing to know, is that when comparing double you must specify a delta.

For example:

@Test
public void test(){
    double a = 0.45555;
    double b = 0.455556;
    
    assertEquals(a, b, 0.001);
}

In this case, the delta is 0.001. This specifies the maximum delta between expected and actual for which both numbers are still considered equal. For example, the above test passes even though the two number are slightly different, because their difference is within the delta = 0.001 amount. What JUnit does internally is:

if(Math.abs(expected - actual) < delta) {
    //the test fails
} else {
    // the test passes
}

If we set a lower delta (e.g., 0.000000001) the test fails:

@Test
public void test(){
    double a = 0.45555;
    double b = 0.455556;
    
    assertEquals(a, b, 0.000000001);
}

public int max(int a, int b, int c) {
    int maxBetweenAandB = max(a, b);
    return max(maxBetweenAandB, c);
}

More practice

Let’s consider this buggy Java class:

import java.util.ArrayList;
import java.util.List;

public class Numbers {

    private final List<Integer> numbers;

    public Numbers() {
        numbers = new ArrayList<>();
    }

    public List<Integer> getNumbers() {
        return numbers;
    }

    /**
     * BUGGY METHOD
     *
     * add the number n only if it is not already inside the list of numbers.
     * If the number is added returns true, returns false otherwise.
     *
     * @param n
     * @return
     */
    public boolean addNumber(int n) {
        if (numbers.contains(n)) {
            numbers.add(n);
            return true;
        }
        return false;
    }
}

The Numbers class has one field numbers, which is a list of integers. It also declares the method boolean addNumber(int n), which should add the number n into the list numbers only if the list does not already contain n. In other words, numbers should not contain repeated numbers. The method returns true if the number is added, false otherwise.

The class contains an obvious bug. Let’s expose the bug by writing and running tests.

Let’s write a test case for the method addNumber() in the classNumbersTest.

import org.junit.Test;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;

public class NumbersTest {

    @Test
    public void test_addNumber_addOneNumber_ShouldBeAdded() {
        Numbers nums = new Numbers();
        boolean result = nums.addNumber(10);
        assertTrue("the result should be true, because the number 10 should be added", result);
        assertFalse("the list should not be empty", nums.getNumbers().isEmpty());
        assertEquals("the first element of the list should be 10", 10, (int) nums.getNumbers().get(0));
    }
}

There are a few important things about this test that we should highlight:

• There are several naming conventions for JUnit test cases. The one that the test is using is test_MethodName_StateUnderTest_ExpectedBehavior.

• A test can contain multiple assertions (in our case we have three assertions). The test passes if all of its assertions pass, and it fails if at least one fails. However, it is very important that these assertions are related, meaning that if a test fails should fail for the same reason! Do not create huge test cases with many unrelated assertions. In the example above, all the assertions are checking the same thing but in different ways.

• We added a message to each assertion method. The message should describe the expected behaviour. This is useful because when a test fails, JUnit automatically prints the message. Such messages help the developers to understand why the test(s) failed.

• JUnit gives 8 different types of assertion oracles. For instance, the test case is using assertFalse(b), which makes the test pass if the boolean condition b is false. It makes the test fail if b is true. You should avoid to write something like assertTrue(!nums.getNumbers().isEmpty());, which is equivalent to assertFalse(nums.getNumbers().isEmpty()); but much harder to read (because of the negation).

This is a complete list of assertion methods of JUnit:

The test case fails 🟥, with the following message:

java.lang.AssertionError: the result should be true because the number 10 should be added

	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at testNumbers.NumbersTest.test_addNumber_addOneNumber_ShouldBeAdded(NumbersTest.java:18)

JUnit will show the message of the first failing assertion, regardless of whether the remaining assertions also fail. JUnit will also show the test name and line number of the failing assertion.

After debugging our code we will easily find the bug 🐞! We forgot to negate the if condition of the method addNumber:

if (numbers.contains(n)) {

should become:

if (!numbers.contains(n)) {

Now, the test will pass :)

Additional annotations

Besides @Test, Junit offers additional annotations:

One very useful annotation is @Before that allows you to specify that a certain method should be executed before each test execution. For example, instead of instantiating the object under test at the beginning of each test case, we could do like this:

import org.junit.Before;
import org.junit.Test;

public class NumbersTest {

    private  Numbers nums;
    
    @Before
    public void setup() {
        nums = new Numbers();
    }
    
    @Test
    public void test1() {
        boolean result = nums.addNumber(10);
        // rest of the test
    }

    @Test
    public void test2() {
        boolean result = nums.addNumber(5);
        // rest of the test
    }

    @Test
    public void test3() {
        boolean result = nums.addNumber(4);
        // rest of the test
    }
}

If there are 10 tests, the method setUp() will be executed 10 times before each test case. Generally, this is very useful to make sure that each test is running in a clean state. An important principle of unit testing is that each test should be able to run independently from other tests.

JUnit tests and exceptions

Let’s consider this Java class:

import java.util.ArrayList;
import java.util.List;
import java.util.NoSuchElementException;

public class MyList {

    List<Integer> list;

    public MyList() {
        list = new ArrayList<>();
    }

    public void addElement(int n) {
        list.add(n);
    }

    public int getFirstElement() {
        if (list.isEmpty()) {
            throw new NoSuchElementException();
        }
        return list.get(0);
    }

}

How to test that the method getFirstElement() correctly throws the exception?

We can do this as follows:

import org.junit.Test;

import java.util.NoSuchElementException;

import static org.junit.Assert.*;

public class MyListTest {

    @Test
    public void test_getFirstElement_emptyList_shouldThrowException() {
        MyList list = new MyList();
        try {
            list.getFirstElement();
            fail("it should throw exception");
        } catch(NoSuchElementException e) {
        }
    }

    @Test
    public void test_getFirstElement_addOneElement_shouldNotThrowException() {
        MyList list = new MyList();
        list.addElement(10);
        try {
            list.getFirstElement();
        } catch(NoSuchElementException e) {
            fail("it should not throw an exception");
        }
    }

}

The JUnit method fail() makes the test fail if it is encountered. In other words, if the execution flow reaches an invocation to fail(), the test fails.

In the first test case, we expect that the method getFirstElement() throws the exception because the list is empty. If the method reaches fail("it should throw exception"); it means that the exception was not thrown and thus the test should fail.

Conversely, in the second test case, we expect that the method getFirstElement() does not throw the exception, because we added an element before invoking it. So, we put fail("it should not throw an exception"); inside the catch block. As such, if the method wrongly throws the exception the test should fail.

Alternatively, we could have written the first test using the annotation @Test(expected=Exception.class) as follows:

public class MyListTest {

    @Test(expected=NoSuchElementException.class)
    public void test_getFirstElement_emptyList_shouldThrowException(){
        MyList list = new MyList();
        list.getFirstElement();
    }

}

The test fails if the exception NoSuchElementException.class is not thrown.

import java.util.ArrayList;
import java.util.List;

public class MaxNumbers {

    private final List<Integer> numbers;

    public MaxNumbers() {
        numbers = new ArrayList<>();
    }

    public List<Integer> getNumbers() {
        return numbers;
    }

    /**
     * add the number n only if it is not inside the list numbers.
     * If the number is added, it returns true, returns false otherwise.
     *
     * @param n
     * @return
     */
    public boolean addNumber(int n) {
        if (!numbers.contains(n)) {
            numbers.add(n);
            return true;
        }
        return false;
    }

    /**
     * BUGGY METHOD
     *
     * @return the maximum element in the list
     */
    public int getMax() {
        if (numbers.isEmpty()) {
            throw new CannotComputeTheMaximumListEmpty();
        }
        
        int max = numbers.get(0);
        
        for (int i = 0; i < numbers.size() - 1; i++) {
            if (numbers.get(i) > max) {
                max = numbers.get(i);
            }
        }
        
        return max;
    }
}