Lab: Skills - JUnit and JaCoCo in VSCode

Instructions: Answer the following questions one at a time. After answering each question, check your answer (by clicking on the check-mark icon if it is available) before proceeding to the next question.

Getting Ready: Before going any further, you should:

Download the following files:

GiftCard.java

to an appropriate directory/folder (e.g., the course downloads directory/folder). In most browsers/OSs, the easiest way to do this is by right-clicking/control-clicking on each of the links above and then selecting Save as... or Save link as....
If you don't already have one from earlier in the semester, create a directory/folder named skills under the src directory/folder.
Outside of VSCode, copy the file GiftCard.java into that directory.
Start VSCode and Open GiftCard.java.

1. JUnit Basics: JUnit is is an open-source testing framework. It provides a way to write, organize, and run repeatable test. This part of the lab will help you become familiar with JUnit. (Help is also available on the CS wiki.)

Create an empty JUnit test named GiftCardTest.java in the apropriate directory/folder by right-clicking on the directory/folder, pulling down to New File..., and entering the file name in the textfield.
Copy the following code into GiftCardTest.java , after the package statement but before the class declaration.
```
import static org.junit.jupiter.api.Assertions.*;

import org.junit.jupiter.api.Test;
```
What is the purpose of these statements?
.

It makes the code in the JUnit library available for use.

Copy the following code into GiftCardTest.java, after the declaration of the class.

    @Test
    public void getIssuingStoreTest() {
        double       balance;        
        GiftCard     card;
        int          issuingStore;
        
        issuingStore = 1337;
        balance      = 100.00;
        card = new GiftCard(issuingStore, balance);

        assertEquals(issuingStore, card.getIssuingStore(), "getIssuingStore()");
    }

A JUnit test suite is a class, much like any other class. Tests are methods that are preceded with the annotation @Test. (Note: An annotation provides information about a program but is not part of the program. Annotations have no effect on the operation of the program. Instead, they are used to provide information to tools that might use the program as input.)
How many tests are in the test suite GiftCardTest?

.
One, the getIssuingStoreTest() method.
JUnit has an Assert class that has a static assertEquals() method with the following signature that is used to compare expected and actual results:
public static void assertEquals(int expected, int actual, String description)
where description is a human-readable String describing the test, expected is the expected result, and actual is the result of actually running the code being tested.

How would you call this method and pass it the String "getIssuingStore()", the int issuingStore, and the int returned by the card object's getIssuingStore() method?

.
Assert.assertEquals(issuingStore, card.getIssuingStore(), "getIssuingStore()");

How is this method actually being called in GiftCardTest?


                                            assertEquals(issuingStore, card.getIssuingStore(), "getIssuingStore()");

Why isn't the class name needed?
.

If you look at the top of the class you will see that the import statement for org.junit.jupiter.api.Assertions.* has a static modifier. This tells the compiler that the class name can be omitted from calls to static methods in this package.
This feature should be used sparingly because it makes code very difficult to read. In this case, writing Assert.assertEquals() seems redundant, so people frequently use a static import.
Click on the icon in the left-side toolbar. What happens?
.

The explorer view goes away and the testing view appears.
Expand the skills folder by clicking on the greater than sign that is next to it in the testing view. What happens?
.

GiftCardTest appears.
You can execute all of the tests in a directory/folder by hovering the cursor over its name in the testing view and clicking on the icon.
You can execute all of the tests in a file in two ways. (1) You can hover the cursor over its name in the testing view and clicking on the icon. (2) You can hover the cursor over the class declaration and click on the icon.

You can execute one test by hovering the cursor over the method declaration and clicking on the icon.
Execute all of the tests in GiftCardTest.java. Why is it somewhat surprising that you can execute GiftCardTest.java?
.

Because GiftCardTest doesn't have a main() method, and, in the past, we've only been able to execute classes that have a main() method.
In fact, JUnit has a class (called org.junit.runner.JUnitCore with a main() method and that class is what is being executed. It then calls the various tests in GiftCardTest. The class with the main() method is being hidden from us by VSCode to make our lives easier.
What output was generated?
.

None, but the test results appeared in the test results view (in the panel) that includes a summary on the right side. All of the tests are listed with a green checkmark, indicating that that all of the tests passed.
To see what happens when a test fails, modify the getIssuingStore() method in the GiftCard class so that it returns issuingStore + 1, save GiftCard.java, and re-run the test suite.
Now what happens?

.

Several things!!
In the test results view, interpret the line:
```
java.lang.AssertionError: getIssuingStore() expected:<1337> but was:<1338>
```
Note: You may have to scroll the test results view are to see the whole message.

.

It says that when the test described as "getIssuingStore()" was run, the result returned should have been 1337 but was 1338.
What mechanism is JUnit using to indicate an abnormal return?
.

It's throwing an exception (in this case, an AssertionFailedError).
Where else does VSCode indicate that a test failed?
.

In GiftCardTest.java a red X next to the class declaration indicates that a test in the file failed, a red X next to the method declaration indicates which test failed, and an editor decoration is added under the assert___() method indicates where the exception was thrown and what the expected and actual values are.
In the summary on the right side of the test results view, the same information is presented in condensed form.
Before you forget, correct the fault in the getIssuingStore() method.
The Assert class in JUnit also has a static assertEquals() method with the following signature:
public static void assertEquals(double expected, double actual, double tolerance, String description)
where tolerance determines how close two double values have to be in order to be considered "approximately equal".

Add a test named getBalanceTest() that includes a call to assertEquals() that can be used to test the getBalance() method in the card class (with a tolerance of 0.001).

.
@Test public void getBalanceTest() { double balance; GiftCard card; int issuingStore; issuingStore = 1337; balance = 100.00; card = new GiftCard(issuingStore, balance); assertEquals(balance, card.getBalance(), 0.001, "getBalance()"); }
How many tests are in your test suite now?
.

2
Suppose you had put both calls to assertEquals() in one method (named, say, getIssuingStoreTest()). How many tests would be in your test suite?
.

Unfortunately, different people/tools use different terminology. In JUnit parlance, there is only one test, corresponding to the method getIssuingStoreTest(). In our parlance, a set of inputs and expected outputs (which might require several assertions) defines a test. Nothing in JUnit prevents someone from putting more than one test (in our parlance) into one @Test-annotated method. This is one of the reasons it can be a little difficult to interpret the output from testing tools.
Suppose, for example, you have two @Test-annotated methods, one with 99 assertions and the other with 1. Whether 1 of the 99 assertions fails or 98 out of the 99 assertions fails, JUnit will report that you failed one of two tests (i.e., 50% of the tests).

So, unless you have a good reason to do so, you should not include multiple tests inside of an @Test-annotated method.
Execute the entire test suite (i.e., all of the tests in the directory/folder). How many tests were run?
.

The right side of the test results view shows that two tests were executed and that they both passed.

The Assert class in JUnit also has a static assertEquals() method with the following signature:


    public static void assertEquals(String expected, String  actual, String description)

Using JUnit terminology, add a test named deductTest_Balance() to your test suite that can be used to test the deduct() method in the GiftCard class. Note: Be careful, the variables that are declared in the getIssuingStore() method are local.


                                        @Test
    public void deductTest_Balance() {
        double       balance;        
        GiftCard     card;
        int          issuingStore;
        String       s;
        
        
        issuingStore = 1337;
        balance      = 100.00;
        card = new GiftCard(issuingStore, balance);

        s = "Remaining Balance: " + String.format("%6.2f", 90.00);
        assertEquals(s, card.deduct(10.0), "deduct(10.00)");
    }

Execute the entire test suite.

2. Coverage: This part of the lab will help you understand coverage tools and coverage metrics.

There is a library called JaCoCo that provides coverage metrics for Java tests.
To run GiftCardTest using JaCoCo, first click on the checkbox in the status bar (at the bottom of the VSCode window) and re-execute all of the tests in GiftCardTest.java. (Note: DO NOT click on "Run Test With Coverage", click on "Run Test".)
Why is the coverage of GiftCardTest.java uninteresting/unimportant?

.

This just means that all of the tests were executed. That isn't at all surprising.
How many of the tests passed?
.

All three of them.
Does this mean that the GiftCard class is correct?
.

I hope so, but I'm guessing from the tone of the question that there may not be enough tests.
Select the tab containing the source code for GiftCard.java. What is different about it?
.

Statements are now highlighted in different colors.
What do you think it means when a statement is highlighted in red?
.

The statement was never executed by the tests. In other words, the statement was not covered.
Are all of the statements in GiftCard.java covered?
.

No, many statements are shaded red.
What do you think it means when a statement is highlighted in yellow?
.

Some of the branches in the statement were not covered.
This color-coding scheme does not provide as much information as it might. So, first turn it off by clicking on the checkbox. Next, type Ctrl (or Cmd)+P+task and a space, and type or select "Create coverage report", type the name of the lab, and press Enter. Finally, type Ctrl (or Cmd)+P+task and a space, and type or select "Show coverage report". What happens?
.

A browser opens up that shows a page with a variety of clickable links.
Click "Source Files" and then click on "GiftCard.java". What happens?
.

A color-coded version of the code (that can't be edited) is displayed, but this version contains some diamond icons (in different colors) on the left.
Hover the cursor over a green diamond. What message is displayed?
.

All N branches covered.
Hover the cursor over a yellow diamond. What message is displayed?
.

M of N branches missed.

Add tests to your test suite so that it covers all of the statements and branches in the deduct() method in the GiftCard class.


                                        @Test
    public void deductTest_AmountDue() {
        double       balance;        
        GiftCard     card;
        int          issuingStore;
        String       s;
        
        
        issuingStore = 1337;
        balance      = 100.00;
        card = new GiftCard(issuingStore, balance);

        s = "Amount Due: " + String.format("%6.2f", 10.00);
        assertEquals(s, card.deduct(110.0), "deduct 110.00 from 100.00");
    }

    @Test
    public void deduct_InvalidTransaction() {
        double       balance;        
        GiftCard     card;
        int          issuingStore;
        String       s;
        
        
        issuingStore = 1337;
        balance      = 100.00;
        card = new GiftCard(issuingStore, balance);

        s = "Invalid Transaction";
        assertEquals(s, card.deduct(-10.0), "deduct -10.00 from 100.00");
    }

Run the entire test suite.
Your test suite still may not cover every statement in the GiftCard class. If this is the case, what is different about the statements that remain untested?
.

They all involve exceptions. (If you left other statements uncovered then you were careless.)

3. Testing Methods that Throw Exceptions: This part of the lab will help you learn how to test methods that throw exceptions.

The easiest (though not the most sophisticated) way to test for exceptions is to use include a try-catch statement in the test. For example, add the following test to your test suite.


                         @Test
    public void constructorTest_IncorrectBalance() {
        try {
            new GiftCard(1, -100.00);
            fail("Constructor, Negative Balance: Should have thrown an IllegalArgumentException");
        } catch (IllegalArgumentException iae) {
            // This is expected
        }
    }

Run all of the tests. Why are there uncovered statements in GiftCardTest.java and why is this unimportant?
.

There are statements in the test suite that should not be executed when GiftCard.java is correct. So, the fact that the test suite isn't completely covered doesn't matter -- after all, we're not trying to test the test suite

Add a test to your test suite named constructorTest_IncorrectID_Low() that covers the case when the storeID is less than 0. What code did you add?


                                        @Test
    public void constructorTest_IncorrectID_Low() {
        try {
            new GiftCard(-1, 100.00);
            fail("Constructor, Low ID: Should have thrown an IllegalArgumentException");
        } catch (IllegalArgumentException iae) {
            // This is expected
        }
    }

4. Coverage and Completeness: This part of the lab will help you better understand code coverage and the completeness of test suites.

Run all of the tests in your current test suite. Are all of the statements in GiftCard.java covered now?
.

Yes
What branch does the test suite fail to test?
.

So far, the test suite is only covering the storeID < 0 branch and not the storeID > MAX_ID branch.

Add a test to your test suite named constructorTest_IncorrectID_High() that covers the other branch.


                                        @Test
    public void constructorTest_IncorrectID_High() {
        try {
            new GiftCard(100000, 100.00);
            fail("Constructor, Low ID: Should have thrown an IllegalArgumentException");
        } catch (IllegalArgumentException iae) {
            // This is expected
        }
    }

Run all of the tests. Are all branches covered now?
.

Yes.
From a "white box" testing perspective, is your test suite complete? Conversely, can you think of tests that should be added?
.

I would certainly add a test that passed the constructor both an illegal storeID and an illegal openingBalance. Though each was handled properly individually, I can imagine situations where that might not be the case.
From a "black box" testing perspective, is your test suite complete? Conversely, can you think of tests that could be added?
.

It would probably be a good idea to test multiple different legal and illegal parameters, rather than just one (which is the case now). For example, maybe really big numbers or really small numbers will trigger a failure.