Testing Happens at Multiple Levels

• Unit Testing - Test individual classes in isolation.
  • Focus is on making sure that each method works according to specification.
• Integration Testing - Test the interaction between classes.
• Validation Testing - Test the entire system in context.

Different Perspectives

• Black-box testing
  • Develop tests on the basis of class specifications and documentation.
• White-box testing
  • Develop tests on the basis of implementation.
  • Aim for high code coverage.

JUnit - Sample Test Class

import static org.junit.jupiter.api.Assertions.*;

import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;

class PointTest {

    private Point testPoint;

    @BeforeEach
    void setUp() throws Exception {
        testPoint = new Point(5.0, 6.0);
    }

    @Test
    void testToString() {
        assertEquals("(5.0, 6.0)", testPoint.toString());
    }

}

Testing Example

public class Estimator {
    
    public static int totalCost(boolean fast, boolean good) {
        
        int total = 0;
        
        if (fast) {
            total += 5;
        }
        
        if (good) {
            total += 10;
        }
        
        return total;
    }
    
}

Method Coverage

• 100% method coverage: testing code calls each method at least once.

      @Test
      public void testTotalCostSlowBad() {
          assertEquals(0, Estimator.totalCost(false, false));
      }

• Done! Reassuring?

• No, but better than NOT having 100% method coverage.

Style Suggestion

• JUnit assertion methods take an optional failure message:

  @Test
  public void testTotalCostSlowBad() {
      assertEquals(0, Estimator.totalCost(false, false),
              "Estimate should have been 0.");
  }

• My suggestion: don’t provide a message
• Instead:
  • One(ish) assertion per test method
  • Very descriptive method names

Statement Coverage

• 100% statement coverage: testing executes every statement.

  @Test
  public void testTotalCostFastGood() {
      assertEquals(15, Estimator.totalCost(true, true));
  }

• Better? Happy?

Branch Coverage

• 100% branch coverage: all outcomes are executed for each branch.

  @Test
  public void testTotalCostSlowBad() {
      assertEquals(0, Estimator.totalCost(false, false));
  }
  
  @Test
  public void testTotalCostFastGood() {
      assertEquals(15, Estimator.totalCost(true, true));
  }

• Good enough?

Path Coverage

• 100% path coverage: testing exercises every possible path through the code.

  @Test
  public void testTotalCostSlowBad() {
      assertEquals(0, Estimator.totalCost(false, false));
  }
  @Test
  public void testTotalCostFastGood() {
      assertEquals(15, Estimator.totalCost(true, true));
  }
  @Test
  public void testTotalCostSlowGood() {
      assertEquals(10, Estimator.totalCost(false, true));
  }
  @Test
  public void testTotalCostFastBad() {
      assertEquals(5, Estimator.totalCost(true, false));
  }

Path Coverage

• 100% path coverage is typically considered an impractical target.
• It is a useful idea to have in mind while developing tests.

Developing Test Cases

• To guarantee correctness, test every possible sequence of method calls, with every possible input value.
  • Usually not possible.
• Instead, look for boundary conditions
  • Points where the behavior of the code should change
  • Test at the boundaries and on either side
• Also test erroneous inputs

Socrative Quiz #1

public class ShopUtil {

    public static final int MIN_FOR_DISCOUNT = 4;
    public static final int DISCOUNT_PER_ITEM = 2;
    
    public static int discount(int numBought) {   
        if (numBought < 0) {
            throw new IllegalArgumentException();
        }
        
        int discount = 0;
        if (numBought >= MIN_FOR_DISCOUNT ) {
            discount = numBought * DISCOUNT_PER_ITEM;
        } 
        return discount;
    }
}

class ShopUtilTest {

    @Test
    void testDiscountNegativeBoughtThrowsException() {
        assertThrows(IllegalArgumentException.class, () -> {
            ShopUtil.discount(-1);
        });
    }

    @Test
    void testDiscountNumBoughtAboveMin() {
        assertEquals(20, ShopUtil.discount(10));
    }

    @Test
    void testDiscountNumBoughtBelowMin() {
        assertEquals(0, ShopUtil.discount(1));
    }
}

Socrative Quiz #2

Select the best option. This test class..

1. Is broken! It provides good coverage, but the tests are incorrect.
2. Is correct, but not high-quality.
3. Is good! It contains no logic errors and provides 100% coverage.

Black box Testing of Estimator

    /**
     * Estimate the total cost of a job. The base cost is 0, requesting a fast
     * job adds $5 to the cost, requesting a good job adds $10 to the cost.
     * 
     * @param fast Does the customer want the job done fast?
     * @param good Does the customer want the job done correctly?
     * @return Estimate of the job cost in dollars.
     */
    public static int totalCost(boolean fast, boolean good) {

Test-Driven Development

• Write tests first.
  • Helps clarify specifications.
  • Helps avoid mistakes in development.

Regression Testing

• Testing is not a one-time process.
• Ideally, unit tests are maintained along with the code.
• This makes it safer to change the code:
  • All tests can be run after every change.

If time… One More Quiz

Open up a copy of LetterTest.java from PA1.

Take a minute to read over the tests then answer the following questions (privately) in the chat:

1. Indicate places in these tests (if any) where our tests don’t follow the advice from today’s lecture.