Skip to content

HW4: Unit Testing

Learning Objectives

This homework assignment is designed to help you understand unit testing and debugging. This includes an understanding of black-box/closed-box testing, white-box/open-box testing (and the JUnit framework), test coverage (and the EclEmma/JaCoCo framework), the debugging process, and instrumentation techniques. The following definitions are relevant in this regard.

Black-Box/Closed-Box Tests
Tests that are based on specifications, rather than the actual implementation of the code.
White Box/Open-Box Tests
Tests that are based on the actual implementation of the code.
Heuristic
An acceptable approximation.
Rule of Thumb
A guideline that is informed by experience.
Random Tests
Tests with inputs that were chosen at random by the tester. As with any test, it includes the expected output.

Motivation

Thus far, your testing has been haphazard (i.e., you have not thought carefully about how to test your code to ensure that it is correct) and you have probably relied to heavily on the submission system to test your code (because we have not penalized you for “excess” submissions). Going forward, you will have to do a much better job of testing your code, and this assignment is designed to help you do so.

To help motivate some of the aspects of this assignment, it is useful to briefly return to the previous assignment, test it using JUnit, consider some issues related to coverage, and think about the implications for the current assignment.

To that end, download the following file (that contains JUnit tests for the previous assignment) into the directory/folder hw3:

PokeAtYouWhiteBoxTests.java

and answer the following questions. (Note: You do not have to submit the answers to these questions.)

Questions About The Importance of Test Coverage

  1. RunPokeAtYouWhiteBoxTests (i.e. click on the ▷ icon to the left of the class declaration for PokeAtYouWhiteBoxTests). Does your code fail any of the tests? In other words, are any tests highlighted? (It should pass all of the tests if your solution is correct or you are using the solution that was provided to you.)

  2. Check how these tests covered PokeEvent.java. There are two ways to do this. One way is to right-click on ▷ icon to the left of the class declaration for PokeAtYouWhiteBoxTests and pull-down to “Run with Coverage”. The other is to open the “Testing” view (using the menu [View]+[Testing] or by clicking on the beaker icon) and then by hovering over ‘PokeAtYouWhiteBoxTestsand clicking on "Run Test with Coverage". Do the tests completely cover the statements and branches inPokeEvent.java`?

  3. Comment-out the last statement in the testBattle() method (i.e., the call to increaseRightScore() in the PokeAtYouWhiteBoxTests class), and run PokeAtYouWhiteBoxTests using the coverage tool. What is now true of the branch coverage of the points() method? Why?

  4. Why is it important to have 100% statement coverage and 100% branch coverage?

Before you proceed, undo the change you made to the PokeAtYouWhiteBoxTests class.

Questions About The Need For More Than Just Coverage

  1. Temporarily, change the increaseLeftScore() method in the Battle class so that it increases the scoreLeft attribute by 50, and run PokeAtYouWhiteBoxTests using the coverage tool. Despite the fact that the Battle class now contains a defect, does it pass all of the tests? Is the Battle class completely covered?

  2. Does the PokeAtYouWhiteBoxTests class actually test the increaseLeftScore() method or does it just execute it? Does the coverage tool indicate that statements/branches have been tested, or just that they’ve been executed?

  3. Add the statement assertEquals("Deadpool 1, Wolverine 1", battle.toString(), "increase scores"); to the end of the testBattleMethod() and rerun the tests. Does the code now pass all of the tests?

  4. Correct the defect in the Battle class and rerun the tests. Does the code now pass all of the tests?

  5. In the PokeEvent enum, temporarily change the weight of the LARGE instance to 50 rather than 500, and launch PokeAtYouWhiteBoxTests using the coverage tool. Is the PokeEvent enum completely covered? Despite the fact that the PokeEvent enum now contains a defect, does it pass all of the tests? Why?

  6. Is it enough to just cover code when writing unit tests?

  7. What are the implications of your answer to the previous questions? In other words, what must you be sure to do when you are required to write and submit unit tests for future assignments?

Before you proceed, undo the change you made to the PokeEvent class.

Questions About Unit Testing

  1. Which testing process was easier to use, the one that you used for the previous assignment (i.e., the one using the Test class you wrote) or the one you just used (i.e., the one using JUnit)? Focus just on the process and ignore whether you had to write the tests or not. Why?

  2. If you become a professional programmer, there will not be a submission system for you to rely on. Instead, what must you do to ensure that your code is correct? With that in mind, what must you now start doing this semester (including on this assignment)?

Background

PolySounder Logo

Danger

As discussed below, on this assignment you may submit to Gradescope at most ten times. Hence, you should be confident that your code is correct before you submit the first time.

The (fictitious) company Lexicality has written a product that can be used to determine when words sound alike. They have also written a suite of white box tests that covers all statements and branches in this product. They have come to you to do additional testing of the two classes that comprise the product and to debug them if necessary. (Hint: It will be necessary – both classes contain errors.)

Algorithms, Heuristics, Formulas, and Examples

The product converts a word (consisting only of alphabetic characters) into a code (consisting of a smaller number of numeric characters). It is supposed to proceed as follows:

  1. Convert the given word to uppercase (e.g., bucket1 becomes BUCKET1).
  2. Remove all non-alphabetic characters (e.g., BUCKET1 becomes BUCKET).
  3. Remove the vowels (e.g., BUCKET becomes BCKT).
  4. Remove the characters H and W (e.g., no change).
  5. Code B, F, P, and V as 1 (e.g., BCKT becomes 1CKT).
  6. Code C, G, J, K, Q, S, X, and Z as 2 (e.g., 1CKT becomes 122T).
  7. Code D and T as 3 (e.g., 122T becomes 1223).
  8. Code L as 4 (e.g., no change).
  9. Code M and N as 5 (e.g., no change).
  10. Code R as 6 (e.g., no change).
  11. Remove any consecutive repetitions (e.g., 1223 becomes 123. Note: 12223 would also become 123!).
  12. If there are too many characters, remove the extras from the right. On the other hand, if there are too few characters, add 0s either to the left or right (as specified) to create a code with the specified number of characters (e.g., 123 becomes 123000 when padded on the right to a length of six).

As an example, bucked and packed sound alike and would both be coded as 123000

Existing Classes

Lexicality has provided you with the PolySounder class and a utility class that it uses named TextUtils.

TextUtils.java

PolySounder.java

They have also provided you with a white box test suite (written in Java using the JUnit framework) that covers all statements and branches in both classes.

TextUtilsWhiteBoxTests.java

PolySounderWhiteBoxTests.java

Your Tasks

You must test and debug the classed that you have been given. The end result must be a correct version of the two classes with every change you make documented in the comments. To that end you must:

  1. Create a class named TextUtilsHeuristicTests.java that contains JUnit tests based on “rules of thumb” about the inputs for all of the methods in the TextUtils class.

  2. Create a class named TextUtilsRandomTests.java that contains JUnit tests based on random inputs for all of the methods in the TextUtils class.

  3. Create a class named PolySounderHeuristicTests.java that contains JUnit tests based on “rules of thumb” about the inputs for all of the methods in the PolySounder class.

  4. Create a class named PolySounderRandomTests.java that contains JUnit tests based on random inputs for all of the methods in the PolySounder class.

Note

The random test classee should not use the Random class in Java. You must construct inputs randomly by hand (along with the expected outputs) before you create the JUnit code. Nothing random should happen at run-time.

Both TextUtilsHeuristicTests.java and TextUtilsRandomTests.java must eventually have at least four methods. These methods must be preceded by the JUnit @Test annotation and must have names that begin with padTest, findIndexOfTest, ‘findReplacementForTest, andremoveFromTest. (They can have suffixes likepadTest1(),padTest2()`, etc. if you would like to have more than four in total).

Similarly, both PolySounderHeuristicTests.java and PolySounderRandomTests.java must have at least one method with a name that begins with toCodeTest.

You should add these methods one at a time, running all of the tests each time you do so. When the actual output of one of your tests does not equal the expected output you should understand the symptom and the trigger condition (i.e., stabilize the fault). Then, localize the fault, correct the fault (documenting the correction in the source code), and verify the correction.

Some Likely Rules of Thumb

There are a variety of rules of thumb that people use when testing code that involves String objects, including:

  • A String that is missing letters
  • A String that is missing the first letter
  • A String that is missing the last letter
  • A String that contains no letters

There are a variety of rules of thumb that people use when testing code that involves int values, including:

  • Positive values
  • 0 values
  • Negative values

There are a variety of rules of thumb that people use when testing code that involves arrays, including:

  • An array with 1 element
  • A long array
  • An array in which all of the elements are the same
  • An array sorted in ascending order
  • An array sorted in descending order

You should be able to think of other rules-of-thumb that might be likely to uncover defects in the two classes that you have been given.

Submission

You must submit (using Gradescope):

  1. Your corrected implementation of the TextUtils class (with appropriate comments).
  2. Your corrected implementation of the PolySounderclass (with appropriate comments).
  3. Your TextUtilsRandomTests.java class (containing at least four methods).
  4. Your TextUtilsHeuristicTests.java (containing at least four methods).
  5. Your PolySounderRandomTests.java.
  6. Your PolySounderHeuristicTests.java.

Because you should not be using Gradescope to verify your code (i.e., because you must be able to test and debug code without the help of a submission system), you may submit to Gradescope at most ten times and you will not receive any hints from Gradescope. If you submit more than ten times you will receive a grade of 0.

Note that your test classes do not need to comply with the style guide.

Grading

Your code will first be graded by Gradescope and then by the Professor. The grade you receive from Gradescope is the maximum grade that you can receive on the assignment

Gradescope Grading

Your code must compile (in Gradescope, this will be indicated in the section on “Does your code compile?”) and all class names and method signatures must comply with the specifications (in Gradescope, this will be indicated in the section on “Do your class names, method signatures, etc. comply with the specifications?”) for you to receive any points on this assignment. Gradescope will then grade your submission as follows:

Criterion Points Details
Conformance to the Style Guide (Style) 0 points (All or Nothing; Success Required)
Passing Your Tests (SelfTests) 20 points (All or Nothing; Success Required)
Correctness (Official Tests) 80 points (Partial Credit Possible)

Note that you are not being given any points for writing tests that cover the TextUtils and PolySounder classes. By now you should realize that 100% coverage is necessary but not sufficient.

Manual Grading

After the due date, the Professor may manually review your code. At this time, points may be deducted for inelegant code, inappropriate variable names, bad comments, etc.

First, you should stub-out all four of the test classes. (Note: Gradescope will not assess your submission if it does not contain all four test classes with all of the required methods.)

After that, you should test the methods in the following order:

  1. pad()
  2. findIndexOf()
  3. findReplacementFor()
  4. removeFrom()
  5. toCode()

using both heuristic tests and random tests.

Help

An understanding of the following programming patterns will help you complete this assignment:

Questions to Think About

You don’t have to submit your answers to these questions, but you should try to answer them because they will help you determine whether or not you understand some of the important concepts covered in this assignment.

  1. The white-box tests completely covered the code. How could it still contain defects?

  2. Why are both heuristic tests (based on rules of thumb) and random tests (based on random inputs) necessary (in addition to white-box tests)?

  3. As you know, applications written in Java must have a main class that contains a method with the signature public static void main(String[]) that is called the entry point of the application. When you write JUnit tests, you do not need to create such a class. What must the JUnit framework be providing behind the scenes?

  4. How does your answer to the previous question make it a little more convenient to use JUnit than the Test class you have used on previous assignments?

  5. Why can the methods in the TextUtils class be static?

  6. Similarly, why can’t the toCode() method in the PolySounder class be static?

  7. Why are there two classes in this product? In other words, why weren’t the methods in the TextUtils class included in the PolySounder class? Hint: This has nothing to do with whether the methods are static or not.

  8. Why is it a good idea to put the tests for each class in distinct classes?

  9. Why did we recommend the testing order above?