Skip to content

Run Tests with Coverage

Test coverage is a measure of how much of your code is run by your tests. Using coverage helps you find areas of code that are not being tested.

Installation

The pytest-cov package adds coverage features to pytest. Install pytest-cov in Thonny's package manager (Tools -> Manage packages…) or run pip install pytest-cov in VS Code's terminal.

Note

As shown below, running coverage in Thonny is somewhat complicated. You have to type a long command in the shell, and you have to view the results in a web browser. In contrast, running coverage in VS Code is automatic. Choose your own adventure!

Running in Thonny

In Thonny, pytest is run manually from the shell. Running pytest with the --cov option displays coverage results:

>>> !pytest -q --cov
...                                                                      [100%]

---------- coverage: platform linux, python 3.12.3-final-0 -----------
Name                Stmts   Miss  Cover
---------------------------------------
test_triangles.py      11      0   100%
triangles.py           28      9    68%
---------------------------------------
TOTAL                  39      9    77%

3 passed in 0.04s

The above results show that 68% of the statements in triangles.py were covered by the tests. To see which lines were not tested, you can use the term-missing ("show missing lines in the terminal") coverage report:

>>> !pytest -q --cov --cov-report=term-missing
...                                                                      [100%]

---------- coverage: platform linux, python 3.12.3-final-0 -----------
Name                Stmts   Miss  Cover   Missing
-------------------------------------------------
test_triangles.py      11      0   100%
triangles.py           28      9    68%   21, 25, 58, 61, 63, 69-72
-------------------------------------------------
TOTAL                  39      9    77%

3 passed in 0.04s

In other words, lines 21, 25, 58, 61, 63, and 69–72 of triangles.py were not tested. A more convenient way to view the results is to generate an html report:

>>> !pytest -q --cov --cov-report=html
...                                                                      [100%]

---------- coverage: platform linux, python 3.8.10-final-0 -----------
Coverage HTML written to dir htmlcov

3 passed in 0.05s

The --cov-report=html option creates a folder named htmlcov. Open the index.html file in your web browser to see the results. Click here for an example html report.

Running in VS Code

You can run pytest automatically in VS Code.

  1. Click "Testing" (bottom icon) on the Activity Bar:

    VS Code Screenshot

  2. Click the "Configure Python Tests" button.

    • Select pytest
    • Select ". Root directory"

  3. Now you can click any of the "" buttons to run pytest. The third button "Run Test with Coverage" automatically runs pytest-cov.

NOTE: Over time, many folders named __pycache__ and .pytest_cache will be created automatically. You can hide these folders in VS Code by following these steps:

  1. Click "Explorer" (top icon) on the Activity Bar.

  2. Add the following code to your .vscode/settings.json file after the opening brace: 

        "files.exclude": {
        "**/__pycache__": true,
        "**/.pytest_cache": true,
        "**/.coverage": true
    },

Tip

The "" button is also shown in the editor to the left of each test function. You can right-click that button to see the "Run with Coverage" option.

Coverage Lab

Complete the following exercises to practice running tests with coverage.

Part 1: Triangles

Download triangles.py and test_triangles.py from the Installing/Running pytest instructions. Notice the provided test functions in test_triangles are incomplete.

  1. Run coverage to see which lines of triangles.py need to be tested.

  2. Add more assert statements to each function in test_trianges.py.

  3. Continue steps 1 and 2 until you get 100% coverage.

NOTE: You won't get 100% coverage of the triangles.py module, because some of the lines run only when __name__ == "__main__". However, you can add the comment # pragma: no cover to exclude the __main__ block from coverage analysis:

if __name__ == "__main__":  # pragma: no cover
    t = (3, 4, 5)
    print(f"valid({t}):", valid(t))
    print(f"area({t}):", area(t))
    print(f"classify({t}):", classify(t))

Part 2: Weather

Suppose that JMU had the following severe weather cancellation policy: (here's the real one)

JMU (FAKE) SEVERE WEATHER POLICY

If any of the following conditions are met, campus will be closed:

  • More than 6.0 inches of rain are predicted within the next 24 hours.
  • Wind speeds greater than 70.0 mph are predicted within the next 24 hours.
  • Predicting more than 4.0 inches of rain and wind speeds above 45.0 mph.

If none of the above conditions are met, the University may still issue a warning and encourage individuals to avoid non-essential outdoor activities. A warning will be issued if any of the following conditions are met:

  • Predicted wind speeds are above 45.0 mph.
  • Predicted precipitation is above 4.0 inches.

The following Python modules have been developed as part of JMU's decision support systems:

The weather_advice function provides an implementation of the cancellation policy above. Your job is to write a test module to confirm that this function is implemented correctly.

Submission

When finished, upload test_trianges.py and test_weather.py to Gradescope.