Introduction
In this post, I’ll set up unit testing with the Python unittest framework [1] and Coverage.py [2].
This is the third in a series of posts about creating a basic Python package and publishing it on PyPI. Here are the last two posts in the series:
You can download a zip file containing the code for the first version of my project. I’m using this workstation today.
This is a formal definition of unit testing:
Unit testing, a.k.a. component or module testing, is a form of software testing by which isolated source code is tested to validate expected behavior. Unit testing describes tests that are run at the unit-level to contrast testing at the integration or system level. [3]
There’s an overhead with unit testing, in writing the tests themselves. It’s also practically impossible to test every scenario on any imaginable platform that the software will run on. There have to be trade-offs - a compromise between sufficient coverage and effort. That said, there is huge value in being able to regression test the entire codebase instantly.
Personally, I don’t follow test-driven development (TDD) [4] strictly, in that I don’t write all of my tests before the code, but I do write tests at the same time or shortly after each discrete section (as small as I can manage). This keeps me disciplined and focused, and maintains quality. It cuts down time chasing bugs in big chunks of pre-written code. It also makes me pause before getting too enthusiastic about adding a bunch of “nice-to-have” features (because I don’t love writing unit tests to go with them). Making changes to code after writing unit tests requires a bit more work, so it also makes me plan more thoroughly.
Setting up
The first step in setting up unit tests is to create a place to keep them. In the project root, I’ll create the directory tests, then file tests/test_finance.py for the finance module tests.
I’ll create the test case structure, testing function finance.effective_rate(). I’m just demonstrating how to run tests so my test self.assertEqual(1, 1) will always be true.
import unittest
class TestFinance(unittest.TestCase):
def test_effective_rate(self):
self.assertEqual(1, 1)Running the tests is very easy - unittest will automatically find and run all tests for the project. I’ll go to the tests directory:
$ cd testsThen find and run the new test:
$ python -m unittest discover
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OKThe output tells me that the test was found and the result was OK.
Next I’ll modify my code so that the test fails, to demonstrate what an exception looks like. The assert statement now looks like this, which will always fail:
self.assertEqual(1, 2)And here is the test result:
$ python -m unittest discover
F
======================================================================
FAIL: test_effective_rate (test_finance.TestFinance.test_effective_rate)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests/test_finance.py", line 6, in test_effective_rate
self.assertEqual(1, 2)
~~~~~~~~~~~~~~~~^^^^^^
AssertionError: 1 != 2
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (failures=1)First test
In the previous post in this series, I wrote a test for function finance.effective_rate() where the expected output was from a Wikipedia example [5]:
$ python
Python 3.13.3 (main, Apr 9 2025, 08:55:03) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pydatk as ptk
>>> assert round(100 * ptk.finance.effective_rate(0.06, 12), 2) == 6.17
>>> exit()Because I’m running tests from the tests directory instead of the project root, Python can’t find the package directory. I can fix this by adding the src directory to the system path [6].
import os
import sys
import unittest
sys.path.append(os.path.dirname(os.path.realpath(__file__)) + "/../src")
import pydatk as ptk
class TestFinance(unittest.TestCase):
def test_effective_rate(self):
er = round(100 * ptk.finance.effective_rate(0.06, 12), 2)
self.assertEqual(er, 6.17)Coverage
Python’s unittest module makes it incredibly easy to write and run tests, but it gets even better: Coverage.py [2] can automatically detect how much of our code has been tested, and what we’ve missed.
This is especially useful if the code contains complex branches (e.g. if... else... conditions). It’s easy to accidentally miss some scenarios.
I’ll install and run Coverage. Detection of tests is still automated:
$ pip install coverage
$ coverage run -m unittest discover
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OKI can then check the report generated by Coverage. The --skip-empty parameter tells Coverage to ignore empty lines of code, and the --show-missing parameter indicates code that wasn’t covered by tests (I’ll include an example of this below).
$ coverage report --skip-empty --show-missing
Name Stmts Miss Cover Missing
-------------------------------------------------------------
pydatk/src/pydatk/__init__.py 1 0 100%
pydatk/src/pydatk/finance.py 2 0 100%
test_finance.py 9 0 100%
-------------------------------------------------------------
TOTAL 12 0 100%This tells us:
- Name: The name of the file. The test file
test_finance.pyis also included - this is useful to identify any redundant test code, or test code that is being skipped which should be included. - Stmts: The number of statements tested.
- Miss: The number of missed statements.
- Cover: Test coverage as a percentage.
- Missing: The line numbers of any code that wasn’t covered by tests.
I’ll add a dummy function to test_finance.py to demonstrate what happens when code isn’t covered:
def not_tested(number):
print(number)
return number + 20I’ll use coverage erase to clear previous results then re-run:
$ coverage erase
$ coverage run -m unittest discover
$ coverage report --skip-empty --show-missing
Name Stmts Miss Cover Missing
-------------------------------------------------------------
pydatk/src/pydatk/__init__.py 1 0 100%
pydatk/src/pydatk/finance.py 5 2 60% 5-6
test_finance.py 9 0 100%
-------------------------------------------------------------
TOTAL 15 2 87%The Missing column tells us that lines 5-6 of finance.py were not tested: This corresponds with the not_tested() function I just added. I’ll add a test for this to see what happens:
def test_not_tested(self):
self.assertEqual(ptk.finance.not_tested(10), 30)And I’ll run the tests again:
$ coverage erase
$ coverage run -m unittest discover
$ coverage report --skip-empty --show-missing
Name Stmts Miss Cover Missing
-------------------------------------------------------------
pydatk/src/pydatk/__init__.py 1 0 100%
pydatk/src/pydatk/finance.py 5 0 100%
test_finance.py 11 0 100%
-------------------------------------------------------------
TOTAL 17 0 100%The Missing line numbers have gone, indicating that the new test covered the dummy function.
For convenience, I’ll put the coverage commands into a bash script: tests/run_all.sh
#!/usr/bin/bash
coverage erase
coverage run -m unittest discover
coverage report --skip-empty --show-missingI’ll make it executable then check that it works:
$ chmod +x run_all.sh
$ ./run_all.sh
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
Name Stmts Miss Cover Missing
-------------------------------------------------------------
pydatk/src/pydatk/__init__.py 1 0 100%
pydatk/src/pydatk/finance.py 5 0 100%
test_finance.py 11 0 100%
-------------------------------------------------------------
TOTAL 17 0 100%As I add more features and unit tests, I can use the ./run_all.sh command to fully regression test the whole package. New tests will be identified automatically.
The limitation with this approach is that I’ve only identified which code has been executed at least once. I don’t know how many of the potential scenarios have been tested, depending on factors such as input arguments or platform configuration. I still need to rely on the trade-off between effort and completeness for this.
Summary
In this post, I’ve added unit testing to the pydatk package using the Python unittest module [1] and Coverage.py [2].
In the next post in this series, I’ll be adding inline code documentation and generating an API Reference.