In this article, we’ll look at what software testing is, and why you should care about it. We’ll learn how to design unit tests and how to write Python unit tests. In particular, we’ll look at two of the most used unit testing frameworks in Python, unittest
and pytest
.
Introduction to Software Testing
Software testing is the process of examining the behavior of a software product to evaluate and verify that it’s coherent with the specifications. Software products can have thousands of lines of code, and hundreds of components that work together. If a single line doesn’t work properly, the bug can propagate and cause other errors. So, to be sure that a program acts as it’s supposed to, it has to be tested.
Since modern software can be quite complicated, there are multiple levels of testing that evaluate different aspects of correctness. As stated by the ISTQB Certified Test Foundation Level syllabus, there are four levels of software testing:
- Unit testing, which tests specific lines of code
- Integration testing, which tests the integration between many units
- System testing, which tests the entire system
- Acceptance testing, which checks the compliance with business goals
In this article, we’ll talk about unit testing, but before we dig deep into that, I’d like to introduce an important principle in software testing.
Testing shows the presence of defects, not their absence.
In other words, even if all the tests you run don’t show any failure, this doesn’t prove that your software system is bug-free, or that another test case won’t find a defect in the behavior of your software.
What is Unit Testing?
This is the first level of testing, also called component testing. In this part, the single software components are tested. Depending on the programming language, the software unit might be a class, a function, or a method. For example, if you have a Java class called ArithmeticOperations
that has multiply
and divide
methods, unit tests for the ArithmeticOperations
class will need to test both the correct behavior of the multiply
and divide
methods.
Unit tests are usually performed by software testers. To run unit tests, software testers (or developers) need access to the source code, because the source code itself is the object under test. For this reason, this approach to software testing that tests the source code directly is called white-box testing.
You might be wondering why you should worry about software testing, and whether it’s worth it or not. In the next section, we’ll analyze the motivation behind testing your software system.
Why you should do unit testing
The main advantage of software testing is that it improves software quality. Software quality is crucial, especially in a world where software handles a wide variety of our everyday activities. Improving the quality of the software is still too vague a goal. Let’s try to specify better what we mean by quality of software. According to the ISO/IEC Standard 9126-1 ISO 9126, software quality includes these factors:
- reliability
- functionality
- efficiency
- usability
- maintainability
- portability
If you own a company, software testing is an activity that you should consider carefully, because it can have an impact on your business. For example, in May 2022, Tesla recalled 130,000 cars due to an issue in vehicles’ infotainment systems. This issue was then fixed with a software update distributed “over the air”. These failures cost time and money to the company, and they also caused problems for the customers, because they couldn’t use their cars for a while. Testing software indeed costs money, but it’s also true that companies can save millions in technical support.
Unit testing focuses on checking whether or not the software is behaving correctly, which means checking that the mapping between the inputs and the outputs are all done correctly. Being a low-level testing activity, unit testing helps in the early identification of bugs so that they aren’t propagated to higher levels of the software system.
Other advantages of unit testing include:
- Simplifying integration: by ensuring that all the components work well individually, it’s easier to solve integration problems.
- Minimizing code regression: with a good amount of test cases, if some modifications to the source code in the future will cause problems, it’s easier to locate the issue.
- Providing documentation: by testing the correct mapping between input and output, unit tests provide documentation on how the method or class under test works.
Designing a Test Strategy
Let’s now look at how to design a testing strategy.
Definition of test scope
Before starting to plan a test strategy, there’s an important question to answer. What parts of your software system do you want to test?
This is a crucial question, because exhaustive testing is impossible. For this reason, you can’t test every possible input and output, but you should prioritize your tests based on the risks involved.
Many factors need to be taken into account when defining your test scope:
- Risk: what business consequences would there be if a bug were to affect this component?
- Time: how soon do you want your software product to be ready? Do you have a deadline?
- Budget: how much money are you willing to invest in the testing activity?
Once you define the testing scope, which specifies what you should test and what you shouldn’t test, you’re ready to talk about the qualities that a good unit test should have.
Qualities of a unit test
- Fast. Unit tests are mostly executed automatically, which means they must be fast. Slow unit tests are more likely to be skipped by developers because they don’t provide instant feedback.
- Isolated. Unit tests are standalone by definition. They test the individual unit of code, and they don’t depend on anything external (like a file or a network resource).
- Repeatable. Unit tests are executed repeatedly, and the result must be consistent over time.
- Reliable. Unit tests will fail only if there’s a bug in the system under test. The environment or the order of execution of the tests shouldn’t matter.
- Named properly. The name of the test should provide relevant information about the test itself.
There’s one last step missing before diving deep into unit testing in Python. How do we organize our tests to make them clean and easy to read? We use a pattern called Arrange, Act and Assert (AAA).
The AAA pattern
The Arrange, Act and Assert pattern is a common strategy used to write and organize unit tests. It works in the following way:
- During the Arrange phase, all the objects and variables needed for the test are set.
- Next, during the Act phase, the function/method/class under test is called.
- In the end, during the Assert phase, we verify the outcome of the test.
This strategy provides a clean approach to organizing unit tests by separating all the main parts of a test: setup, execution and verification. Plus, unit tests are easier to read, because they all follow the same structure.
Unit Testing in Python: unittest or pytest?
We’ll now talk about two different unit testing frameworks in Python. The two frameworks are unittest
and pytest
.
Introduction to unittest
The Python standard library includes the unittest unit testing framework. This framework is inspired by JUnit, which is a unit testing framework in Java.
As stated in the official documentation, unittest
supports a few important concepts that we will mention in this article:
- test case, which is the single unit of testing
- test suite, which is a group of test cases that are executed together
- test runner, which is the component that will handle the execution and the result of all the test cases
unittest
has its way to write tests. In particular, we need to:
- write our tests as methods of a class that subclasses
unittest.TestCase
- use special assertion methods
Since unittest
is already installed, we’re ready to write our first unit test!
Writing unit tests using unittest
Let’s say that we have the BankAccount
class:
import unittest
class BankAccount:
def __init__(self, id):
self.id = id
self.balance = 0
def withdraw(self, amount):
if self.balance >= amount:
self.balance -= amount
return True
return False
def deposit(self, amount):
self.balance += amount
return True
We can’t withdraw more money than the deposit availability, so let’s test that this scenario is handled correctly by our source code.
In the same Python file, we can add the following code:
class TestBankOperations(unittest.TestCase):
def test_insufficient_deposit(self):
# Arrange
a = BankAccount(1)
a.deposit(100)
# Act
outcome = a.withdraw(200)
# Assert
self.assertFalse(outcome)
We’re creating a class called TestBankOperations
that’s a subclass of unittest.TestCase
. In this way, we’re creating a new test case.
Inside this class, we define a single test function with a method that starts with test
. This is important, because every test method must start with the word test
.
We expect this test method to return False
, which means that the operation failed. To assert the result, we use a special assertion method called assertFalse()
.
We’re ready to execute the test. Let’s run this command on the command line:
python -m unittest example.py
Here, example.py
is the name of the file containing all the source code. The output should look something like this:
.
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
Good! This means that our test was successful. Let’s see now how the output looks when there’s a failure. We add a new test to the previous class. Let’s try to deposit a negative amount of money, which of course isn’t possible. Will our code handle this scenario?
This is our new test method:
def test_negative_deposit(self):
# Arrange
a = BankAccount(1)
# Act
outcome = a.deposit(-100)
# Assert
self.assertFalse(outcome)
We can use the verbose mode of unittest
to execute this test by putting the -v
flag:
python -m unittest -v example.py
And the output is now different:
test_insufficient_deposit (example.TestBankOperations) ... ok
test_negative_deposit (example.TestBankOperations) ... FAIL
======================================================================
FAIL: test_negative_deposit (example.TestBankOperations)
----------------------------------------------------------------------
Traceback (most recent call last):
File "example.py", line 35, in test_negative_deposit
self.assertFalse(outcome)
AssertionError: True is not false
----------------------------------------------------------------------
Ran 2 tests in 0.002s
FAILED (failures=1)
In this case, the verbose flag gives us more information. We know that the test_negative_deposit
failed. In particular, the AssertionError
tells us that the expected outcome was supposed to be false
but True is not false
, which means that the method returned True
.
The unittest
framework provides different assertion methods, based on our needs:
assertEqual(x,y)
, which tests whetherx == y
assertRaises(exception_type)
, which checks if a specific exception is raisedassertIsNone(x)
, which tests ifx is None
assertIn(x,y)
, which tests ifx in y
Now that we have a basic understanding of how to write unit tests using the unittest
framework, let’s have a look at the other Python framework called pytest
.
Introduction to pytest
The pytest
framework is a Python unit testing framework that has a few relevant features:
- it allows complex testing using less code
- it supports
unittest
test suites - it offers more than 800 external plugins
Since pytest
isn’t installed by default, we have to install it first. Note that pytest
requires Python 3.7+.
Installing pytest
Installing pytest
is quite easy. You just have to run this command:
pip install -U pytest
Then check that everything has been installed correctly by typing this:
pytest --version
The output should look something like this:
pytest 7.1.2
Good! Let’s write the first test using pytest
.
Writing unit tests using pytest
We’ll use the BankAccount
class written before, and we’ll test the same methods as before. In this way, it’s easier to compare the effort needed to write tests using the two frameworks.
To test with pytest
we need to:
- Create a directory and put our test files inside it.
- Write our tests in files whose names start with
test_
or end with_test.py
.pytest
will look for those files in the current directory and its subdirectories.
So, we create a file called test_bank.py
and we put it into a folder. This is what our first test function looks like:
def test_insufficient_deposit():
# Arrange
a = BankAccount(1)
a.deposit(100)
# Act
outcome = a.withdraw(200)
# Assert
assert outcome == False
As you have noticed, the only thing that changed with respect to the unittest
version is the assert section. Here we use plain Python assertion methods.
And now we can have a look at the test_bank.py
file:
class BankAccount:
def __init__(self, id):
self.id = id
self.balance = 0
def withdraw(self, amount):
if self.balance >= amount:
self.balance -= amount
return True
return False
def deposit(self, amount):
self.balance += amount
return True
def test_insufficient_deposit():
# Arrange
a = BankAccount(1)
a.deposit(100)
# Act
outcome = a.withdraw(200)
# Assert
assert outcome == False
To run this test, let’s open a command prompt inside the folder where the test_bank.py
file is located. Then, run this:
pytest
The output will be something like this:
======== test session starts ========
platform win32 -- Python 3.7.11, pytest-7.1.2, pluggy-0.13.1
rootdir: \folder
plugins: anyio-2.2.0
collected 1 item
test_bank.py . [100%]
======== 1 passed in 0.02s ========
In this case, we can see how easy it is to write and execute a test. Also, we can see that we wrote less code compared to unittest
. The result of the test is also quite easy to understand.
Let’s move on to see a failed test!
We use the second method we wrote before, which is called test_negative_deposit
. We refactor the assert section, and this is the result:
def test_negative_deposit():
# Arrange
a = BankAccount(1)
# Act
outcome = a.deposit(-100)
# Assert
assert outcome == False
We run the test in the same way as before, and this should be the output:
======= test session starts =======
platform win32 -- Python 3.7.11, pytest-7.1.2, pluggy-0.13.1
rootdir: \folder
plugins: anyio-2.2.0
collected 2 items
test_bank.py .F [100%]
======= FAILURES =======
_____________ test_negative_deposit _____________
def test_negative_deposit():
# Arrange
a = BankAccount(1)
# Act
outcome = a.deposit(-100)
# Assert
> assert outcome == False
E assert True == False
test_bank.py:32: AssertionError
======= short test summary info =======
FAILED test_bank.py::test_negative_deposit - assert True == False
======= 1 failed, 1 passed in 0.15s =======
By parsing the output, we can read collected 2 items
, which means that two tests have been executed. Scrolling down, we can read that a failure occurred while testing the test_negative_deposit
method. In particular, the error occurred when evaluating the assertion. Plus, the report also says that the value of the outcome
variable is True
, so this means that the deposit
method contains an error.
Since pytest
uses the default Python assertion keyword, we can compare any output we get with another variable that stores the expected outcome. All of this without using special assertion methods.
Conclusion
To wrap it up, in this article we covered the basics of software testing. We discovered why software testing is essential and why everyone should test their code. We talked about unit testing, and how to design and implement simple unit tests in Python.
We used two Python frameworks called unittest
and pytest
. Both have useful features, and they’re two of the most-used frameworks for Python unit testing.
In the end, we saw two basic test cases to give you an idea of how tests are written following the Arrange, Act and Assert pattern.
I hope I’ve convinced you of the importance of software testing. Choose a framework such as unittest
or pytest
, and start testing — because it’s worth the extra effort!
If you enjoyed this article, you might also find the following useful:
- Cypress Testing: A Guide to Running Web Application Tests
- How to Test React Components Using Jest
- Learn End-to-end Testing with Puppeteer
- 3 Methods for Hands-free Continuous Testing
- Re-Introducing Jenkins: Automated Testing with Pipelines
FAQs About Python Unit Testing
Unit testing in Python is a software testing technique where individual units or components of a program are tested in isolation to ensure that each unit works as intended.
Unit testing helps ensure the correctness of individual components in a Python program. It facilitates early detection of bugs, provides a safety net for code changes, and supports code maintainability.
Unit tests in Python are typically written using the built-in unittest
module. You create test classes that inherit from unittest.TestCase
and write test methods within these classes. Test methods usually start with the word “test.”
unittest
for Python unit testing? Yes, besides unittest
, there are other popular testing frameworks for Python, such as pytest
and nose2
. These frameworks offer different features and syntax, providing developers with flexibility in choosing the one that best fits their needs.
Fixtures are a way to set up preconditions and clean up after tests in Python. They help ensure that tests are isolated and can be run independently.
Test coverage measures the percentage of your codebase that is exercised by your tests. It helps identify untested code and ensures that your tests are comprehensive, reducing the likelihood of undiscovered bugs.
Yes, some best practices include writing independent and isolated tests, using descriptive test method names, and testing boundary cases. Additionally, strive for good test coverage and run tests frequently.
![Lorenzo Bonannella](https://uploads.sitepoint.com/wp-content/uploads/2022/07/1658286002picture-120x120.png)
I am a computer science student fond of asking questions and learning new things. Traveller, musician and occasional writer. Check me out on my website.