Testing

Testing determines whether software runs correctly based on specific inputs and identifies defects that need to be fixed.

Why is testing important?

As software scales in codebase size, it's impossible for a person or even a large team to keep up with all of the changes and the interactions between the changes. Automated testing is the only proven method for building reliable software once they grow past the point of a simple prototype. Many major software program development failures can be traced back to inadequate or a complete lack of testing.

It's impossible to know whether software works properly unless it is tested. While testing can be done manually, by a user clicking buttons or typing in input, it should be performed automatically by writing software programs that test the application under test.

There are many forms of testing and they should all be used together. When a single function of a program is isolated for testing, that is called unit testing. Testing more than a single function in an application at the same time is known as integration testing. User interface testing ensures the correctness of how a user would interact with the software. There are even more forms of testing that large programs need, such as load testing, database testing, and browser testing (for web applications).

Testing in Python

Python software development culture is heavy on software testing. Because Python is a dynamically-typed language as opposed to a statically-typed language, testing takes on even greater importance for ensuring program correctness.

Python testing tools

The Python ecosystem has a wealth of tools to make it easier to run your tests and interpret the results. The following tools encompass test runners, coverage reports and related libraries.

• green is a test runner that has pretty printing on output to make the results easier to read and understand.

• requestium merges the Requests library with Selenium to make it easier to run automated browser tests.

• coverage.py is a tool for measuring code coverage when running tests.

Testing resources

• The Minimum Viable Test Suite shows how to set unit tests and integration tests for a Flask example application.

• BDD Testing a Restful Web Application in Python is an introduction to behavior-driven development (BDD) and uses a Flask web application as an example project for learning.

• Testing, for people who hate testing provides examples for how to improve your testing environment such as using a new test harness and getting your test suite to run fast.

• Getting started with Pytest goes over some code challenges as examples for how to use Pytest in your own projects.

• Good test, bad test explains the difference between a "good" test case and one that is not as useful. Along the way the post breaks down some myths about common testing subjects such as code coverage, assertions and mocking.

• Python Testing is a site devoted to testing in - you guessed it - the Python programming language.

• In praise of property-based testing is an article by the author of the Hypothesis testing tool written in Python. The article explains the shortcomings of example-based tests and how property-based testing can augment or replace those example-based tests to find more defects in software. There is also a great introductory post on Hypothesis that goes into further detail about getting your first Hypothesis tests up and running.

• Google has a testing blog where they write about various aspects of testing software at scale.

• A beginner's guide to Python testing covers test-driven development for unit, integration and smoke tests.

• Testing a Twilio Interactive Voice Response (IVR) System With Python and pytest is an incredibly in-depth tutorial on testing a Python-powered IVR using pytest. There is also a tutorial on how to build the IVR before this testing tutorial, although you can just clone it to jump into the testing walkthrough.

• Still confused about the difference between unit, functional and integration tests? Check out this top answer on Stack Overflow to that very question.

• Testing Python applications with Pytest walks through the basics of using Pytest and some more advanced ways to use it such as continuous testing through Semiphore CI.

• The cleaning hand of Pytest provides a couple of case studies for how companies set up their testing systems. It then gives the boilerplate code the author uses for Pytest and goes through a bunch of code samples for common situations.

• Using pytest with Django shows how to get a basic pytest test case running for a Django project and explains why the author prefers pytest over standard unittest testing.

• Distributed Testing with Selenium Grid and Docker shows how to distribute automated, browser tests with Selenium Grid and Docker Swarm. It also looks at how to run tests against a number of browsers and automate the provisioning and deprovisioning of machines to keep costs down.

• Principles of Automated Testing explains how to prioritize what to test, goes through some levels of testing from unit to integration and examines when to use example and bulk tests.

• How to run tests continuously while coding contains a Python script that uses the watchdog to check for changes to source code files in your project directory. If changes are detected then your tests will be run to check that everything is still working as intended.

• 8 great pytest plugins is a list of plugins that go with PyTest and help with tasks like reducing the friction around testing Django, as well as running tests in parallel.

• This test-driven development series shows you how to write an interpreter in Python and contains a ton of great code samples to learn from:

  • A game of tokens: write an interpreter in Python with TDD - Part 1
  • A game of tokens - part 2
  • A game of tokens - part 3
  • A game of tokens - part 4

• Pytest leaking examines situations where tests leak memory and can cause abnormal results if they are not fixed.

Mocking resources

Mocking allows you to isolate parts of your code under test and avoid areas that are not critical to the tests you are writing. For example, you may want to test how you handle text message responses but do not want to actually receive text messages in your application. You can mock a part of the code that would provide a happy-path scenario so you can run your tests properly. The following resources show you how to use mocks in your test cases.

• Getting Started with Mocking in Python provides a whole code example based on a blog project that shows how to use mock when testing.

• Python Mocking 101: Fake It Before You Make It explains what mocking is and is not, and shows how to use the patch function to accomplish it in your project. Revisiting Unit Testing and Mocking in Python is a follow-up post that expands upon using the patch function along with dependency injection.

• Mocks and Monkeypatching in Python uses the mock and monkeypatch tools to show how to mock your test cases.

• Mock yourself, not your tests examines when mocks are necessary and when they are not as useful so you can avoid them in your test cases.

• Better tests for Redis integrations with redislite is a great example of how using the right mocking library can clean up existing hacky testing code and make it more straightforward for any developer that happens upon the tests in the future.

What's next after testing your application?