pytest-vcr-delete-on-fail

A pytest plugin that automates vcrpy cassettes deletion on test failure.

$ pip install pytest-vcr-delete-on-fail

Then, in your test:

import pytest
import requests
import vcr

my_vcr = vcr.VCR(record_mode="once")

cassette_path = "tests/cassettes/this.yaml"

@pytest.mark.vcr_delete_on_fail(cassette_path)
def test_this():
    with my_vcr.use_cassette(cassette_path):
        requests.get("https://github.com")
    assert False

In this example a cassette will be saved on disk when exiting the use_cassette context manager, but since the test eventually fails, the cassette will be deleted after the test teardown.

Rationale

Sometimes when testing a function containing multiple http requests a failure will occur halfway through (this happens all the time when doing TDD). When using vcrpy to cache http requests, this could result in a test cache that only cover a fraction of the function under test, which in turn could prevent the function to ever succeed or the test to pass in subsequent run if the http requests that didn't get cached depended on a fresh context (maybe they are time sensitive or there's randomness involved).

This possibility leads to doubt and lack of trust towards the test suite, which is wrong on too many level.

This plugin provides tools to solve this uncertainty, by deleting a test http requests cache if it fails, so that it can start fresh on the next run.

Docs

More information and examples can be found in the in-depth documentation.

Development

Install invoke and poetry:

$ pip install invoke poetry

Now clone the git repo:

$ git clone https://github.com/CarloDePieri/pytest-vcr-delete-on-fail.git
$ cd pytest-vcr-delete-on-fail
$ inv install

This will try to create a virtualenv based on python3.8 and install there all project's dependencies. If a different python version is preferred, it can be selected by specifying the --python (-p) flag like this:

$ inv install -p python3.9

The test suite can be run with commands:

$ inv test         # run the test suite
$ inv test-cov     # run the tests suite and produce a coverage report

To run the test suite against all supported python version (they must be in path!) run:

$ inv test-all-python-version

To test the GitHub workflow with act:

$ inv act-dev               # test the dev workflow
$ inv act-dev -c shell      # open a shell in the act container (the above must fail first!)
$ inv act-dev -c clean      # stop and delete a failed act container

To write the documentation with autobuild and livereload launch:

$ inv docs-serve