Skip coverage locally

Coverage is currently very slow to run, causing the unit tests to go from
taking 20 secs to 2 mins on my machine. This change means the default test
command only runs the fast tests directly, without running coverage. This gives
us much faster feedback during development, but still gives us the benefit of
coverage in CI.

This also uses xdist with its default settings to run the tests in parallel
locally.

.github/workflows/main.yml

@@ -84,7 +84,7 @@ jobs:
         run: |
           just check-migrations
           # hardcode n because auto=2 in CI for some reason
-          just test --migrations -n 4
+          just test-all --migrations -n 4
 
       - name: Upload HTML coverage report if tests failed
         uses: actions/upload-artifact@v4

DEVELOPERS.md

@@ -274,16 +274,16 @@ The Bennett Institute job server is deployed to our `dokku4` instance, instructi
 
 ## Testing
 
-Run the tests with:
+Run the unit tests:
 
 ```sh
-just test-dev
+just test
 ```
 
-To run the tests in parallel with using multiple cores, run them with:
+Run all of the tests (including slow tests) and run coverage, as it's done in CI:
 
 ```sh
-just test-dev -n auto
+just test-all
 ```
 
 More details on testing can be found in [TESTING.md](TESTING.md).

TESTING.md

@@ -85,13 +85,13 @@ We also provide DRF's testing tools as fixtures to mirror pytest-django.
 We use a fake object, `FakeGitHubAPI`, to test our uses of GitHub's API.
 However, we want to [verify that fake](https://pythonspeed.com/articles/verified-fakes/), so we have a set of verification tests in `tests/verification/`.
 
-`just test` will run those tests, but `just test-dev` will not, this way CI verifies our faster tests are correct.
+`just test-all` will run those tests, but `just test` will not, this way CI verifies our faster tests are correct.
 We have a separate GitHub org, `opensafely-testing`, and bot user, `opensafely-testing-bot`, for performing these tests.
 We use a different env var, `GITHUB_TOKEN_TESTING`, to pass the required PAT in.
 
 
 ## Useful Flows
-`just test` will run the tests as CI does, however as the suite grows this gets slower over time.
+`just test-all` will run the tests as CI does, however as the suite grows this gets slower over time.
 Below is [very!] non-exhaustive list of useful methods we have found to make running tests easier.
 
 * `pytest -k <partial test name>`: working on a new view? `pytest -k yourviewname` is a quick way to only run those tests.

justfile

@@ -133,15 +133,12 @@ run-telemetry: devenv
     $BIN/opentelemetry-instrument $BIN/python manage.py runserver --noreload
 
 
-test *args: assets
+test-all *args: assets
     COVERAGE_PROCESS_START="pyproject.toml" ./scripts/test-coverage.sh {{ args }}
 
 
-test-dev *args: assets
-    #!/bin/bash
-    export COVERAGE_REPORT_ARGS="--omit=jobserver/github.py,jobserver/opencodelists.py,tests/fakes.py,tests/integration/test_interactive.py,tests/verification/*,tests/unit/jobserver/models/test_paired_fields.py"
-    export COVERAGE_PROCESS_START="pyproject.toml"
-    ./scripts/test-coverage.sh -m "not verification and not slow_test" {{ args }}
+test *args: assets
+    $BIN/pytest -n auto -m "not verification and not slow_test" {{ args }}
 
 
 black *args=".": devenv