Skip to content
This repository has been archived by the owner on Jan 7, 2024. It is now read-only.

Use pytest without unittest's TestCase class #151

Merged
merged 6 commits into from
Dec 11, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 19 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ When tests are run, they replay recorded API request and response data instead o

**Note:** We have a CI test that does not use the recorded API request and response data in order to make sure we are testing the latest changes to the SDK against the latest server API (see `test-against-latest-api` in https://github.com/freedomofpress/securedrop-sdk/blob/main/.circleci/config.yml).

We use [vcrpy](https://vcrpy.readthedocs.io/en/latest/) to record and replay API calls made over HTTP and a decorator called `@dastollervey_datasaver` to record and replay API calls made over qrexec. Each request made from a test and its response from the server is stored in a "cassette" in the `data` directory. Tests replay these cassettes instead of making actual API calls to a server.
We use [vcrpy](https://vcrpy.readthedocs.io/en/latest/) to record and replay API calls made over HTTP and a decorator called `@qrexec_datasaver` to record and replay API calls made over qrexec. Each request made from a test and its response from the server is stored in a "cassette" in the `data` directory. Tests replay these cassettes instead of making actual API calls to a server.

If you run the tests and see the following vcrpy warning, then you'll need to re-record cassettes because none of the existing cassettes contain the expected API call and we don't allow existing cassettes to be overwritten:

Expand Down Expand Up @@ -108,36 +108,42 @@ Note: Some tests alter source and conversation data on the server so you may nee

In order to generate cassettes for tests that make API calls over qrexec, you'll need to run the server and proxy on a separate VM. If this is the first time you are generating cassettes, first follow the steps outlined in the [Test setup for qrexec communication](#test-setup-for-qrexec-communication) section, which will help you set up a new VM called `sd-dev-proxy`.

Once your proxy are set up, follow these steps:
Once your proxy VM is set up, follow these steps:

1. Start the server in a docker container on `sd-dev-proxy` by running:

```bash
NUM_SOURCES=5 make dev
```

2. [Skip if adding a new test] Delete the cassettes you wish to regenerate or just delete all json files by running:
2. Delete the cassettes you wish to regenerate or just delete all JSON files by running:

```bash
rm data/*.json
```

3. Comment out the `@dastollervey_datasaver` decorator above the test you want to generate a new cassette for or just generate all new cassettes by commenting out the decorator above all methods in the `test_apiproxy.py::TestAPIProxy` class.
If you are only adding a new test and not modifying existing ones, you can
skip this step, but you still need to remove the authentication setup during
cassette generation. Otherwise you will get 403 errors for API endpoints that
require a valid token. Remove the setup cassette by running:

4. Make qrexec calls to the server and collect real response data:
```bash
rm data/setup_method.json
```

```bash
make test TESTS=tests/test_apiproxy.py
```
(You can reinstate the unmodified version later.)

5. Uncomment the `@dastollervey_datasaver` decorator wherever you commented it out.
6. Record new cassettes from the response data collected in step 4:
3. Make qrexec calls to the server and collect response data. To run all proxy
tests:

```bash
make test TESTS=tests/test_apiproxy.py
```

**Note:** If you get a 403 error it's becuase the test is trying to reuse an old TOTP code, so wait for 60 seconds and try again. Some tests alter source and conversation data on the server so you should restart the server in between test runs.
**Note:** If you get a 403 error it may also be because the test is trying to
reuse an old TOTP code, so wait for 60 seconds and try again. Some tests alter
source and conversation data on the server so you should restart the server in
between test runs.

## Test setup for qrexec communication

Expand All @@ -151,7 +157,7 @@ If this is the first time you are generating new cassettes that make API calls o
dpkg -i <latest-package>.deb
```

3. Create `/etc/sd-proxy.yaml` with the following contents (assuming the VM you'll be running the SDK tests from is called **sd-dev**):
3. Create `/home/user/.securedrop_proxy/sd-proxy.yaml` with the following contents (assuming the VM you'll be running the SDK tests from is called **sd-dev**):

```
host: 127.0.0.1
Expand Down Expand Up @@ -201,7 +207,7 @@ sd-dev-proxy sd-dev allow
```bash
NUM_SOURCES=5 make dev
```
b. With the main branch of this repo checked out on `sd-dev`, comment out the `@dastollervey_datasaver` decorator above the `test_apiproxy.py::TestAPIProxy::setUp` method so that `test_api_auth` makes an actual API call over qrexec.
b. With the main branch of this repo checked out on `sd-dev`, comment out the `@qrexec_datasaver` decorator above the `test_apiproxy.py::TestAPIProxy::setUp` method so that `test_api_auth` makes an actual API call over qrexec.
c. Run `test_api_auth`:

```bash
Expand Down
1 change: 0 additions & 1 deletion data/setUp.json

This file was deleted.

1 change: 1 addition & 0 deletions data/setup_method.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"{\"body\": \"{\\\"passphrase\\\": \\\"correct horse battery staple profanity oil chewy\\\", \\\"username\\\": \\\"journalist\\\"}\", \"method\": \"POST\", \"path_query\": \"api/v1/token\", \"timeout\": 20}+1": "{\"status\": 200, \"body\": \"{\\n \\\"expiration\\\": \\\"2020-12-04T09:22:13.577450Z\\\", \\n \\\"journalist_first_name\\\": null, \\n \\\"journalist_last_name\\\": null, \\n \\\"journalist_uuid\\\": \\\"24289313-c416-47eb-9d2f-2f504b0ab7a2\\\", \\n \\\"token\\\": \\\"eyJpYXQiOjE2MDcwNDQ5MzMsImV4cCI6MTYwNzA3MzczMywiYWxnIjoiSFMyNTYifQ.eyJpZCI6MX0.1aFf3Qe32xpCMKCwEh7oiBiZyUKV2n3gNJ2UsxHEZkU\\\"\\n}\\n\", \"headers\": {\"Content-Type\": \"application/json\", \"Content-Length\": \"317\", \"Server\": \"Werkzeug/0.16.0 Python/3.5.2\", \"Date\": \"Fri, 04 Dec 2020 01:22:13 GMT\"}, \"version\": \"0.3.1\\n\"}"}
2 changes: 1 addition & 1 deletion data/test-badotp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ interactions:
Content-Type:
- application/json
Date:
- Wed, 14 Oct 2020 22:59:55 GMT
- Wed, 02 Dec 2020 00:56:36 GMT
Server:
- Werkzeug/0.16.0 Python/3.5.2
status:
Expand Down
4 changes: 2 additions & 2 deletions data/test-badpassword.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
interactions:
- request:
body: '{"username": "journalist", "passphrase": "no", "one_time_code": "877484"}'
body: '{"username": "journalist", "passphrase": "no", "one_time_code": "436824"}'
headers:
Accept:
- '*/*'
Expand All @@ -24,7 +24,7 @@ interactions:
Content-Type:
- application/json
Date:
- Wed, 14 Oct 2020 22:59:55 GMT
- Wed, 02 Dec 2020 00:56:36 GMT
Server:
- Werkzeug/0.16.0 Python/3.5.2
status:
Expand Down
4 changes: 2 additions & 2 deletions data/test-baduser.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
interactions:
- request:
body: '{"username": "no", "passphrase": "correct horse battery staple profanity
oil chewy", "one_time_code": "877484"}'
oil chewy", "one_time_code": "436824"}'
headers:
Accept:
- '*/*'
Expand All @@ -25,7 +25,7 @@ interactions:
Content-Type:
- application/json
Date:
- Wed, 14 Oct 2020 22:59:55 GMT
- Wed, 02 Dec 2020 00:56:36 GMT
Server:
- Werkzeug/0.16.0 Python/3.5.2
status:
Expand Down
Loading