Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: K8s and Tests #38

Merged
merged 1 commit into from
May 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion docs/deploy.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ podman run -it -p 8080:8080 --env-file .env -v LOCAL_DB_PATH:DB_FILE gantry

Where `LOCAL_DB_PATH` is the absolute path to the database file on your local system. `DB_FILE` is where you would like the application to access the database. Make sure this lines up with your environment.

When running Gantry within Kubernetes, you could use [persistent volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/). The only requirement is that the database should exist outside the container for backup and persistence purposes.
When running Gantry within Kubernetes, you could use [persistent volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/). The only requirement is that a copy of the database should exist outside the container for backup and persistence purposes.

## Environment

Expand All @@ -28,3 +28,15 @@ The following variables should be exposed to the container. Those **bolded** are
- **`GITLAB_API_TOKEN`** - this token should have API read access
- **`GITLAB_WEBHOOK_TOKEN`** - coordinate this value with the collection webhook
- **`DB_FILE`** - path where the application can access the SQLite file

## Kubernetes

Gantry can be deployed in a Kubernetes cluster. For an example of how we do this within Spack's web infrastructure, see [this folder](https://github.com/spack/spack-infrastructure/tree/main/k8s/production/spack/gantry-spack-io) containing configuration files and instructions.

While most details are better suited to be documented with the cluster, there are some considerations that are general to any Gantry deployment.

### Database

We have made an architectural decision to depend on SQLite as the database engine. Before you deploy Gantry into a cluster, you should ensure that the file will be backed up on a regular basis, in the case that unexpected circumstances corrupt your data. This can be achieved using [Litestream](https://litestream.io), which will continuously replicate the database with the storage provider of your choice. See the cluster configuration linked above for details.

When you first deploy the application on the cluster, be sure to run `/db/init_db.py <db_path>` to initialize an SQLite file with default tables, and to apply any migrations.
5 changes: 3 additions & 2 deletions docs/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,6 @@
2. [Data Collection](data-collection.md)
3. [Architecture](arch.md)
4. [Prediction](prediction.md)
5. [Deployment](deploy.md)
6. [API Endpoints](api.md)
5. [Testing](testing.md)
6. [Deployment](deploy.md)
7. [API Endpoints](api.md)
19 changes: 19 additions & 0 deletions docs/testing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Testing

Gantry uses `pytest` to manage its testing suite. Make sure you have installed `pytest`, `pytest-aiohttp`, and `pytest-mock` in your environment.

The tests cover the collection and prediction functionalities, as well as core functionalities like helper functions and database actions.

Running all tests:

```
python -m pytest -s gantry
```

If you would like to see details about test coverage, install `coverage` and run:

```
coverage run -m pytest -s gantry && coverage html
```

This will create a folder in the top-level directory containing an `index.html` file which you can open in a browser.