forked from 2i2c-org/infrastructure
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Cleanup documentation on deploying from your laptop
Ref 2i2c-org#351
- Loading branch information
Showing
1 changed file
with
61 additions
and
39 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,52 +1,74 @@ | ||
| # Manually deploy a hub | ||
| # Set up and use the the deployment scripts locally | ||
|
|
||
| The [`deploy.py`](https://github.com/2i2c-org/pilot-hubs/blob/master/deploy.py) script is a cmd utility built to help deploy one or all the hubs to their Kubernetes clusters, but also to build the user image. Checkout the [](../configure/update-env.md) section on how to use the deploy script to update the user environment. | ||
| While deploys generally go through our GitHub Actions workflow, sometimes you | ||
| need to deploy from your laptop - primarily when testing changes on staging or | ||
| actively fixing an outage. This isn't ideal, but this is where we are now. | ||
|
|
||
| ## Deploy script arguments and usage | ||
| Running: | ||
| ``` | ||
| python3 deploy.py deploy --help | ||
| ``` | ||
| Will provide information about the `deploy.py` deploy options: | ||
| Our deployment scripts live in the [`deploy/`](https://github.com/2i2c-org/pilot-hubs/blob/master/deploy.py) | ||
| of this repository, and can deploy one or more hubs to our clusters. | ||
|
|
||
| ```bash | ||
| usage: deploy.py deploy [-h] [--skip-hub-health-test] | ||
| [cluster_name] [hub_name] | ||
|
|
||
| positional arguments: | ||
| cluster_name | ||
| hub_name | ||
| ## Setting up local environment | ||
|
|
||
| optional arguments: | ||
| -h, --help show this help message and exit | ||
| --skip-hub-health-test | ||
| 1. Create a virtual environment for use with this repository | ||
|
|
||
| ``` | ||
| ```bash | ||
| python3 -m venv . | ||
| source bin/activate | ||
| ``` | ||
|
|
||
| * `cluster_name` argument | ||
| You can also use `conda` if you prefer instead. | ||
|
|
||
| Deploys all the hubs in the cluster passed as argument. Cluster config is read | ||
| from `config/hubs/{cluster_name}.cluster.yaml` | ||
| ```bash | ||
| python3 deploy.py deploy <cluster> | ||
| ``` | ||
| * `hub_name` argument | ||
| 2. Install python packages required by our deployment script | ||
|
|
||
| ```bash | ||
| pip install -r requirements.txt | ||
| ``` | ||
|
|
||
| Deploys only the hub passed as argument. | ||
| 3. Install the [Google Cloud SDK](https://cloud.google.com/sdk) so | ||
| our scripts can authenticate to Google Cloud Platform for various | ||
| functions. | ||
|
|
||
| ```bash | ||
| python3 deploy.py deploy <cluster> <hub> | ||
| ``` | ||
| * `--skip-hub-health-test` flag | ||
| 4. Authenticate to Google Cloud Platform! First, you should run | ||
| `gcloud auth login` and follow the prompts - this authenticates your | ||
| `gcloud` commandline tool. Next, you should run `gcloud auth application-default login`, | ||
| and follow the prompts - this authenticates you to any tools that | ||
| want to authenticate to Google Cloud *on your behalf* - including | ||
| our deploy scripts! See [Application Default Credentials](https://cloud.google.com/docs/authentication/production#automatically) | ||
| for more information. | ||
|
|
||
| Skips the default health checks ran by the deploy script. Checkout [testing the hub section](test-hub.md) to learn more about the hub health tests. | ||
| ```bash | ||
| python3 deploy.py deploy --skip-hub-health-test | ||
| ``` | ||
| 5. We use [`sops`](https://github.com/mozilla/sops) for secret management. | ||
| We use `sops` with [Google KMS key](https://cloud.google.com/security-key-management) to | ||
| automatically encrypt/decrypt files when needed. So access to | ||
| the KMS key grants access to the secrets. Setting up the | ||
| application default login in step 4 means sops can use that | ||
| to decrypt the secrets when necessary. The key currently in use is | ||
| in the `two-eye-two-see` GCP project, so you must already have | ||
| access to it to decrypt the files | ||
|
|
||
| ### Authentication | ||
| You can test it by running `sops config/secrets.yaml`, and checking | ||
| if your `$EDITOR` pops up with the decrypted contents of the secret | ||
| file. | ||
|
|
||
| Secret keys required to deploy hubs are located in `config/secrets.yaml`, and encrypted | ||
| via [`sops`](https://github.com/mozilla/sops/). You must have acces to the `two-eye-two-see` | ||
| GCP project, and [set up sops](https://github.com/mozilla/sops#23encrypting-using-gcp-kms) | ||
| to be usable with it before you can do deployments. | ||
| ## Doing a deploy | ||
|
|
||
| 1. Make the [config change](../topic/config.md) you want to deploy. | ||
|
|
||
| 2. Deploy just a single hub: | ||
|
|
||
| ```bash | ||
| python3 deploy deploy <cluster-name> <hub-name> | ||
| ``` | ||
|
|
||
| The script will look for a hub named `<hub-name>` in the cluster config | ||
| defined at `config/hubs/<cluster-name>.cluster.yaml`. | ||
|
|
||
| 3. You can deploy to *all* hubs on a given cluster by omitting the hub name. | ||
|
|
||
| ```bash | ||
| python3 deploy deploy <cluster-name> | ||
| ``` | ||
|
|
||
| ```{note} | ||
| You should mostly use the `staging` hub in the `2i2c` cluster for testing. | ||
| ``` |