diff --git a/README.md b/README.md index 30a7bbf3..4e484203 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Microsoft Planetary Computer APIs -__Note__: This repository serves as a reference implementation for deploying APIs on Azure. This code supports the production deployment of the Planetary Computer APIs. This repository is not meant to be reusable in other situations without significant modification, and the repository maintainers will not provide any support for non-development deployments of this code. +**Note**: This repository serves as a reference implementation for deploying APIs on Azure. This code supports the production deployment of the Planetary Computer APIs. This repository is not meant to be reusable in other situations without significant modification, and the repository maintainers will not provide any support for non-development deployments of this code. That said, feel free to crib any code that is useful! @@ -30,7 +30,7 @@ For documentation of how you can deploy your own test version of these services, After building the project locally using the instructions below, you can access the development version of the services by pointing your browser to the following URLs: | | | -|----------------------|----------------------------------------| +| -------------------- | -------------------------------------- | | STAC API (via nginx) | | | Tiler (via nginx) | | | Funcs (vai nginx) | , etc.. | @@ -41,7 +41,7 @@ After building the project locally using the instructions below, you can access To see the HTTP endpoints available for FastAPI servers, visit the OpenAPI documentation for each service: | | | -|-----------|-----------------------------------| +| --------- | --------------------------------- | | STAC API | | | Tiler API | | @@ -53,7 +53,7 @@ The development data only includes a single collection `naip`, with a few items ### Requirements -The development environment is run almost entirely through docker containers. Developing locally requires docker-compose v1.27+. +The development environment is run almost entirely through docker containers. Developing locally requires docker compose v1.27+. ### Running the Planetary Computer API services in a local development environment @@ -61,10 +61,11 @@ This project uses a variation on [scripts to rule them all](https://github.com/g #### Environment setup and building images -Before setting up the local environment, ensure that you have set the AZURITE_ACCOUNT_KEY environment variable. +Before setting up the local environment, ensure that you have set the AZURITE_ACCOUNT_KEY environment variable. The account key can be found in the [Azurite GitHub repository](https://github.com/Azure/Azurite?tab=readme-ov-file#usage-with-azure-storage-sdks-or-tools) For example, you can set the environment variable in your terminal with: + ```console > export AZURITE_ACCOUNT_KEY= ``` @@ -125,7 +126,7 @@ It can sometimes be convenient to test against other services, e.g. a test datab To do that, you can create a new environment file for the services based on `./pc-stac.dev.env`, `./pc-tiler.dev.env`, and/or `./pc-funcs.dev.env`. Any environment file named similarly will be .gitignore'd, so you can leave them in your local clone and avoid committing (e.g. `./pc-stac.testing.env`). You then need to set the `PC_STAC_ENV_FILE`, `PC_TILER_ENV_FILE`, and `PC_FUNCS_ENV_FILE` to the -environment files you want to use before running `scripts/server`. __Note__: Be careful not to run migrations +environment files you want to use before running `scripts/server`. **Note**: Be careful not to run migrations with a non-dev database set - avoid `scripts/setup`, or ensure the migration connection is still using the local dev database even if using a remote test db. diff --git a/docs/01-deployment.md b/docs/01-deployment.md index a245422b..d3c29898 100644 --- a/docs/01-deployment.md +++ b/docs/01-deployment.md @@ -8,9 +8,9 @@ The publicly available Planetary Computer APIs is a collection of related servic that run as containers in Azure Kubernetes Service (AKS). Getting those containers deployed requires a few steps, that we'll go into more below: -* Configure Azure resources -* Build and publish containers -* Create Helm apps consuming those containers +- Configure Azure resources +- Build and publish containers +- Create Helm apps consuming those containers There are many other resources deployed in the public Planetary Computer APIs, but we'll focus here just on the "run containers in the cloud" piece of the @@ -33,7 +33,7 @@ configuration in [aks.tf](../deployment/terraform/resources/aks.tf). ## Build and publish containers Building containers happens in the [`update` script](../scripts/update). You can -see there that there are a few `build` commands starting with `docker-compose`. +see there that there are a few `build` commands starting with `docker compose`. Those commands build the containers necessary for running the services, running migrations, and more. @@ -63,4 +63,4 @@ The entire workflow for testing, building, publishing, and deploying the data and metadata query engines is publicly viewable in this repository. You can view workflow runs in the [`Actions` tab](https://github.com/microsoft/planetary-computer-apis/actions/workflows/cicd.yml) -for this repo at any time. \ No newline at end of file +for this repo at any time. diff --git a/scripts/cideploy b/scripts/cideploy index 3954187f..0d23a6b6 100755 --- a/scripts/cideploy +++ b/scripts/cideploy @@ -47,7 +47,7 @@ while [[ "$#" -gt 0 ]]; do case $1 in ;; esac done -DOCKER_COMPOSE="docker-compose -f deployment/docker-compose.yml" +DOCKER_COMPOSE="docker compose -f deployment/docker compose.yml" # Set terraform to deploy to dev or staging if [ "${DEV_DEPLOY}" = "1" ]; then diff --git a/scripts/cipublish-func b/scripts/cipublish-func index 90a6ef23..67716ce5 100755 --- a/scripts/cipublish-func +++ b/scripts/cipublish-func @@ -30,7 +30,7 @@ while [[ "$#" -gt 0 ]]; do case $1 in ;; esac done -DOCKER_COMPOSE="docker-compose -f deployment/docker-compose.yml" +DOCKER_COMPOSE="docker compose -f deployment/docker compose.yml" # Set terraform to deploy to dev or staging if [[ "${CI}" != "true" ]]; then diff --git a/scripts/console b/scripts/console index ab6f5b74..38be6717 100755 --- a/scripts/console +++ b/scripts/console @@ -57,17 +57,17 @@ while [[ "$#" > 0 ]]; do case $1 in if [ "${BASH_SOURCE[0]}" = "${0}" ]; then if [[ "${DB_CONSOLE}" ]]; then - docker-compose \ - -f docker-compose.yml \ + docker compose \ + -f docker compose.yml \ exec database psql postgres://username:password@database:5432/postgis exit 0 fi if [[ "${DEV_TILER_CONSOLE}" ]]; then - docker-compose \ - -f docker-compose.yml \ - -f docker-compose.dev.yml \ + docker compose \ + -f docker compose.yml \ + -f docker compose.dev.yml \ run --rm \ tiler-dev /bin/bash @@ -75,8 +75,8 @@ if [ "${BASH_SOURCE[0]}" = "${0}" ]; then fi if [[ "${FUNCS_CONSOLE}" ]]; then - docker-compose \ - -f docker-compose.yml \ + docker compose \ + -f docker compose.yml \ run --rm \ funcs /bin/bash @@ -84,8 +84,8 @@ if [ "${BASH_SOURCE[0]}" = "${0}" ]; then fi if [[ "${DEPLOY_CONSOLE}" ]]; then - GIT_COMMIT="$(git rev-parse --short HEAD)" docker-compose \ - -f deployment/docker-compose.yml \ + GIT_COMMIT="$(git rev-parse --short HEAD)" docker compose \ + -f deployment/docker compose.yml \ run --rm \ deploy /bin/bash @@ -93,9 +93,9 @@ if [ "${BASH_SOURCE[0]}" = "${0}" ]; then fi if [[ "${DEV_STAC_CONSOLE}" ]]; then - docker-compose \ - -f docker-compose.yml \ - -f docker-compose.dev.yml \ + docker compose \ + -f docker compose.yml \ + -f docker compose.dev.yml \ run --rm \ stac-dev /bin/bash diff --git a/scripts/format b/scripts/format index 555b6b60..ef8ed0f3 100755 --- a/scripts/format +++ b/scripts/format @@ -16,33 +16,33 @@ Runs formatting for the project. if [ "${BASH_SOURCE[0]}" = "${0}" ]; then - docker-compose \ - -f docker-compose.yml \ - -f docker-compose.dev.yml \ + docker compose \ + -f docker compose.yml \ + -f docker compose.dev.yml \ run --rm \ stac-dev scripts/bin/format-common; - docker-compose \ - -f docker-compose.yml \ - -f docker-compose.dev.yml \ + docker compose \ + -f docker compose.yml \ + -f docker compose.dev.yml \ run --rm \ tiler-dev scripts/bin/format-tiler; - docker-compose \ - -f docker-compose.yml \ - -f docker-compose.dev.yml \ + docker compose \ + -f docker compose.yml \ + -f docker compose.dev.yml \ run --rm \ stac-dev scripts/bin/format-stac; - docker-compose \ - -f docker-compose.yml \ - -f docker-compose.dev.yml \ + docker compose \ + -f docker compose.yml \ + -f docker compose.dev.yml \ run --rm \ funcs /bin/bash -c "cd /opt/src && scripts/bin/format-funcs"; - docker-compose \ - -f docker-compose.yml \ - -f docker-compose.dev.yml \ + docker compose \ + -f docker compose.yml \ + -f docker compose.dev.yml \ run --rm \ stac-dev scripts/bin/format-scripts; diff --git a/scripts/generate-requirements b/scripts/generate-requirements index 26293130..6ceac211 100755 --- a/scripts/generate-requirements +++ b/scripts/generate-requirements @@ -5,30 +5,30 @@ set -e echo "== Generating requirements.txt ..." pip_compile_options="--emit-index-url --annotate --verbose" -docker-compose \ - -f docker-compose.yml \ - -f docker-compose.dev.yml \ +docker compose \ + -f docker compose.yml \ + -f docker compose.dev.yml \ run --rm \ stac-dev \ pip-compile ./pcstac/setup.py --extra server -o pcstac/requirements-server.txt $pip_compile_options -docker-compose \ - -f docker-compose.yml \ - -f docker-compose.dev.yml \ +docker compose \ + -f docker compose.yml \ + -f docker compose.dev.yml \ run --rm \ tiler-dev \ pip-compile ./pctiler/setup.py --extra server -o pctiler/requirements-server.txt $pip_compile_options -docker-compose \ - -f docker-compose.yml \ - -f docker-compose.dev.yml \ +docker compose \ + -f docker compose.yml \ + -f docker compose.dev.yml \ run --rm \ tiler-dev \ pip-compile ./pctiler/setup.py --extra dev -o pctiler/requirements-dev.txt $pip_compile_options -docker-compose \ - -f docker-compose.yml \ - -f docker-compose.dev.yml \ +docker compose \ + -f docker compose.yml \ + -f docker compose.dev.yml \ run --rm \ tiler-dev \ pip-compile ./pccommon/setup.py --extra server -o pccommon/requirements.txt $pip_compile_options diff --git a/scripts/migrate b/scripts/migrate index be8ad3cd..8ddba538 100755 --- a/scripts/migrate +++ b/scripts/migrate @@ -16,8 +16,8 @@ Run migrations against the development database. if [ "${BASH_SOURCE[0]}" = "${0}" ]; then # Run database migrations - docker-compose \ - -f docker-compose.yml \ + docker compose \ + -f docker compose.yml \ run --rm stac \ bash -c "pypgstac pgready && pypgstac migrate" diff --git a/scripts/server b/scripts/server index a7edee02..7a667a6c 100755 --- a/scripts/server +++ b/scripts/server @@ -26,7 +26,7 @@ while [[ "$#" > 0 ]]; do case $1 in esac; done if [ "${BASH_SOURCE[0]}" = "${0}" ]; then - docker-compose \ - -f docker-compose.yml \ + docker compose \ + -f docker compose.yml \ up ${DETACHED_ARG} $@; fi diff --git a/scripts/setup b/scripts/setup index 31bcc3c1..bcae53fd 100755 --- a/scripts/setup +++ b/scripts/setup @@ -27,17 +27,17 @@ if [ "${BASH_SOURCE[0]}" = "${0}" ]; then echo "Ingesting development data..." - docker-compose \ - -f docker-compose.yml \ + docker compose \ + -f docker compose.yml \ run --rm \ stac \ python3 /opt/src/pcstac/tests/loadtestdata.py echo "Setting up azurite..." - docker-compose \ - -f docker-compose.yml \ - -f docker-compose.dev.yml \ + docker compose \ + -f docker compose.yml \ + -f docker compose.dev.yml \ run --rm \ stac-dev \ python3 /opt/src/scripts/bin/setup_azurite.py diff --git a/scripts/test b/scripts/test index 8f283525..8f268212 100755 --- a/scripts/test +++ b/scripts/test @@ -65,34 +65,34 @@ if [ "${BASH_SOURCE[0]}" = "${0}" ]; then if [ -z "${TILER_ONLY}${STAC_ONLY}${FUNCS_ONLY}" ]; then - docker-compose \ - -f docker-compose.yml \ - -f docker-compose.dev.yml \ + docker compose \ + -f docker compose.yml \ + -f docker compose.dev.yml \ run --rm \ stac-dev scripts/bin/test-common fi if [ -z "${STAC_ONLY}${COMMON_ONLY}${FUNCS_ONLY}" ]; then - docker-compose \ - -f docker-compose.yml \ - -f docker-compose.dev.yml \ + docker compose \ + -f docker compose.yml \ + -f docker compose.dev.yml \ run --rm \ tiler-dev scripts/bin/test-tiler ${NO_INTEGRATION} fi if [ -z "${TILER_ONLY}${COMMON_ONLY}${FUNCS_ONLY}" ]; then - docker-compose \ - -f docker-compose.yml \ - -f docker-compose.dev.yml \ + docker compose \ + -f docker compose.yml \ + -f docker compose.dev.yml \ run --rm \ stac-dev scripts/bin/test-stac fi if [ -z "${STAC_ONLY}${TILER_ONLY}${COMMON_ONLY}" ]; then - docker-compose \ - -f docker-compose.yml \ + docker compose \ + -f docker compose.yml \ run --rm \ funcs /bin/bash -c "cd /opt/src && scripts/bin/test-funcs ${NO_INTEGRATION}" fi diff --git a/scripts/update b/scripts/update index 31fd7714..c0694187 100755 --- a/scripts/update +++ b/scripts/update @@ -29,16 +29,16 @@ if [ "${BASH_SOURCE[0]}" = "${0}" ]; then if [ "${RESTART_SERVERS}" ]; then echo "==Bringing down servers.." - docker-compose \ - -f docker-compose.yml \ - -f docker-compose.dev.yml \ + docker compose \ + -f docker compose.yml \ + -f docker compose.dev.yml \ down --remove-orphans fi echo "==Building images..." - docker-compose \ - -f docker-compose.yml \ - -f docker-compose.dev.yml \ + docker compose \ + -f docker compose.yml \ + -f docker compose.dev.yml \ build ${NO_CACHE} fi