Skip to content

Commit

Permalink
replace docker-compose with docker compose
Browse files Browse the repository at this point in the history
  • Loading branch information
elay committed Aug 12, 2024
1 parent 568a5b7 commit a6f6191
Show file tree
Hide file tree
Showing 12 changed files with 79 additions and 78 deletions.
13 changes: 7 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Microsoft Planetary Computer APIs

__Note__: This repository serves as a reference implementation for deploying APIs on Azure. This code supports the production deployment of the Planetary Computer APIs. This repository is not meant to be reusable in other situations without significant modification, and the repository maintainers will not provide any support for non-development deployments of this code.
**Note**: This repository serves as a reference implementation for deploying APIs on Azure. This code supports the production deployment of the Planetary Computer APIs. This repository is not meant to be reusable in other situations without significant modification, and the repository maintainers will not provide any support for non-development deployments of this code.

That said, feel free to crib any code that is useful!

Expand Down Expand Up @@ -30,7 +30,7 @@ For documentation of how you can deploy your own test version of these services,
After building the project locally using the instructions below, you can access the development version of the services by pointing your browser to the following URLs:

| | |
|----------------------|----------------------------------------|
| -------------------- | -------------------------------------- |
| STAC API (via nginx) | <http://localhost:8080/stac> |
| Tiler (via nginx) | <http://localhost:8080/data> |
| Funcs (vai nginx) | <http://localhost:8080/f/image>, etc.. |
Expand All @@ -41,7 +41,7 @@ After building the project locally using the instructions below, you can access
To see the HTTP endpoints available for FastAPI servers, visit the OpenAPI documentation for each service:

| | |
|-----------|-----------------------------------|
| --------- | --------------------------------- |
| STAC API | <http://localhost:8080/stac/docs> |
| Tiler API | <http://localhost:8080/data/docs> |

Expand All @@ -53,18 +53,19 @@ The development data only includes a single collection `naip`, with a few items

### Requirements

The development environment is run almost entirely through docker containers. Developing locally requires docker-compose v1.27+.
The development environment is run almost entirely through docker containers. Developing locally requires docker compose v1.27+.

### Running the Planetary Computer API services in a local development environment

This project uses a variation on [scripts to rule them all](https://github.com/github/scripts-to-rule-them-all).

#### Environment setup and building images

Before setting up the local environment, ensure that you have set the AZURITE_ACCOUNT_KEY environment variable.
Before setting up the local environment, ensure that you have set the AZURITE_ACCOUNT_KEY environment variable.
The account key can be found in the [Azurite GitHub repository](https://github.com/Azure/Azurite?tab=readme-ov-file#usage-with-azure-storage-sdks-or-tools)

For example, you can set the environment variable in your terminal with:

```console
> export AZURITE_ACCOUNT_KEY=<azurite_account_key>
```
Expand Down Expand Up @@ -125,7 +126,7 @@ It can sometimes be convenient to test against other services, e.g. a test datab
To do that, you can create a new environment file for the services based on `./pc-stac.dev.env`, `./pc-tiler.dev.env`, and/or `./pc-funcs.dev.env`.
Any environment file named similarly will be .gitignore'd, so you can leave them in your local clone and avoid
committing (e.g. `./pc-stac.testing.env`). You then need to set the `PC_STAC_ENV_FILE`, `PC_TILER_ENV_FILE`, and `PC_FUNCS_ENV_FILE` to the
environment files you want to use before running `scripts/server`. __Note__: Be careful not to run migrations
environment files you want to use before running `scripts/server`. **Note**: Be careful not to run migrations
with a non-dev database set - avoid `scripts/setup`, or ensure the migration connection is still using the local
dev database even if using a remote test db.

Expand Down
10 changes: 5 additions & 5 deletions docs/01-deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ The publicly available Planetary Computer APIs is a collection of related servic
that run as containers in Azure Kubernetes Service (AKS). Getting those
containers deployed requires a few steps, that we'll go into more below:

* Configure Azure resources
* Build and publish containers
* Create Helm apps consuming those containers
- Configure Azure resources
- Build and publish containers
- Create Helm apps consuming those containers

There are many other resources deployed in the public Planetary Computer APIs,
but we'll focus here just on the "run containers in the cloud" piece of the
Expand All @@ -33,7 +33,7 @@ configuration in [aks.tf](../deployment/terraform/resources/aks.tf).
## Build and publish containers

Building containers happens in the [`update` script](../scripts/update). You can
see there that there are a few `build` commands starting with `docker-compose`.
see there that there are a few `build` commands starting with `docker compose`.
Those commands build the containers necessary for running the services, running
migrations, and more.

Expand Down Expand Up @@ -63,4 +63,4 @@ The entire workflow for testing, building, publishing, and deploying the data
and metadata query engines is publicly viewable in this repository. You can view
workflow runs in the [`Actions`
tab](https://github.com/microsoft/planetary-computer-apis/actions/workflows/cicd.yml)
for this repo at any time.
for this repo at any time.
2 changes: 1 addition & 1 deletion scripts/cideploy
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ while [[ "$#" -gt 0 ]]; do case $1 in
;;
esac done

DOCKER_COMPOSE="docker-compose -f deployment/docker-compose.yml"
DOCKER_COMPOSE="docker compose -f deployment/docker compose.yml"

# Set terraform to deploy to dev or staging
if [ "${DEV_DEPLOY}" = "1" ]; then
Expand Down
2 changes: 1 addition & 1 deletion scripts/cipublish-func
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ while [[ "$#" -gt 0 ]]; do case $1 in
;;
esac done

DOCKER_COMPOSE="docker-compose -f deployment/docker-compose.yml"
DOCKER_COMPOSE="docker compose -f deployment/docker compose.yml"

# Set terraform to deploy to dev or staging
if [[ "${CI}" != "true" ]]; then
Expand Down
24 changes: 12 additions & 12 deletions scripts/console
Original file line number Diff line number Diff line change
Expand Up @@ -57,45 +57,45 @@ while [[ "$#" > 0 ]]; do case $1 in
if [ "${BASH_SOURCE[0]}" = "${0}" ]; then

if [[ "${DB_CONSOLE}" ]]; then
docker-compose \
-f docker-compose.yml \
docker compose \
-f docker compose.yml \
exec database psql postgres://username:password@database:5432/postgis

exit 0
fi

if [[ "${DEV_TILER_CONSOLE}" ]]; then
docker-compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
docker compose \
-f docker compose.yml \
-f docker compose.dev.yml \
run --rm \
tiler-dev /bin/bash

exit 0
fi

if [[ "${FUNCS_CONSOLE}" ]]; then
docker-compose \
-f docker-compose.yml \
docker compose \
-f docker compose.yml \
run --rm \
funcs /bin/bash

exit 0
fi

if [[ "${DEPLOY_CONSOLE}" ]]; then
GIT_COMMIT="$(git rev-parse --short HEAD)" docker-compose \
-f deployment/docker-compose.yml \
GIT_COMMIT="$(git rev-parse --short HEAD)" docker compose \
-f deployment/docker compose.yml \
run --rm \
deploy /bin/bash

exit 0
fi

if [[ "${DEV_STAC_CONSOLE}" ]]; then
docker-compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
docker compose \
-f docker compose.yml \
-f docker compose.dev.yml \
run --rm \
stac-dev /bin/bash

Expand Down
30 changes: 15 additions & 15 deletions scripts/format
Original file line number Diff line number Diff line change
Expand Up @@ -16,33 +16,33 @@ Runs formatting for the project.

if [ "${BASH_SOURCE[0]}" = "${0}" ]; then

docker-compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
docker compose \
-f docker compose.yml \
-f docker compose.dev.yml \
run --rm \
stac-dev scripts/bin/format-common;

docker-compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
docker compose \
-f docker compose.yml \
-f docker compose.dev.yml \
run --rm \
tiler-dev scripts/bin/format-tiler;

docker-compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
docker compose \
-f docker compose.yml \
-f docker compose.dev.yml \
run --rm \
stac-dev scripts/bin/format-stac;

docker-compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
docker compose \
-f docker compose.yml \
-f docker compose.dev.yml \
run --rm \
funcs /bin/bash -c "cd /opt/src && scripts/bin/format-funcs";

docker-compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
docker compose \
-f docker compose.yml \
-f docker compose.dev.yml \
run --rm \
stac-dev scripts/bin/format-scripts;

Expand Down
24 changes: 12 additions & 12 deletions scripts/generate-requirements
Original file line number Diff line number Diff line change
Expand Up @@ -5,30 +5,30 @@ set -e
echo "== Generating requirements.txt ..."
pip_compile_options="--emit-index-url --annotate --verbose"

docker-compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
docker compose \
-f docker compose.yml \
-f docker compose.dev.yml \
run --rm \
stac-dev \
pip-compile ./pcstac/setup.py --extra server -o pcstac/requirements-server.txt $pip_compile_options

docker-compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
docker compose \
-f docker compose.yml \
-f docker compose.dev.yml \
run --rm \
tiler-dev \
pip-compile ./pctiler/setup.py --extra server -o pctiler/requirements-server.txt $pip_compile_options

docker-compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
docker compose \
-f docker compose.yml \
-f docker compose.dev.yml \
run --rm \
tiler-dev \
pip-compile ./pctiler/setup.py --extra dev -o pctiler/requirements-dev.txt $pip_compile_options

docker-compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
docker compose \
-f docker compose.yml \
-f docker compose.dev.yml \
run --rm \
tiler-dev \
pip-compile ./pccommon/setup.py --extra server -o pccommon/requirements.txt $pip_compile_options
4 changes: 2 additions & 2 deletions scripts/migrate
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ Run migrations against the development database.
if [ "${BASH_SOURCE[0]}" = "${0}" ]; then

# Run database migrations
docker-compose \
-f docker-compose.yml \
docker compose \
-f docker compose.yml \
run --rm stac \
bash -c "pypgstac pgready && pypgstac migrate"

Expand Down
4 changes: 2 additions & 2 deletions scripts/server
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ while [[ "$#" > 0 ]]; do case $1 in
esac; done

if [ "${BASH_SOURCE[0]}" = "${0}" ]; then
docker-compose \
-f docker-compose.yml \
docker compose \
-f docker compose.yml \
up ${DETACHED_ARG} $@;
fi
10 changes: 5 additions & 5 deletions scripts/setup
Original file line number Diff line number Diff line change
Expand Up @@ -27,17 +27,17 @@ if [ "${BASH_SOURCE[0]}" = "${0}" ]; then

echo "Ingesting development data..."

docker-compose \
-f docker-compose.yml \
docker compose \
-f docker compose.yml \
run --rm \
stac \
python3 /opt/src/pcstac/tests/loadtestdata.py

echo "Setting up azurite..."

docker-compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
docker compose \
-f docker compose.yml \
-f docker compose.dev.yml \
run --rm \
stac-dev \
python3 /opt/src/scripts/bin/setup_azurite.py
Expand Down
22 changes: 11 additions & 11 deletions scripts/test
Original file line number Diff line number Diff line change
Expand Up @@ -65,34 +65,34 @@ if [ "${BASH_SOURCE[0]}" = "${0}" ]; then

if [ -z "${TILER_ONLY}${STAC_ONLY}${FUNCS_ONLY}" ]; then

docker-compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
docker compose \
-f docker compose.yml \
-f docker compose.dev.yml \
run --rm \
stac-dev scripts/bin/test-common
fi

if [ -z "${STAC_ONLY}${COMMON_ONLY}${FUNCS_ONLY}" ]; then
docker-compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
docker compose \
-f docker compose.yml \
-f docker compose.dev.yml \
run --rm \
tiler-dev scripts/bin/test-tiler ${NO_INTEGRATION}
fi

if [ -z "${TILER_ONLY}${COMMON_ONLY}${FUNCS_ONLY}" ]; then

docker-compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
docker compose \
-f docker compose.yml \
-f docker compose.dev.yml \
run --rm \
stac-dev scripts/bin/test-stac
fi

if [ -z "${STAC_ONLY}${TILER_ONLY}${COMMON_ONLY}" ]; then

docker-compose \
-f docker-compose.yml \
docker compose \
-f docker compose.yml \
run --rm \
funcs /bin/bash -c "cd /opt/src && scripts/bin/test-funcs ${NO_INTEGRATION}"
fi
Expand Down
12 changes: 6 additions & 6 deletions scripts/update
Original file line number Diff line number Diff line change
Expand Up @@ -29,16 +29,16 @@ if [ "${BASH_SOURCE[0]}" = "${0}" ]; then

if [ "${RESTART_SERVERS}" ]; then
echo "==Bringing down servers.."
docker-compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
docker compose \
-f docker compose.yml \
-f docker compose.dev.yml \
down --remove-orphans
fi

echo "==Building images..."
docker-compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
docker compose \
-f docker compose.yml \
-f docker compose.dev.yml \
build ${NO_CACHE}

fi

0 comments on commit a6f6191

Please sign in to comment.