Skip to content

Commit

Permalink
Merge pull request #54 from br3ndonland/buildkit
Browse files Browse the repository at this point in the history
Use BuildKit for Docker builds
  • Loading branch information
br3ndonland authored Jul 9, 2022
2 parents 0452dde + 79ce14e commit 5968ae2
Show file tree
Hide file tree
Showing 4 changed files with 110 additions and 19 deletions.
6 changes: 5 additions & 1 deletion .github/workflows/builds.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ on:
workflow_dispatch:

env:
DOCKER_BUILDKIT: "1"
PIPX_VERSION: "1.1.0"
POETRY_VERSION: "1.1.11"

Expand Down Expand Up @@ -109,19 +110,22 @@ jobs:
- name: Build Docker images
run: |
docker build . --rm --target base \
--build-arg BUILDKIT_INLINE_CACHE=1 \
--build-arg LINUX_VERSION="$LINUX_VERSION" \
--build-arg PIPX_VERSION="$PIPX_VERSION" \
--build-arg POETRY_VERSION="$POETRY_VERSION" \
--build-arg PYTHON_VERSION="$PYTHON_VERSION" \
--cache-from python:"$PYTHON_VERSION$LINUX_TAG" \
--cache-from ghcr.io/br3ndonland/inboard \
-t ghcr.io/br3ndonland/inboard:base"$LINUX_TAG"
docker build . --rm --target starlette \
--build-arg BUILDKIT_INLINE_CACHE=1 \
--build-arg LINUX_VERSION="$LINUX_VERSION" \
--build-arg PIPX_VERSION="$PIPX_VERSION" \
--build-arg POETRY_VERSION="$POETRY_VERSION" \
--build-arg PYTHON_VERSION="$PYTHON_VERSION" \
-t ghcr.io/br3ndonland/inboard:starlette"$LINUX_TAG"
docker build . --rm --target fastapi \
--build-arg BUILDKIT_INLINE_CACHE=1 \
--build-arg LINUX_VERSION="$LINUX_VERSION" \
--build-arg PIPX_VERSION="$PIPX_VERSION" \
--build-arg POETRY_VERSION="$POETRY_VERSION" \
Expand Down
63 changes: 51 additions & 12 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# syntax=docker/dockerfile:1
ARG PYTHON_VERSION=3.10 LINUX_VERSION=
FROM python:${PYTHON_VERSION}${LINUX_VERSION:+-$LINUX_VERSION} AS base
FROM python:${PYTHON_VERSION}${LINUX_VERSION:+-$LINUX_VERSION} AS builder
LABEL org.opencontainers.image.authors="Brendon Smith <[email protected]>"
LABEL org.opencontainers.image.description="Docker images and utilities to power your Python APIs and help you ship faster."
LABEL org.opencontainers.image.licenses="MIT"
Expand All @@ -8,21 +9,59 @@ LABEL org.opencontainers.image.title="inboard"
LABEL org.opencontainers.image.url="https://github.com/br3ndonland/inboard/pkgs/container/inboard"
ARG LINUX_VERSION PIPX_VERSION=1.1.0 POETRY_VERSION=1.1.11
ENV APP_MODULE=inboard.app.main_base:app LINUX_VERSION=$LINUX_VERSION PATH=/opt/pipx/bin:/app/.venv/bin:$PATH PIPX_BIN_DIR=/opt/pipx/bin PIPX_HOME=/opt/pipx/home PIPX_VERSION=$PIPX_VERSION POETRY_VERSION=$POETRY_VERSION PYTHONPATH=/app
COPY poetry.lock poetry.toml pyproject.toml /app/
COPY --link poetry.lock poetry.toml pyproject.toml /app/
WORKDIR /app
RUN sh -c 'if [ "$LINUX_VERSION" = "slim" ]; then apt-get update -qy && apt-get install -qy --no-install-recommends gcc libc-dev make wget; fi' && \
sh -c '. /etc/os-release; if [ "$ID" = "alpine" ]; then apk add --no-cache --virtual .build-deps gcc libc-dev libffi-dev make openssl-dev; fi' && \
python -m pip install --no-cache-dir --upgrade pip "pipx==$PIPX_VERSION" && pipx install "poetry==$POETRY_VERSION" && poetry install --no-dev --no-interaction --no-root && \
sh -c 'if [ "$LINUX_VERSION" = "slim" ]; then apt-get purge --auto-remove -qy gcc libc-dev make wget; fi' && \
sh -c '. /etc/os-release; if [ "$ID" = "alpine" ]; then apk del .build-deps; fi'
COPY inboard /app/inboard
RUN <<HEREDOC
. /etc/os-release
if [ "$ID" = "alpine" ]; then
apk add --no-cache --virtual .build-deps \
gcc libc-dev libffi-dev make openssl-dev
elif [ "$LINUX_VERSION" = "slim" ]; then
apt-get update -qy
apt-get install -qy --no-install-recommends \
gcc libc-dev make wget
fi
python -m pip install --no-cache-dir --upgrade pip "pipx==$PIPX_VERSION"
pipx install "poetry==$POETRY_VERSION"
poetry install --no-dev --no-interaction --no-root
HEREDOC
COPY --link inboard /app/inboard
ENTRYPOINT ["python"]
CMD ["-m", "inboard.start"]

FROM base AS fastapi
FROM builder as base
RUN <<HEREDOC
. /etc/os-release
if [ "$ID" = "alpine" ]; then
apk del .build-deps
elif [ "$LINUX_VERSION" = "slim" ]; then
apt-get purge --auto-remove -qy \
gcc libc-dev make wget
fi
HEREDOC

FROM builder AS fastapi
ENV APP_MODULE=inboard.app.main_fastapi:app
RUN poetry install --no-dev --no-interaction --no-root -E fastapi
RUN <<HEREDOC
poetry install --no-dev --no-interaction --no-root -E fastapi
. /etc/os-release
if [ "$ID" = "alpine" ]; then
apk del .build-deps
elif [ "$LINUX_VERSION" = "slim" ]; then
apt-get purge --auto-remove -qy \
gcc libc-dev make wget
fi
HEREDOC

FROM base AS starlette
FROM builder AS starlette
ENV APP_MODULE=inboard.app.main_starlette:app
RUN poetry install --no-dev --no-interaction --no-root -E starlette
RUN <<HEREDOC
poetry install --no-dev --no-interaction --no-root -E starlette
. /etc/os-release
if [ "$ID" = "alpine" ]; then
apk del .build-deps
elif [ "$LINUX_VERSION" = "slim" ]; then
apt-get purge --auto-remove -qy \
gcc libc-dev make wget
fi
HEREDOC
4 changes: 4 additions & 0 deletions docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,13 +167,17 @@ docker cp [container_name]:/path/to/file destination.file

### Building development images

Note that Docker builds use BuildKit. See the [BuildKit docs](https://github.com/moby/buildkit/blob/HEAD/frontend/dockerfile/docs/syntax.md) and [Docker docs](https://docs.docker.com/develop/develop-images/build_enhancements/).

To build the Docker images for each stage:

```sh
git clone [email protected]:br3ndonland/inboard.git
cd inboard
export DOCKER_BUILDKIT=1
docker build . --rm --target base -t localhost/br3ndonland/inboard:base && \
docker build . --rm --target fastapi -t localhost/br3ndonland/inboard:fastapi && \
docker build . --rm --target starlette -t localhost/br3ndonland/inboard:starlette
Expand Down
56 changes: 50 additions & 6 deletions docs/docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,14 +241,29 @@ The basic build dependencies used by inboard include `gcc`, `libc-dev`, and `mak
!!! example "Example Alpine Linux _Dockerfile_ for PostgreSQL project"

```dockerfile
# syntax=docker/dockerfile:1
ARG INBOARD_DOCKER_TAG=fastapi-alpine
FROM ghcr.io/br3ndonland/inboard:${INBOARD_DOCKER_TAG}
ENV APP_MODULE=mypackage.main:app
COPY poetry.lock pyproject.toml /app/
WORKDIR /app
RUN sh -c '. /etc/os-release; if [ "$ID" = "alpine" ]; then apk add --no-cache --virtual .build-project build-base freetype-dev gcc libc-dev libpng-dev make openblas-dev postgresql-dev; fi' && \
poetry install --no-dev --no-interaction --no-root && \
sh -c '. /etc/os-release; if [ "$ID" = "alpine" ]; then apk del .build-project && apk add --no-cache libpq; fi'
RUN <<HEREDOC

. /etc/os-release

if [ "$ID" = "alpine" ]; then
apk add --no-cache --virtual .build-project \
build-base freetype-dev gcc libc-dev libpng-dev make openblas-dev postgresql-dev
fi

poetry install --no-dev --no-interaction --no-root

if [ "$ID" = "alpine" ]; then
apk del .build-project
apk add --no-cache libpq
fi

HEREDOC
COPY mypackage /app/mypackage
```

Expand Down Expand Up @@ -277,18 +292,47 @@ A _Dockerfile_ equivalent to the Alpine Linux example might look like the follow
!!! example "Example Debian Linux slim _Dockerfile_ for PostgreSQL project"

```dockerfile
# syntax=docker/dockerfile:1
ARG INBOARD_DOCKER_TAG=fastapi-slim
FROM ghcr.io/br3ndonland/inboard:${INBOARD_DOCKER_TAG}
ENV APP_MODULE=mypackage.main:app
COPY poetry.lock pyproject.toml /app/
WORKDIR /app
ARG INBOARD_DOCKER_TAG
RUN sh -c '. /etc/os-release; if [ "$ID" = "debian" ] && echo "$INBOARD_DOCKER_TAG" | grep -q "slim"; then apt-get update -qy && apt-get install -qy --no-install-recommends gcc libc-dev libpq-dev make wget; fi' && \
poetry install --no-dev --no-interaction --no-root && \
sh -c '. /etc/os-release; if [ "$ID" = "debian" ] && echo "$INBOARD_DOCKER_TAG" | grep -q "slim"; then apt-get purge --auto-remove -qy gcc libc-dev make wget; fi'
RUN <<HEREDOC

. /etc/os-release

if [ "$ID" = "debian" ] && echo "$INBOARD_DOCKER_TAG" | grep -q "slim"; then
apt-get update -qy
apt-get install -qy --no-install-recommends \
gcc libc-dev make wget
fi

poetry install --no-dev --no-interaction --no-root

if [ "$ID" = "debian" ] && echo "$INBOARD_DOCKER_TAG" | grep -q "slim"; then
apt-get purge --auto-remove -qy \
gcc libc-dev make wget
fi

HEREDOC
COPY mypackage /app/mypackage
```

!!! info "Redeclaring Docker build arguments"

Why is `ARG INBOARD_DOCKER_TAG` repeated in the example above? To understand this, it is necessary to [understand how `ARG` and `FROM` interact](https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact). Any `ARG`s before `FROM` are outside the Docker build context. In order to use them again inside the build context, they must be redeclared.

!!! tip "Here-documents in Dockerfiles"

The `RUN` commands in the Dockerfiles above use a special syntax called a [here-document](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07), or "heredoc". This syntax allows multiple lines of text to be passed into a shell command, enabling Dockerfile `RUN` commands to be written like shell scripts, instead of having to jam commands into long run-on lines. Heredoc support was added to Dockerfiles in the [1.4.0 release](https://github.com/moby/buildkit/releases/tag/dockerfile%2F1.4.0).

For more info, see:

- [br3ndonland/inboard#54](https://github.com/br3ndonland/inboard/pull/54)
- [BuildKit docs: Dockerfile frontend syntaxes](https://github.com/moby/buildkit/blob/HEAD/frontend/dockerfile/docs/syntax.md)
- [BuildKit releases: dockerfile/1.4.0](https://github.com/moby/buildkit/releases/tag/dockerfile%2F1.4.0)
- [Docker blog 2021-07-30: Introduction to heredocs in Dockerfiles](https://www.docker.com/blog/introduction-to-heredocs-in-dockerfiles/)
- [Docker docs: Develop with Docker - Build images with BuildKit](https://docs.docker.com/develop/develop-images/build_enhancements/)
- [Docker docs: Dockerfile reference - BuildKit](https://docs.docker.com/engine/reference/builder/#buildkit)

0 comments on commit 5968ae2

Please sign in to comment.