Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Dockerfile configuration #195

Merged
merged 38 commits into from
Dec 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
b6b0b7c
Use the full identifier for source Docker images
mcdonnnj Feb 2, 2024
3b7a9cc
Install cisagov/skeleton-python-library directly
mcdonnnj Feb 20, 2024
db19706
Use a specific version of Alpine Linux
mcdonnnj Feb 13, 2024
9e6eef2
Remove unused OS package dependencies
mcdonnnj Feb 20, 2024
f419d35
Merge pull request #187 from cisagov/improvement/use_full_image_source
mcdonnnj Feb 20, 2024
c516e44
Remove package upgrading
mcdonnnj Feb 20, 2024
460eeec
Change the secret message being checks in tests
mcdonnnj Feb 20, 2024
001e85e
Pin Python packages directly installed
mcdonnnj Feb 20, 2024
e708211
Merge pull request #188 from cisagov/improvement/install_skeleton-pyt…
mcdonnnj Feb 21, 2024
48fa1a5
Prefer calling pip as a module
mcdonnnj Feb 26, 2024
e6f5798
Move WORKDIR instruction
mcdonnnj Feb 23, 2024
95d4a7a
Use a Python virtual environment in the Docker image
mcdonnnj Feb 26, 2024
43bf47b
Merge pull request #189 from cisagov/improvement/make_builds_more_rep…
mcdonnnj Feb 26, 2024
79a4adf
Add a pipenv configuration
mcdonnnj Feb 27, 2024
d3895ef
Explain `ln` options being used
mcdonnnj Feb 27, 2024
6b869bb
Install Python dependencies using pipenv
mcdonnnj Feb 28, 2024
30f6fa9
Merge pull request #190 from cisagov/improvement/use_Python_venv
mcdonnnj Feb 27, 2024
5cbb75b
Use a multi-stage Docker build
mcdonnnj Feb 28, 2024
5dca6d9
Install core Python packages into the system Python environment
mcdonnnj Feb 28, 2024
79d8de9
Fix outdated comment in the Dockerfile
mcdonnnj Feb 28, 2024
440ba70
Fix typo in Dockerfile comment
mcdonnnj Feb 28, 2024
78f4f78
Update image tag information in the README
mcdonnnj Feb 29, 2024
1774267
Add instructions for managing Python dependencies
mcdonnnj Feb 29, 2024
704b76f
Merge pull request #191 from cisagov/improvement/pin_Python_configura…
mcdonnnj Feb 28, 2024
11f8e08
Merge pull request #192 from cisagov/improvement/update_readme
mcdonnnj Mar 4, 2024
af716cc
Bump setuptools from 69.1.0 to 69.1.1
mcdonnnj Feb 28, 2024
a65ebdc
Bump Python from 3.12.0 to 3.12.2
mcdonnnj Feb 28, 2024
5d52367
Bump Alpine Linux from 3.18 to 3.19
mcdonnnj Feb 28, 2024
56afca0
Bump cisagov/skeleton-python-library from 0.0.1 to 0.2.0
mcdonnnj Feb 28, 2024
ae0d4a8
Bump version from 0.0.1 to 0.2.0
mcdonnnj Mar 4, 2024
95e533b
Merge pull request #193 from cisagov/improvement/update_dependencies
mcdonnnj Mar 5, 2024
965da2c
Correct usage of the term "symlink"
mcdonnnj Mar 13, 2024
570b7d5
Bump python from 3.12.2-alpine3.19 to 3.13.1-alpine3.20
mcdonnnj Nov 23, 2024
b3c5df5
Bump Python packages installed in the Dockerfile
mcdonnnj Nov 23, 2024
c09b352
Update Python virtual environment dependencies
mcdonnnj Nov 23, 2024
192d9e6
Merge pull request #200 from cisagov/improvement/update_dependencies
mcdonnnj Dec 6, 2024
a1ece8f
Add a missing Oxford comma to a comment in the Dockerfile
mcdonnnj Dec 6, 2024
fd69f45
Ensure FROM/AS keywords use the same casing
mcdonnnj Dec 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -348,8 +348,6 @@ jobs:
id: docker_build
uses: docker/build-push-action@v6
with:
build-args: |
VERSION=${{ needs.prepare.outputs.source_version }}
cache-from: type=local,src=${{ env.BUILDX_CACHE_DIR }}
cache-to: type=local,dest=${{ env.BUILDX_CACHE_DIR }}
context: .
Expand Down Expand Up @@ -516,8 +514,6 @@ jobs:
id: docker_build
uses: docker/build-push-action@v6
with:
build-args: |
VERSION=${{ needs.prepare.outputs.source_version }}
cache-from: type=local,src=${{ env.BUILDX_CACHE_DIR }}
cache-to: type=local,dest=${{ env.BUILDX_CACHE_DIR }}
context: .
Expand Down
118 changes: 67 additions & 51 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,8 +1,59 @@
ARG VERSION=unspecified
# Official Docker images are in the form library/<app> while non-official
# images are in the form <user>/<app>.
FROM docker.io/library/python:3.13.1-alpine3.20 AS compile-stage

FROM python:3.12.0-alpine
###
# Unprivileged user variables
###
ARG CISA_USER="cisa"
ENV CISA_HOME="/home/${CISA_USER}"
ENV VIRTUAL_ENV="${CISA_HOME}/.venv"

ARG VERSION
# Versions of the Python packages installed directly
ENV PYTHON_PIP_VERSION=24.3.1
ENV PYTHON_PIPENV_VERSION=2024.4.0
ENV PYTHON_SETUPTOOLS_VERSION=75.6.0
ENV PYTHON_WHEEL_VERSION=0.45.1

###
# Install the specified versions of pip, setuptools, and wheel into the system
# Python environment; install the specified version of pipenv into the system Python
# environment; set up a Python virtual environment (venv); and install the specified
# versions of pip, setuptools, and wheel into the venv.
#
# Note that we use the --no-cache-dir flag to avoid writing to a local
# cache. This results in a smaller final image, at the cost of
# slightly longer install times.
###
RUN python3 -m pip install --no-cache-dir --upgrade \
pip==${PYTHON_PIP_VERSION} \
felddy marked this conversation as resolved.
Show resolved Hide resolved
setuptools==${PYTHON_SETUPTOOLS_VERSION} \
wheel==${PYTHON_WHEEL_VERSION} \
&& python3 -m pip install --no-cache-dir --upgrade \
pipenv==${PYTHON_PIPENV_VERSION} \
# Manually create the virtual environment
&& python3 -m venv ${VIRTUAL_ENV} \
# Ensure the core Python packages are installed in the virtual environment
&& ${VIRTUAL_ENV}/bin/python3 -m pip install --no-cache-dir --upgrade \
pip==${PYTHON_PIP_VERSION} \
setuptools==${PYTHON_SETUPTOOLS_VERSION} \
wheel==${PYTHON_WHEEL_VERSION}

###
# Check the Pipfile configuration and then install the Python dependencies into
# the virtual environment.
#
# Note that pipenv will install into a virtual environment if the VIRTUAL_ENV
# environment variable is set.
###
WORKDIR /tmp
COPY src/Pipfile src/Pipfile.lock ./
RUN pipenv check --verbose \
&& pipenv install --clear --deploy --extra-pip-args "--no-cache-dir" --verbose

# Official Docker images are in the form library/<app> while non-official
# images are in the form <user>/<app>.
FROM docker.io/library/python:3.13.1-alpine3.20 AS build-stage

###
# For a list of pre-defined annotation keys and value types see:
Expand All @@ -27,15 +78,7 @@ ARG CISA_GID=${CISA_UID}
ARG CISA_USER="cisa"
ENV CISA_GROUP=${CISA_USER}
ENV CISA_HOME="/home/${CISA_USER}"

###
# Upgrade the system
#
# Note that we use apk --no-cache to avoid writing to a local cache.
# This results in a smaller final image, at the cost of slightly
# longer install times.
###
RUN apk --update --no-cache --quiet upgrade
ENV VIRTUAL_ENV="${CISA_HOME}/.venv"

###
# Create unprivileged user
Expand All @@ -44,52 +87,25 @@ RUN addgroup --system --gid ${CISA_GID} ${CISA_GROUP} \
&& adduser --system --uid ${CISA_UID} --ingroup ${CISA_GROUP} ${CISA_USER}

###
# Dependencies
# Copy in the Python virtual environment created in compile-stage, symlink the
# Python binary in the venv to the system-wide Python, and add the venv to the PATH.
#
# Note that we use apk --no-cache to avoid writing to a local cache.
# This results in a smaller final image, at the cost of slightly
# longer install times.
###
ENV DEPS \
ca-certificates \
openssl \
py-pip
RUN apk --no-cache --quiet add ${DEPS}

###
# Make sure pip, setuptools, and wheel are the latest versions
#
# Note that we use pip3 --no-cache-dir to avoid writing to a local
# cache. This results in a smaller final image, at the cost of
# slightly longer install times.
###
RUN pip3 install --no-cache-dir --upgrade \
pip \
setuptools \
wheel

WORKDIR ${CISA_HOME}

###
# Install Python dependencies
#
# Note that we use pip3 --no-cache-dir to avoid writing to a local
# cache. This results in a smaller final image, at the cost of
# slightly longer install times.
###
RUN wget --output-document sourcecode.tgz \
https://github.com/cisagov/skeleton-python-library/archive/v${VERSION}.tar.gz \
&& tar --extract --gzip --file sourcecode.tgz --strip-components=1 \
&& pip3 install --no-cache-dir --requirement requirements.txt \
&& ln -snf /run/secrets/quote.txt src/example/data/secret.txt \
&& rm sourcecode.tgz
# Note that we symlink the Python binary in the venv to the system-wide Python so that
# any calls to `python3` will use our virtual environment. We are using short flags
felddy marked this conversation as resolved.
Show resolved Hide resolved
# because the ln binary in Alpine Linux does not support long flags. The -f instructs
# ln to remove the existing file and the -s instructs ln to create a symbolic link.
###
COPY --from=compile-stage --chown=${CISA_USER}:${CISA_GROUP} ${VIRTUAL_ENV} ${VIRTUAL_ENV}
RUN ln -fs "$(command -v python3)" "${VIRTUAL_ENV}"/bin/python3
felddy marked this conversation as resolved.
Show resolved Hide resolved
ENV PATH="${VIRTUAL_ENV}/bin:$PATH"

###
# Prepare to run
###
ENV ECHO_MESSAGE="Hello World from Dockerfile"
WORKDIR ${CISA_HOME}
USER ${CISA_USER}:${CISA_GROUP}
EXPOSE 8080/TCP
VOLUME ["/var/log"]
ENTRYPOINT ["example"]
CMD ["--log-level", "DEBUG"]
CMD ["--log-level", "DEBUG", "8", "2"]
53 changes: 41 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ appropriate for Docker containers and the major languages that we use.
To run the `cisagov/example` image via Docker:

```console
docker run cisagov/example:0.0.1
docker run cisagov/example:0.2.0
```

### Running with Docker Compose ###
Expand All @@ -37,7 +37,7 @@ docker run cisagov/example:0.0.1

services:
example:
image: cisagov/example:0.0.1
image: cisagov/example:0.2.0
volumes:
- type: bind
source: <your_log_dir>
Expand Down Expand Up @@ -82,7 +82,7 @@ environment variables. See the

services:
example:
image: cisagov/example:0.0.1
image: cisagov/example:0.2.0
volumes:
- type: bind
source: <your_log_dir>
Expand Down Expand Up @@ -125,23 +125,52 @@ environment variables. See the
1. Pull the new image:

```console
docker pull cisagov/example:0.0.1
docker pull cisagov/example:0.2.0
```

1. Recreate and run the container by following the [previous instructions](#running-with-docker).

## Updating Python dependencies ##

This image uses [Pipenv] to manage Python dependencies using a [Pipfile](https://github.com/pypa/pipfile).
Both updating dependencies and changing the [Pipenv] configuration in `src/Pipfile`
will result in a modified `src/Pipfile.lock` file that should be committed to the
repository.

> [!WARNING]
> The `src/Pipfile.lock` as generated will fail `pre-commit` checks due to JSON formatting.

### Updating dependencies ###

If you want to update existing dependencies you would run the following command
in the `src/` subdirectory:

```console
pipenv lock
```

### Modifying dependencies ###

If you want to add or remove dependencies you would update the `src/Pipfile` file
and then update dependencies as you would above.

> [!NOTE]
> You should only specify packages that are explicitly needed for your Docker
> configuration. Allow [Pipenv] to manage the dependencies of the specified
> packages.

## Image tags ##

The images of this container are tagged with [semantic
versions](https://semver.org) of the underlying example project that they
containerize. It is recommended that most users use a version tag (e.g.
`:0.0.1`).
`:0.2.0`).

| Image:tag | Description |
|-----------|-------------|
|`cisagov/example:1.2.3`| An exact release version. |
|`cisagov/example:1.2`| The most recent release matching the major and minor version numbers. |
|`cisagov/example:1`| The most recent release matching the major version number. |
|`cisagov/example:0.2.0`| An exact release version. |
|`cisagov/example:0.2`| The most recent release matching the major and minor version numbers. |
|`cisagov/example:0`| The most recent release matching the major version number. |
|`cisagov/example:edge` | The most recent image built from a merge into the `develop` branch of this repository. |
|`cisagov/example:nightly` | A nightly build of the `develop` branch of this repository. |
|`cisagov/example:latest`| The most recent release image pushed to a container registry. Pulling an image using the `:latest` tag [should be avoided.](https://vsupalov.com/docker-latest-tag/) |
Expand Down Expand Up @@ -196,8 +225,7 @@ Build the image locally using this git repository as the [build context](https:/

```console
docker build \
--build-arg VERSION=0.0.1 \
--tag cisagov/example:0.0.1 \
--tag cisagov/example:0.2.0 \
https://github.com/cisagov/example.git#develop
```

Expand Down Expand Up @@ -227,9 +255,8 @@ Docker:
docker buildx build \
--file Dockerfile-x \
--platform linux/amd64 \
--build-arg VERSION=0.0.1 \
--output type=docker \
--tag cisagov/example:0.0.1 .
--tag cisagov/example:0.2.0 .
```

## New repositories from a skeleton ##
Expand All @@ -256,3 +283,5 @@ dedication](https://creativecommons.org/publicdomain/zero/1.0/).
All contributions to this project will be released under the CC0
dedication. By submitting a pull request, you are agreeing to comply
with this waiver of copyright interest.

[Pipenv]: https://pypi.org/project/pipenv/
1 change: 1 addition & 0 deletions requirements-dev.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
--requirement requirements-test.txt
ipython
pipenv
semver>=3
13 changes: 13 additions & 0 deletions src/Pipfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

# List any Python dependencies for the image here
[packages]
# This should match the version of the image
example = {file = "https://github.com/cisagov/skeleton-python-library/archive/v0.2.0.tar.gz"}

# This version should match the version of Python in the image
[requires]
python_full_version = "3.13.1"
45 changes: 45 additions & 0 deletions src/Pipfile.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion src/version.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.0.1
0.2.0
6 changes: 3 additions & 3 deletions tests/container_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,8 @@
ENV_VAR = "ECHO_MESSAGE"
ENV_VAR_VAL = "Hello World from docker compose!"
READY_MESSAGE = "This is a debug message"
SECRET_QUOTE = (
"There are no secrets better kept than the secrets everybody guesses." # nosec
)
DIVISION_MESSAGE = "8 / 2 == 4.000000"
SECRET_QUOTE = "Three may keep a secret, if two of them are dead." # nosec
RELEASE_TAG = os.getenv("RELEASE_TAG")
VERSION_FILE = "src/version.txt"

Expand Down Expand Up @@ -54,6 +53,7 @@ def test_output(dockerc, main_container):
# make sure container exited if running test isolated
dockerc.wait(main_container.id)
log_output = main_container.logs()
assert DIVISION_MESSAGE in log_output, "Division message not found in log output."
assert SECRET_QUOTE in log_output, "Secret not found in log output."


Expand Down
Loading