Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alpine-based Dockerfile, bump psycopg, tweak prod deployment vars, close cursor in readiness check, no error on email login #293

Merged
merged 9 commits into from
Sep 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .flake8

This file was deleted.

51 changes: 33 additions & 18 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,37 +1,52 @@
# syntax=docker/dockerfile:1
# Prepare the base environment.
FROM python:3.12.4-slim AS builder_base_rt
FROM python:3.12.6-alpine AS builder_base
LABEL [email protected]
LABEL org.opencontainers.image.source=https://github.com/dbca-wa/resource_tracking

RUN apt-get update -y \
&& apt-get upgrade -y \
&& apt-get install -y libmagic-dev gcc binutils gdal-bin proj-bin python3-dev libpq-dev curl \
&& rm -rf /var/lib/apt/lists/* \
&& pip install --root-user-action=ignore --upgrade pip
# Install system requirements to build Python packages.
RUN apk add --no-cache \
gcc \
libressl-dev \
musl-dev \
libffi-dev
# Create a non-root user to run the application.
ARG UID=10001
ARG GID=10001
RUN addgroup -g ${GID} appuser \
&& adduser -H -D -u ${UID} -G appuser appuser

# Install Python libs using Poetry.
FROM builder_base_rt AS python_libs_rt
FROM builder_base AS python_libs_resourcetracking
# Add system dependencies required to use GDAL
# Ref: https://stackoverflow.com/a/59040511/14508
RUN apk add --no-cache \
gdal \
geos \
proj \
binutils \
&& ln -s /usr/lib/libproj.so.25 /usr/lib/libproj.so \
&& ln -s /usr/lib/libgdal.so.35 /usr/lib/libgdal.so \
&& ln -s /usr/lib/libgeos_c.so.1 /usr/lib/libgeos_c.so
WORKDIR /app
ARG POETRY_VERSION=1.8.3
RUN pip install --no-cache-dir --root-user-action=ignore poetry=="${POETRY_VERSION}"
COPY poetry.lock pyproject.toml ./
RUN poetry config virtualenvs.create false \
ARG POETRY_VERSION=1.8.3
RUN pip install --no-cache-dir --root-user-action=ignore poetry==${POETRY_VERSION} \
&& poetry config virtualenvs.create false \
&& poetry install --no-interaction --no-ansi --only main

# Create a non-root user.
ARG UID=10001
ARG GID=10001
RUN groupadd -g "${GID}" appuser \
&& useradd --no-create-home --no-log-init --uid "${UID}" --gid "${GID}" appuser
# Remove system libraries, no longer required.
RUN apk del \
gcc \
libressl-dev \
musl-dev \
libffi-dev

# Install the project.
FROM python_libs_rt
FROM python_libs_resourcetracking AS project_resourcetracking
COPY gunicorn.py manage.py ./
COPY resource_tracking ./resource_tracking
COPY tracking ./tracking
RUN python manage.py collectstatic --noinput

USER ${UID}
EXPOSE 8080
CMD ["gunicorn", "resource_tracking.wsgi", "--config", "gunicorn.py"]
37 changes: 37 additions & 0 deletions Dockerfile.debian
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# syntax=docker/dockerfile:1
# Prepare the base environment.
FROM python:3.12.4-slim AS builder_base_rt
LABEL [email protected]
LABEL org.opencontainers.image.source=https://github.com/dbca-wa/resource_tracking

RUN apt-get update -y \
&& apt-get upgrade -y \
&& apt-get install -y libmagic-dev gcc binutils gdal-bin proj-bin python3-dev libpq-dev curl \
&& rm -rf /var/lib/apt/lists/* \
&& pip install --root-user-action=ignore --upgrade pip

# Install Python libs using Poetry.
FROM builder_base_rt AS python_libs_rt
WORKDIR /app
ARG POETRY_VERSION=1.8.3
RUN pip install --no-cache-dir --root-user-action=ignore poetry==${POETRY_VERSION}
COPY poetry.lock pyproject.toml ./
RUN poetry config virtualenvs.create false \
&& poetry install --no-interaction --no-ansi --only main

# Create a non-root user.
ARG UID=10001
ARG GID=10001
RUN groupadd -g ${GID} appuser \
&& useradd --no-create-home --no-log-init --uid ${UID} --gid ${GID} appuser

# Install the project.
FROM python_libs_rt
COPY gunicorn.py manage.py ./
COPY resource_tracking ./resource_tracking
COPY tracking ./tracking
RUN python manage.py collectstatic --noinput

USER ${UID}
EXPOSE 8080
CMD ["gunicorn", "resource_tracking.wsgi", "--config", "gunicorn.py"]
49 changes: 31 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,29 @@
# Resource Tracking application

Django and Leaflet application that collects tracking information using IMAP
from a mailbox and displays it on a collection of layers provided by
Geoserver. The application also downloads observation data from
automatic weather stations.
Django and Leaflet application that collects tracking device information from a
variety of sources and aggregates it into a single database.

# Installation
## Installation

The recommended way to set up this project for development is using
[Poetry](https://python-poetry.org/docs/) to install and manage a virtual Python
environment. With Poetry installed, change into the project directory and run:

poetry install

To run Python commands in the virtualenv, thereafter run them like so:
Activate the virtualenv like so:

poetry run python manage.py
poetry shell

To run Python commands in the activated virtualenv, thereafter run them as normal:

python manage.py

Manage new or updating project dependencies with Poetry also, like so:

poetry add newpackage==1.0

# Environment variables
## Environment variables

This project uses confy to set environment variables (in a `.env` file).
The following variables are required for the project to run:
Expand All @@ -30,39 +32,50 @@ The following variables are required for the project to run:
SECRET_KEY="ThisIsASecretKey"

Other environment variables will be required to run the project in production
(these are context-dependent).
(these are context-dependent). These variables include:

ALLOWED_HOSTS
CSRF_TRUSTED_ORIGINS
EMAIL_HOST
EMAIL_USER
EMAIL_PASSWORD
TRACPLUS_URL
DFES_URL
DFES_USER
DFES_PASS
GEOSERVER_URL

# Running
## Running

Use `runserver` to run a local copy of the application:

poetry run python manage.py runserver 0:8080
python manage.py runserver 0:8080

Run console commands manually:

poetry run python manage.py shell_plus
python manage.py shell_plus

# Unit tests
## Unit tests

Run unit tests like so:

poetry run python manage.py test --keepdb -v2
python manage.py test --keepdb -v2

# Docker image
## Docker image

To build a new Docker image from the `Dockerfile`:

docker image build -t ghcr.io/dbca-wa/resource_tracking .

# Pre-commit hooks
## Pre-commit hooks

This project includes the following pre-commit hooks:

- TruffleHog: https://docs.trufflesecurity.com/docs/scanning-git/precommit-hooks/
- TruffleHog: <https://docs.trufflesecurity.com/docs/scanning-git/precommit-hooks/>

Pre-commit hooks may have additional system dependencies to run. Optionally
install pre-commit hooks locally like so:

poetry run pre-commit install

Reference: https://pre-commit.com/
Reference: <https://pre-commit.com/>
30 changes: 11 additions & 19 deletions kustomize/overlays/prod/deployment_patch.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ spec:
- name: resourcetracking
imagePullPolicy: IfNotPresent
env:
- name: PROD_SCARY_WARNING
value: "True"
- name: DATABASE_URL
valueFrom:
secretKeyRef:
Expand All @@ -27,21 +29,6 @@ spec:
secretKeyRef:
name: resourcetracking-env-prod
key: SECRET_KEY
- name: DFES_PASS
valueFrom:
secretKeyRef:
name: resourcetracking-env-prod
key: DFES_PASS
- name: DFES_URL
valueFrom:
secretKeyRef:
name: resourcetracking-env-prod
key: DFES_URL
- name: DFES_USER
valueFrom:
secretKeyRef:
name: resourcetracking-env-prod
key: DFES_USER
- name: EMAIL_HOST
valueFrom:
secretKeyRef:
Expand All @@ -57,16 +44,21 @@ spec:
secretKeyRef:
name: resourcetracking-env-prod
key: EMAIL_USER
- name: FLEETCARE_CONNECTION_STRING
- name: DFES_PASS
valueFrom:
secretKeyRef:
name: resourcetracking-env-prod
key: DFES_PASS
- name: DFES_URL
valueFrom:
secretKeyRef:
name: resourcetracking-env-prod
key: FLEETCARE_CONNECTION_STRING
- name: FLEETCARE_CONTAINER
key: DFES_URL
- name: DFES_USER
valueFrom:
secretKeyRef:
name: resourcetracking-env-prod
key: FLEETCARE_CONTAINER
key: DFES_USER
- name: TRACPLUS_URL
valueFrom:
secretKeyRef:
Expand Down
2 changes: 1 addition & 1 deletion kustomize/overlays/prod/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,4 @@ patches:
- path: service_patch.yaml
images:
- name: ghcr.io/dbca-wa/resource_tracking
newTag: 1.4.16
newTag: 1.4.17
5 changes: 3 additions & 2 deletions manage.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@
import sys

# These lines are required for interoperability between local and container environments.
dot_env = os.path.join(os.getcwd(), '.env')
dot_env = os.path.join(os.getcwd(), ".env")
if os.path.exists(dot_env):
from dotenv import load_dotenv

load_dotenv()


Expand All @@ -23,5 +24,5 @@ def main():
execute_from_command_line(sys.argv)


if __name__ == '__main__':
if __name__ == "__main__":
main()
Loading