Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix failing builds to bump Gunicorn, Django, and gevent versions #53

Merged
merged 2 commits into from
Jun 6, 2018
Merged

Fix failing builds to bump Gunicorn, Django, and gevent versions #53

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
70de3e6
Fix failing builds to bump Gunicorn, Django, and gevent versions
rbreslow Jun 4, 2018
2a2c54c
Update compatibility matrix and refactor README.md
rbreslow Jun 6, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 14 additions & 14 deletions .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,20 @@ language: bash
services: docker

env:
- VERSION=1.9 PYTHON_VERSION=2.7 PG_MAJOR=9.5 PG_VERSION=9.4+165+deb8u3 VARIANT=slim
- VERSION=1.9 PYTHON_VERSION=3.5 PG_MAJOR=9.5 PG_VERSION=9.4+165+deb8u3 VARIANT=slim
- VERSION=1.9 PYTHON_VERSION=2.7 PG_MAJOR=9.5 PG_VERSION=9.5.12-r0 VARIANT=alpine
- VERSION=1.9 PYTHON_VERSION=3.5 PG_MAJOR=9.5 PG_VERSION=9.5.12-r0 VARIANT=alpine
- VERSION=1.10 PYTHON_VERSION=2.7 PG_MAJOR=9.6 PG_VERSION=10+190.pgdg80+1 VARIANT=slim
- VERSION=1.10 PYTHON_VERSION=3.5 PG_MAJOR=9.6 PG_VERSION=10+190.pgdg80+1 VARIANT=slim
- VERSION=1.10 PYTHON_VERSION=2.7 PG_MAJOR=9.6 PG_VERSION=9.5.12-r0 VARIANT=alpine
- VERSION=1.10 PYTHON_VERSION=3.5 PG_MAJOR=9.6 PG_VERSION=9.5.12-r0 VARIANT=alpine
- VERSION=1.11 PYTHON_VERSION=2.7 PG_MAJOR=9.6 PG_VERSION=10+190.pgdg80+1 VARIANT=slim
- VERSION=1.11 PYTHON_VERSION=3.6 PG_MAJOR=9.6 PG_VERSION=10+190.pgdg80+1 VARIANT=slim
- VERSION=1.11 PYTHON_VERSION=2.7 PG_MAJOR=9.6 PG_VERSION=9.5.12-r0 VARIANT=alpine
- VERSION=1.11 PYTHON_VERSION=3.6 PG_MAJOR=9.6 PG_VERSION=9.5.12-r0 VARIANT=alpine
- VERSION=2.0 PYTHON_VERSION=3.6 PG_MAJOR=9.6 PG_VERSION=10+190.pgdg80+1 VARIANT=slim
- VERSION=2.0 PYTHON_VERSION=3.6 PG_MAJOR=9.6 PG_VERSION=9.5.12-r0 VARIANT=alpine
- VERSION=1.9 PYTHON_VERSION=2.7 PG_MAJOR=9.4 PG_VERSION=9.4.18-2.pgdg90+1 VARIANT=slim
- VERSION=1.9 PYTHON_VERSION=3.6 PG_MAJOR=9.4 PG_VERSION=9.4.18-2.pgdg90+1 VARIANT=slim
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1.9 does not support Python 3.6, so I think that we should drop those items from the matrix (the image will still exist in Quay, it just won't get updated).

See: https://docs.djangoproject.com/en/2.0/faq/install/#what-python-version-can-i-use-with-django

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

- VERSION=1.9 PYTHON_VERSION=2.7 PG_MAJOR=10 PG_VERSION=10.4-r0 VARIANT=alpine
- VERSION=1.9 PYTHON_VERSION=3.6 PG_MAJOR=10 PG_VERSION=10.4-r0 VARIANT=alpine
- VERSION=1.10 PYTHON_VERSION=2.7 PG_MAJOR=10 PG_VERSION=10.4-2.pgdg90+1 VARIANT=slim
- VERSION=1.10 PYTHON_VERSION=3.6 PG_MAJOR=10 PG_VERSION=10.4-2.pgdg90+1 VARIANT=slim
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here. 1.10 does not support Python 3.6.

- VERSION=1.10 PYTHON_VERSION=2.7 PG_MAJOR=10 PG_VERSION=10.4-r0 VARIANT=alpine
- VERSION=1.10 PYTHON_VERSION=3.6 PG_MAJOR=10 PG_VERSION=10.4-r0 VARIANT=alpine
- VERSION=1.11 PYTHON_VERSION=2.7 PG_MAJOR=10 PG_VERSION=10.4-2.pgdg90+1 VARIANT=slim
- VERSION=1.11 PYTHON_VERSION=3.6 PG_MAJOR=10 PG_VERSION=10.4-2.pgdg90+1 VARIANT=slim
- VERSION=1.11 PYTHON_VERSION=2.7 PG_MAJOR=10 PG_VERSION=10.4-r0 VARIANT=alpine
- VERSION=1.11 PYTHON_VERSION=3.6 PG_MAJOR=10 PG_VERSION=10.4-r0 VARIANT=alpine
- VERSION=2.0 PYTHON_VERSION=3.6 PG_MAJOR=10 PG_VERSION=10.4-2.pgdg90+1 VARIANT=slim
- VERSION=2.0 PYTHON_VERSION=3.6 PG_MAJOR=10 PG_VERSION=10.4-r0 VARIANT=alpine

script:
- ./scripts/cibuild
Expand Down
4 changes: 2 additions & 2 deletions 1.10/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
gunicorn==19.7.1
gunicorn==19.8.1
django==1.10.8
psycopg2==2.7.4 --no-binary psycopg2
gevent==1.2.2
gevent==1.3.2.post0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to confirm, you're using the 1.3.2.post0 package because the 1.3.2 one was busted?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. There was a packaging error I didn't dive too deep into.

4 changes: 2 additions & 2 deletions 1.11/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
gunicorn==19.7.1
gunicorn==19.8.1
django==1.11.13
psycopg2==2.7.4 --no-binary psycopg2
gevent==1.2.2
gevent==1.3.2.post0
4 changes: 2 additions & 2 deletions 1.9/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
gunicorn==19.7.1
gunicorn==19.8.1
django==1.9.13
psycopg2==2.7.4 --no-binary psycopg2
gevent==1.2.2
gevent==1.3.2.post0
6 changes: 3 additions & 3 deletions 2.0/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
gunicorn==19.7.1
django==2.0.5
gunicorn==19.8.1
django==2.0.6
psycopg2==2.7.4 --no-binary psycopg2
gevent==1.2.2
gevent==1.3.2.post0
6 changes: 4 additions & 2 deletions Dockerfile-alpine.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,13 @@ COPY requirements.txt /tmp/
RUN set -ex \
&& apk add --no-cache --virtual .build-deps \
gcc \
postgresql-dev=${PG_VERSION} \
postgresql-dev=$PG_VERSION \
musl-dev \
&& apk add --no-cache \
gettext \
postgresql-client=${PG_VERSION} \
postgresql-client=$PG_VERSION \
&& apk add --no-cache --repository http://dl-cdn.alpinelinux.org/alpine/edge/main \
libressl2.7-libcrypto \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is confusing, but appears to be a required upgrade from 2.6 given that gdal is coming from edge. I wasn't able to make it work with another combination. 😞

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this was my experience as well.

Want to resurface these:

&& apk add --no-cache --repository http://dl-cdn.alpinelinux.org/alpine/edge/testing \
gdal \
&& pip install --no-cache-dir -r /tmp/requirements.txt \
Expand Down
34 changes: 27 additions & 7 deletions Dockerfile-slim.template
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,33 @@
FROM python:%%PYTHON_VERSION%%-slim
FROM python:%%PYTHON_VERSION%%-slim-stretch
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making this more explicit is a good idea.


MAINTAINER Azavea <[email protected]>

RUN apt-key adv --keyserver ha.pool.sks-keyservers.net --recv-keys B97B0AFCAA1A47F044F244A07FCC7D46ACCC4CF8
RUN set -ex; \
if ! command -v gpg > /dev/null; then \
apt-get update; \
apt-get install -y --no-install-recommends \
gnupg \
dirmngr \
; \
rm -rf /var/lib/apt/lists/*; \
fi

RUN set -ex; \
# pub 4096R/ACCC4CF8 2011-10-13 [expires: 2019-07-02]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this might be causing moby/moby#29005

# Key fingerprint = B97B 0AFC AA1A 47F0 44F2 44A0 7FCC 7D46 ACCC 4CF8
# uid PostgreSQL Debian Repository
key='B97B0AFCAA1A47F044F244A07FCC7D46ACCC4CF8'; \
export GNUPGHOME="$(mktemp -d)"; \
gpg --keyserver ha.pool.sks-keyservers.net --recv-keys "$key"; \
gpg --export "$key" > /etc/apt/trusted.gpg.d/postgres.gpg; \
rm -rf "$GNUPGHOME"; \
apt-key list

ENV PG_MAJOR %%PG_MAJOR%%
ENV PG_VERSION %%PG_VERSION%%

RUN echo 'deb http://apt.postgresql.org/pub/repos/apt/ jessie-pgdg main' ${PG_MAJOR} > /etc/apt/sources.list.d/pgdg.list

RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ stretch-pgdg main $PG_MAJOR" > /etc/apt/sources.list.d/pgdg.list
COPY requirements.txt /tmp/

RUN set -ex \
Expand All @@ -19,11 +38,12 @@ RUN set -ex \
&& deps=" \
gdal-bin \
gettext \
postgresql-client=${PG_VERSION} \
postgresql-client-$PG_MAJOR=$PG_VERSION \
" \
&& apt-get update && apt-get install -y ${buildDeps} ${deps} --no-install-recommends \
&& apt-get update && apt-get install -y $buildDeps $deps --no-install-recommends \
&& pip install --no-cache-dir -r /tmp/requirements.txt \
&& apt-get purge -y --auto-remove ${buildDeps} \
&& apt-get purge -y --auto-remove $buildDeps \
$(! command -v gpg > /dev/null || echo 'gnupg dirmngr') \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of this?

Copy link
Contributor Author

@rbreslow rbreslow Jun 6, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removes gnupg and dirmngr if gpg exists (exits zero).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see now. Thanks for clarifying.

&& rm -rf /tmp/requirements.txt /var/lib/apt/lists/*

ENTRYPOINT ["/usr/local/bin/gunicorn"]
12 changes: 6 additions & 6 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,17 @@ This repository contains a collection of templated `Dockerfile` for image varian

### Template Variables

- `PYTHON_VERSION` - Python interpreter version
- `PG_MAJOR` - Major version number of the target PostgreSQL database instance
- `PG_VERSION` - Version number for `postgresql-client` package
- `VARIANT` - Base container image variant
- `PYTHON_VERSION` - Python interpreter version (for `python:*-slim-stretch` or `python:*-alpine` [images](https://hub.docker.com/_/python/))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good README additions.

- `PG_MAJOR` - Major version number of the target PostgreSQL database instance (`9.3`, `9.4`, `9.5`, `9.6`, and `10`)
- `PG_VERSION` - Version number for `postgresql-client-$PG_MAJOR` package
- `VARIANT` - Base container image variant (`alpine` or `slim`)

### Testing

An example of how to use `cibuild` to build and test an image:

```bash
$ CI=1 VERSION=1.9 PYTHON_VERSION=2.7 \
PG_MAJOR=9.6 PG_VERSION=9.5.4-r0 VARIANT=alpine \
$ CI=true VERSION=2.0 PYTHON_VERSION=3.6 \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although both work based on how we check, the use of integers as boolean flags seems more of a convention in Bash scripting than the strings true or false.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

PG_MAJOR=10 PG_VERSION=10.4-2.pgdg90+1 VARIANT=slim
./scripts/cibuild
```