Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(atlantis-image): recombine with atlantis-base #3001

Merged
merged 5 commits into from
Apr 4, 2023
Merged

refactor(atlantis-image): recombine with atlantis-base #3001

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
11b3ec3
refactor: atlantis-image build pipeline and docker images
GenPage Jan 17, 2023
249e47c
fix: apply @nitrocode suggestions from code review
GenPage Jan 23, 2023
d51c9a6
fix: hadolinting
GenPage Apr 4, 2023
41342d2
fix: DL4006 on go mod graph
GenPage Apr 4, 2023
eb46af9
fix: version regressions
GenPage Apr 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 0 additions & 98 deletions .github/workflows/atlantis-base.yml
View file
Open in desktop

This file was deleted.

25 changes: 25 additions & 0 deletions .github/workflows/atlantis-image.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ on:
pull_request:
paths:
- 'Dockerfile'
- 'docker-entrypoint.sh'
- '.github/workflows/atlantis-image.yml'
workflow_dispatch:

Expand All @@ -32,6 +33,12 @@ jobs:
steps:
- uses: actions/checkout@v3

# Lint the Dockerfile first before setting anything up
- name: Lint Dockerfile
uses: hadolint/hadolint-action@master
with:
dockerfile: "Dockerfile"

- name: Set up QEMU
uses: docker/setup-qemu-action@v2
with:
Expand All @@ -56,6 +63,7 @@ jobs:
images: |
${{ env.DOCKER_REPO }}
labels: |
org.opencontainers.image.authors="Anubhav Mishra, Luke Kysow"
Copy link
Member

@nitrocode nitrocode Apr 4, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: This label seems no longer relevant as these authors are no longer actively maintaining the repo. I think we can omit this in the future

org.opencontainers.image.licenses=Apache-2.0
tags: |
# semver
Expand Down Expand Up @@ -86,6 +94,22 @@ jobs:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

# Publish dev image to container registry
- name: Build and push atlantis:dev${{ env.IMAGE_SUFFIX }} image
if: ${{ contains(fromJson('["push", "pull_request"]'), github.event_name) }}
uses: docker/build-push-action@v3
with:
cache-from: type=gha
cache-to: type=gha,mode=max
context: .
platforms: linux/arm64/v8,linux/amd64,linux/arm/v7
push: ${{ github.event_name != 'pull_request' }}
tags: |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems incorrect. We started using the metadata tags in a recent pr.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this entire block can be removed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @jamengual @GenPage

ghcr.io/${{ github.repository_owner }}/atlantis:dev${{ env.IMAGE_SUFFIX }}
ghcr.io/${{ github.repository_owner }}/atlantis:dev-${{ matrix.image_type }}
target: ${{ matrix.image_type }}
labels: ${{ steps.meta.outputs.labels }}

# Publish release to container registry
- name: Populate release version
if: contains(fromJson('["push", "pull_request"]'), github.event_name)
Expand All @@ -106,5 +130,6 @@ jobs:
platforms: linux/arm64/v8,linux/amd64,linux/arm/v7
push: ${{ env.PUSH }}
tags: ${{ steps.meta.outputs.tags }}
target: ${{ matrix.image_type }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=target,annotation-index.org.opencontainers.image.description=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.description'] }}
182 changes: 151 additions & 31 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
ARG ATLANTIS_BASE=ghcr.io/runatlantis/atlantis-base
ARG ATLANTIS_BASE_TAG_DATE=latest
ARG ATLANTIS_BASE_TAG_TYPE=alpine
# syntax=docker/dockerfile:1
GenPage marked this conversation as resolved.
Show resolved Hide resolved
# what distro is the image being built for
ARG ALPINE_TAG=3.17.1
ARG DEBIAN_TAG=11.6-slim

# Stage 1: build artifact
# Stage 1: build artifact and download deps

FROM golang:1.20.2-alpine AS builder

Expand All @@ -19,28 +20,102 @@ WORKDIR /app
# https://github.com/montanaflynn/golang-docker-cache
# https://github.com/golang/go/issues/27719
COPY go.mod go.sum ./
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
RUN go mod graph | awk '{if ($1 !~ "@") print $2}' | xargs go get

COPY . /app

RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
CGO_ENABLED=0 go build -trimpath -ldflags "-s -w -X 'main.version=${ATLANTIS_VERSION}' -X 'main.commit=${ATLANTIS_COMMIT}' -X 'main.date=${ATLANTIS_DATE}'" -v -o atlantis .

# Stage 2
# The runatlantis/atlantis-base is created by docker-base/Dockerfile
FROM ${ATLANTIS_BASE}:${ATLANTIS_BASE_TAG_DATE}-${ATLANTIS_BASE_TAG_TYPE} AS base
FROM debian:${DEBIAN_TAG} as deps

# Get the architecture the image is being built for
ARG TARGETPLATFORM
WORKDIR /tmp/build

# Install packages needed for building/verifying dependencies
# hadolint ignore=DL3008,SC2261
RUN apt-get update \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are these lines repeated on 180? Couldn't we install these packages just once in the debian release?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I can improve this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revisiting this, we re-install twice because we throw away the builder image which has more package installs, and this is a slimmed down install for the actual release image.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But now we have 2 places where we need to make sure these dependencies are up to date... Seems like extra maintenance and an easy thing to bump in one section and forget to bump in another

&& apt-get install -y --no-install-recommends \
ca-certificates>=20210119 \
curl>=7.74 \
git>=1:2.30 \
unzip>=6.0 \
bash>=5.1 \
openssh-server>=1:8.4p1 \
libcap2>=1:2.44 \
dumb-init>=1.2 \
gnupg>=2.2 \
openssl>=1.1.1n

# install conftest
# renovate: datasource=github-releases depName=open-policy-agent/conftest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This renovate comment should be directly before the conftest line

@jamengual @GenPage

SHELL ["/bin/bash", "-o", "pipefail", "-c"]
ENV DEFAULT_CONFTEST_VERSION=0.40.0
RUN AVAILABLE_CONFTEST_VERSIONS=${DEFAULT_CONFTEST_VERSION} && \
case ${TARGETPLATFORM} in \
"linux/amd64") CONFTEST_ARCH=x86_64 ;; \
"linux/arm64") CONFTEST_ARCH=arm64 ;; \
# There is currently no compiled version of conftest for armv7
"linux/arm/v7") CONFTEST_ARCH=x86_64 ;; \
esac && \
for VERSION in ${AVAILABLE_CONFTEST_VERSIONS}; do \
curl -LOs "https://github.com/open-policy-agent/conftest/releases/download/v${VERSION}/conftest_${VERSION}_Linux_${CONFTEST_ARCH}.tar.gz" && \
curl -LOs "https://github.com/open-policy-agent/conftest/releases/download/v${VERSION}/checksums.txt" && \
sed -n "/conftest_${VERSION}_Linux_${CONFTEST_ARCH}.tar.gz/p" checksums.txt | sha256sum -c && \
mkdir -p "/usr/local/bin/cft/versions/${VERSION}" && \
tar -C "/usr/local/bin/cft/versions/${VERSION}" -xzf "conftest_${VERSION}_Linux_${CONFTEST_ARCH}.tar.gz" && \
ln -s "/usr/local/bin/cft/versions/${VERSION}/conftest" /usr/local/bin/conftest && \
rm "conftest_${VERSION}_Linux_${CONFTEST_ARCH}.tar.gz" && \
rm checksums.txt; \
done

# install gosu
# We use gosu to step down from root and run as the atlantis user
# renovate: datasource=github-releases depName=tianon/gosu
ENV GOSU_VERSION=1.16
GenPage marked this conversation as resolved.
Show resolved Hide resolved

RUN case ${TARGETPLATFORM} in \
"linux/amd64") GOSU_ARCH=amd64 ;; \
"linux/arm64") GOSU_ARCH=arm64 ;; \
"linux/arm/v7") GOSU_ARCH=armhf ;; \
esac && \
curl -L -s --output gosu "https://github.com/tianon/gosu/releases/download/${GOSU_VERSION}/gosu-${GOSU_ARCH}" && \
curl -L -s --output gosu.asc "https://github.com/tianon/gosu/releases/download/${GOSU_VERSION}/gosu-${GOSU_ARCH}.asc" && \
for server in $(shuf -e ipv4.pool.sks-keyservers.net \
hkp://p80.pool.sks-keyservers.net:80 \
keyserver.ubuntu.com \
hkp://keyserver.ubuntu.com:80 \
pgp.mit.edu) ; do \
gpg --keyserver "$server" --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 && break || : ; \
done && \
gpg --batch --verify gosu.asc gosu && \
chmod +x gosu && \
cp gosu /bin && \
gosu --version

# install git-lfs
# renovate: datasource=github-releases depName=git-lfs/git-lfs
ENV GIT_LFS_VERSION=3.3.0

RUN case ${TARGETPLATFORM} in \
"linux/amd64") GIT_LFS_ARCH=amd64 ;; \
"linux/arm64") GIT_LFS_ARCH=arm64 ;; \
"linux/arm/v7") GIT_LFS_ARCH=arm ;; \
esac && \
curl -L -s --output git-lfs.tar.gz "https://github.com/git-lfs/git-lfs/releases/download/v${GIT_LFS_VERSION}/git-lfs-linux-${GIT_LFS_ARCH}-v${GIT_LFS_VERSION}.tar.gz" && \
tar --strip-components=1 -xf git-lfs.tar.gz && \
chmod +x git-lfs && \
mv git-lfs /usr/bin/git-lfs && \
git-lfs --version

# install terraform binaries
# renovate: datasource=github-releases depName=hashicorp/terraform versioning=hashicorp
ENV DEFAULT_TERRAFORM_VERSION=1.4.4

# In the official Atlantis image we only have the latest of each Terraform version.
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
RUN AVAILABLE_TERRAFORM_VERSIONS="1.0.11 1.1.9 1.2.9 1.3.9 ${DEFAULT_TERRAFORM_VERSION}" && \
# In the official Atlantis image, we only have the latest of each Terraform version.
RUN AVAILABLE_TERRAFORM_VERSIONS="1.1.9 1.2.9 1.3.9 ${DEFAULT_TERRAFORM_VERSION}" && \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was the shell argument removed?

case "${TARGETPLATFORM}" in \
"linux/amd64") TERRAFORM_ARCH=amd64 ;; \
"linux/arm64") TERRAFORM_ARCH=arm64 ;; \
Expand All @@ -59,34 +134,79 @@ RUN AVAILABLE_TERRAFORM_VERSIONS="1.0.11 1.1.9 1.2.9 1.3.9 ${DEFAULT_TERRAFORM_V
done && \
ln -s "/usr/local/bin/tf/versions/${DEFAULT_TERRAFORM_VERSION}/terraform" /usr/local/bin/terraform

# renovate: datasource=github-releases depName=open-policy-agent/conftest
ENV DEFAULT_CONFTEST_VERSION=0.40.0

RUN AVAILABLE_CONFTEST_VERSIONS="${DEFAULT_CONFTEST_VERSION}" && \
case "${TARGETPLATFORM}" in \
"linux/amd64") CONFTEST_ARCH=x86_64 ;; \
"linux/arm64") CONFTEST_ARCH=arm64 ;; \
# There is currently no compiled version of conftest for armv7
"linux/arm/v7") CONFTEST_ARCH=x86_64 ;; \
esac && \
for VERSION in ${AVAILABLE_CONFTEST_VERSIONS}; do \
curl -LOs "https://github.com/open-policy-agent/conftest/releases/download/v${VERSION}/conftest_${VERSION}_Linux_${CONFTEST_ARCH}.tar.gz" && \
curl -LOs "https://github.com/open-policy-agent/conftest/releases/download/v${VERSION}/checksums.txt" && \
sed -n "/conftest_${VERSION}_Linux_${CONFTEST_ARCH}.tar.gz/p" checksums.txt | sha256sum -c && \
mkdir -p "/usr/local/bin/cft/versions/${VERSION}" && \
tar -C "/usr/local/bin/cft/versions/${VERSION}" -xzf "conftest_${VERSION}_Linux_${CONFTEST_ARCH}.tar.gz" && \
ln -s "/usr/local/bin/cft/versions/${VERSION}/conftest" "/usr/local/bin/conftest${VERSION}" && \
rm "conftest_${VERSION}_Linux_${CONFTEST_ARCH}.tar.gz" && \
rm checksums.txt; \
done
# Stage 2 - Alpine
# Creating the individual distro builds using targets
FROM alpine:${ALPINE_TAG} AS alpine

RUN ln -s /usr/local/bin/cft/versions/${DEFAULT_CONFTEST_VERSION}/conftest /usr/local/bin/conftest
# atlantis user for gosu and OpenShift compatibility
RUN addgroup atlantis && \
adduser -S -G atlantis atlantis && \
adduser atlantis root && \
chown atlantis:root /home/atlantis/ && \
chmod g=u /home/atlantis/ && \
chmod g=u /etc/passwd

# copy binary
COPY --from=builder /app/atlantis /usr/local/bin/atlantis
# copy terraform
COPY --from=deps /usr/local/bin/terraform* /usr/local/bin/
# copy deps
COPY --from=deps /usr/local/bin/conftest /usr/local/bin/conftest
COPY --from=deps /bin/gosu /bin/gosu
COPY --from=deps /usr/bin/git-lfs /usr/bin/git-lfs
# copy docker entrypoint
COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh

# Install packages needed for running Atlantis.
# We place this last as it will bust less docker layer caches when packages update
RUN apk add --no-cache --repository=http://dl-cdn.alpinelinux.org/alpine/edge/main/ \
git~=2.40 && \
apk add --no-cache \
ca-certificates~=20220614 \
curl~=7.87 \
unzip~=6.0 \
bash~=5.2 \
openssh~=9.1_p1 \
libcap~=2.66 \
dumb-init~=1.2 \
gcompat~=1.1

ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["server"]

# Stage 2 - Debian
FROM debian:${DEBIAN_TAG} AS debian

# copy binary
COPY --from=builder /app/atlantis /usr/local/bin/atlantis
# copy terraform
COPY --from=deps /usr/local/bin/terraform* /usr/local/bin/
# copy deps
COPY --from=deps /usr/local/bin/conftest /usr/local/bin/conftest
COPY --from=deps /bin/gosu /bin/gosu
COPY --from=deps /usr/bin/git-lfs /usr/bin/git-lfs
# copy docker entrypoint
COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh

# Install packages needed for running Atlantis.
# We place this last as it will bust less docker layer caches when packages update
# hadolint ignore explanation
# DL3008 (pin versions using "=") - Ignored to avoid failing the build
# SC2261 (multiple redirections) - This is a bug https://github.com/hadolint/hadolint/issues/782
# hadolint ignore=DL3008,SC2261
RUN apt-get update && \
apt-get install -y --no-install-recommends \
ca-certificates>=20210119 \
curl>=7.74 \
git>=1:2.30 \
unzip>=6.0 \
bash>=5.1 \
openssh-server>=1:8.4p1 \
libcap2>=1:2.44 \
dumb-init>=1.2 && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["server"]
4 changes: 0 additions & 4 deletions Dockerfile.dev
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
FROM ghcr.io/runatlantis/atlantis:latest
COPY atlantis /usr/local/bin/atlantis
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the comments below, I don't think this image is needed anymore.

@jamengual

# TODO: remove this once we get this in the base image
# renovate: datasource=github-releases depName=open-policy-agent/conftest
ENV DEFAULT_CONFTEST_VERSION=0.40.0

WORKDIR /atlantis/src
Loading