Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove entrypoint from all Agent build images #370

Merged
merged 12 commits into from
Jun 13, 2023
2 changes: 1 addition & 1 deletion .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ trigger_build_kernels:
- git clone -b $AGENT_TESTING_BRANCH https://github.com/DataDog/datadog-agent /go/src/github.com/DataDog/datadog-agent
- cd /go/src/github.com/DataDog/datadog-agent
- VERSION="nightly"
- source /root/.bashrc && conda activate ddpy3
- source /root/.bashrc
- inv -e deps
script:
- inv -e agent.omnibus-build --release-version "$VERSION" --major-version "6" --python-runtimes "2,3" --base-dir /.omnibus --skip-deps
Expand Down
38 changes: 15 additions & 23 deletions deb-arm/Dockerfile
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason why rpm is split between arm64 and armhf, and not the same for deb?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good question. It boils down to similarity between the base images.

For the deb images, we use arm64v8/ubuntu:16.04 and arm32v7/ubuntu:16.04 as base images. Since that's the same OS & version, most commands are the same (with only a few differences), which allowed for merging the image definitions.

For the rpm images, we use amazonlinux:2.0.20181114 and arm32v7/centos:7 as base images (for reasons described here). These are different OSes, which implies some more differences in the image build processes, so to be safe these were separated in two files.

I don't think the steps done in the two rpm Dockerfiles are quite different nowadays, so we could look into merging them - that's mainly a matter of preference / legibility of the Dockerfiles.

Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ RUN echo "${CURL_AARCH64_SHA256} curl-aarch64" | sha256sum --check
FROM ${BASE_IMAGE}

# Build Args
ARG GIMME_GO_VERSION=1.19.9
ARG GO_VERSION=1.19.9
ARG CMAKE_VERSION=3.23.0
ARG CMAKE_SHA256="9f8d42ef0b33d1bea47afe15875435dac58503d6a3b58842b473fd811e6df172"
ARG CLANG_VERSION=8.0.0
Expand Down Expand Up @@ -78,22 +78,23 @@ RUN if [ "$DD_TARGET_ARCH" = "aarch64" ] ; then \
/bin/bash -l -c "rvm install --with-openssl-dir=${CONDA_PATH} --with-arch='armv7-a' -C '--build' -C 'arm-linux-gnueabihf' 2.7 && rvm cleanup all" ; \
fi
RUN /bin/bash -l -c "gem install bundler --no-document"
RUN echo 'source /usr/local/rvm/scripts/rvm' >> /root/.bashrc

# Gimme
RUN curl -sL -o /bin/gimme https://raw.githubusercontent.com/travis-ci/gimme/v1.5.4/gimme
RUN echo "03b295636d4e22870b6f6e9bc06a71d65311ae90d3d48cbc7071f82dd5837fbc /bin/gimme" | sha256sum --check
RUN chmod +x /bin/gimme

# GIMME_ARCH = GOARCH, so must be a valid entry from `goarchlist` here:
# https://github.com/golang/go/blob/master/src/go/build/syslist.go
# Also see https://github.com/travis-ci/gimme/blob/master/gimme#L880
# Go
RUN if [ "$DD_TARGET_ARCH" = "aarch64" ] ; then \
GIMME_ARCH=arm64 gimme $GIMME_GO_VERSION ; \
else \
GIMME_ARCH=arm gimme $GIMME_GO_VERSION ; \
fi
curl -sL -o /tmp/golang.tar.gz https://go.dev/dl/go$GO_VERSION.linux-arm64.tar.gz \
&& echo "b947e457be9d7b52a082c68e42b6939f9cc151f1ad5b3d8fd646ca3352f6f2f1 /tmp/golang.tar.gz" | sha256sum --check \
&& tar -C /usr/local -xzf /tmp/golang.tar.gz \
&& rm -f /tmp/golang.tar.gz ; fi

RUN if [ "$DD_TARGET_ARCH" = "armhf" ] ; then \
curl -sL -o /tmp/golang.tar.gz https://go.dev/dl/go$GO_VERSION.linux-armv6l.tar.gz \
&& echo "4a613b3c9d74975470096c5323e54db0f5f0f56dd0471fc1a7ee04e559032dc4 /tmp/golang.tar.gz" | sha256sum --check \
&& tar -C /usr/local -xzf /tmp/golang.tar.gz \
&& rm -f /tmp/golang.tar.gz ; fi

COPY ./gobin.sh /etc/profile.d/
ENV PATH="/usr/local/go/bin:${PATH}"
ENV PATH="${GOPATH}/bin:${PATH}"

# CMake
RUN if [ "$DD_TARGET_ARCH" = "aarch64" ] ; then set -ex \
Expand All @@ -119,14 +120,5 @@ RUN if [ "$DD_TARGET_ARCH" = "aarch64" ] ; then set -ex \
&& /build-gcc.sh \
&& rm /build-gcc.sh ; fi

# Entrypoint
COPY ./entrypoint.sh /
RUN chmod +x /entrypoint.sh

# create the agent build folder within $GOPATH
RUN mkdir -p /go/src/github.com/DataDog/datadog-agent

# Force umask to 0022
RUN echo "umask 0022" >> /root/.bashrc

ENTRYPOINT ["/entrypoint.sh"]
11 changes: 4 additions & 7 deletions deb-x64/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,7 @@ RUN curl -sSL -o get-rvm.sh https://raw.githubusercontent.com/rvm/rvm/1.29.12/bi
RUN /bin/bash -l -c "rvm requirements"
RUN /bin/bash -l -c "rvm install 2.7 --with-openssl-dir=${CONDA_PATH} && rvm cleanup all"
RUN /bin/bash -l -c "gem install bundler --no-document"
RUN echo 'source /usr/local/rvm/scripts/rvm' >> /root/.bashrc

# CMake
RUN set -ex \
Expand Down Expand Up @@ -158,7 +159,9 @@ RUN curl -Sl -O https://dd-agent-omnibus.s3.amazonaws.com/kernel-4.9-headers-deb
# TODO: investigate if we could use gvm to do this for us.
COPY ./setup_go.sh /setup_go.sh
RUN ./setup_go.sh
COPY ./gobin.sh /etc/profile.d/

ENV PATH="/goroot/bin:$PATH"
ENV PATH="/go/bin:$PATH"

# Download and install golangci-lint
RUN curl -sSfL -o golangci-lint-install.sh https://raw.githubusercontent.com/golangci/golangci-lint/v1.44.2/install.sh \
Expand Down Expand Up @@ -195,14 +198,8 @@ RUN chmod +x /build-gcc.sh \
&& /build-gcc.sh \
&& rm /build-gcc.sh

# Entrypoint
COPY ./entrypoint.sh /
RUN chmod +x /entrypoint.sh

# create the agent build folder within $GOPATH
RUN mkdir -p /go/src/github.com/DataDog/datadog-agent

# Force umask to 0022
RUN echo "umask 0022" >> /root/.bashrc

ENTRYPOINT ["/entrypoint.sh"]
27 changes: 10 additions & 17 deletions rpm-arm64/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ RUN echo "${CACERT_BUNDLE_SHA256} /cacert.pem" | sha256sum --check
FROM ${BASE_IMAGE}

# Build Args
ARG GIMME_GO_VERSION=1.19.9
ARG GO_VERSION=1.19.9
ARG CMAKE_VERSION=3.23.0
ARG CMAKE_SHA256="9f8d42ef0b33d1bea47afe15875435dac58503d6a3b58842b473fd811e6df172"
ARG CLANG_VERSION=8.0.0
Expand All @@ -21,7 +21,7 @@ ARG GCC_SHA256=ab1974017834430de27fd803ade4389602a7d6ca1362496c57bef384b2a4cb07

# Environment
ENV GOPATH /go
ENV GIMME_GO_VERSION $GIMME_GO_VERSION
ENV GO_VERSION $GO_VERSION
ENV CMAKE_VERSION $CMAKE_VERSION
ENV CMAKE_SHA256 $CMAKE_SHA256
ENV CLANG_VERSION $CLANG_VERSION
Expand Down Expand Up @@ -80,13 +80,15 @@ RUN curl -sSL -o get-rvm.sh https://raw.githubusercontent.com/rvm/rvm/1.29.12/bi
RUN /bin/bash -l -c "rvm requirements"
RUN /bin/bash -l -c "rvm install 2.7 --with-openssl-dir=${CONDA_PATH} && rvm cleanup all"
RUN /bin/bash -l -c "gem install bundler --no-document"
RUN echo 'source /usr/local/rvm/scripts/rvm' >> /root/.bashrc

# Gimme
RUN curl -sL -o /bin/gimme https://raw.githubusercontent.com/travis-ci/gimme/v1.5.4/gimme
RUN echo "03b295636d4e22870b6f6e9bc06a71d65311ae90d3d48cbc7071f82dd5837fbc /bin/gimme" | sha256sum --check
RUN chmod +x /bin/gimme
RUN gimme $GIMME_GO_VERSION
COPY ./gobin.sh /etc/profile.d/
# Go
RUN curl -sL -o /tmp/golang.tar.gz https://go.dev/dl/go$GO_VERSION.linux-arm64.tar.gz \
&& echo "b947e457be9d7b52a082c68e42b6939f9cc151f1ad5b3d8fd646ca3352f6f2f1 /tmp/golang.tar.gz" | sha256sum --check \
&& tar -C /usr/local -xzf /tmp/golang.tar.gz \
&& rm -f /tmp/golang.tar.gz
ENV PATH="/usr/local/go/bin:${PATH}"
ENV PATH="${GOPATH}/bin:${PATH}"

# CMake
RUN set -ex \
Expand All @@ -111,14 +113,5 @@ RUN chmod +x /build-gcc.sh \
&& /build-gcc.sh \
&& rm /build-gcc.sh

# Entrypoint
COPY ./entrypoint.sh /
RUN chmod +x /entrypoint.sh

# create the agent build folder within $GOPATH
RUN mkdir -p /go/src/github.com/DataDog/datadog-agent

# Force umask to 0022
RUN echo "umask 0022" >> /root/.bashrc

ENTRYPOINT ["/entrypoint.sh"]
32 changes: 10 additions & 22 deletions rpm-armhf/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ RUN echo "${CACERT_BUNDLE_SHA256} /cacert.pem" | sha256sum --check
FROM ${BASE_IMAGE}

# Build Args
ARG GIMME_GO_VERSION=1.19.9
ARG GO_VERSION=1.19.9
ARG DD_TARGET_ARCH=armhf

# Environment
ENV GOPATH /go
ENV GIMME_GO_VERSION $GIMME_GO_VERSION
ENV GO_VERSION $GO_VERSION
ENV DD_TARGET_ARCH $DD_TARGET_ARCH

# configure yum and rpm for running on non-armv7l architectures
Expand Down Expand Up @@ -81,33 +81,21 @@ RUN curl -sSL -o get-rvm.sh https://raw.githubusercontent.com/rvm/rvm/1.29.12/bi
RUN /bin/bash -l -c "rvm requirements"
RUN /bin/bash -l -c "rvm install --with-arch='armv7-a' -C '--build' -C 'arm-linux-gnueabihf' 2.7 && rvm cleanup all"
RUN /bin/bash -l -c "gem install bundler --no-document"
RUN echo 'source /usr/local/rvm/scripts/rvm' >> /root/.bashrc

# Pip & Invoke
COPY ./python-packages-versions.txt /python-packages-versions.txt
COPY ./setup_python.sh /setup_python.sh
COPY ./requirements.txt ./requirements-py2.txt /
RUN ./setup_python.sh

# Gimme
RUN curl -sL -o /bin/gimme https://raw.githubusercontent.com/travis-ci/gimme/v1.5.4/gimme
RUN echo "03b295636d4e22870b6f6e9bc06a71d65311ae90d3d48cbc7071f82dd5837fbc /bin/gimme" | sha256sum --check
RUN chmod +x /bin/gimme

# GIMME_ARCH = GOARCH, so must be a valid entry from `goarchlist` here:
# https://github.com/golang/go/blob/master/src/go/build/syslist.go
# Also see https://github.com/travis-ci/gimme/blob/master/gimme#L880
RUN GIMME_ARCH=arm gimme $GIMME_GO_VERSION

COPY ./gobin.sh /etc/profile.d/

# Entrypoint
COPY ./entrypoint.sh /
RUN chmod +x /entrypoint.sh

# create the agent build folder within $GOPATH
RUN mkdir -p /go/src/github.com/DataDog/datadog-agent
# Go
RUN curl -sL -o /tmp/golang.tar.gz https://go.dev/dl/go$GO_VERSION.linux-armv6l.tar.gz \
&& echo "4a613b3c9d74975470096c5323e54db0f5f0f56dd0471fc1a7ee04e559032dc4 /tmp/golang.tar.gz" | sha256sum --check \
&& tar -C /usr/local -xzf /tmp/golang.tar.gz \
&& rm -f /tmp/golang.tar.gz
ENV PATH="/usr/local/go/bin:${PATH}"
ENV PATH="${GOPATH}/bin:${PATH}"

# Force umask to 0022
RUN echo "umask 0022" >> /root/.bashrc

ENTRYPOINT ["/entrypoint.sh"]
28 changes: 10 additions & 18 deletions rpm-x64/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ ARG BASE_IMAGE
# Build Args
ARG GIT_VERSION=2.10.1
ARG GIT_SHA256="78553f786f1a66cb68983c170be482558028a3376056c0f2ed366f331b1e35f2"
ARG GIMME_GO_VERSION=1.19.9
ARG GO_VERSION=1.19.9
ARG IBM_MQ_VERSION=9.2.4.0
ARG IBM_MQ_SHA256="d0d583eba72daf20b3762976f8831c2e23150ace90509520e12f8cda5b5bdb49"
ARG CMAKE_VERSION=3.23.0
Expand All @@ -37,7 +37,7 @@ ARG GCC_SHA256=ab1974017834430de27fd803ade4389602a7d6ca1362496c57bef384b2a4cb07
ENV GOPATH /go
ENV GIT_VERSION $GIT_VERSION
ENV GIT_SHA256 $GIT_SHA256
ENV GIMME_GO_VERSION $GIMME_GO_VERSION
ENV GO_VERSION $GO_VERSION
ENV IBM_MQ_VERSION $IBM_MQ_VERSION
ENV IBM_MQ_SHA256 $IBM_MQ_SHA256
ENV CMAKE_VERSION $CMAKE_VERSION
Expand Down Expand Up @@ -163,6 +163,7 @@ RUN curl -sSL -o get-rvm.sh https://raw.githubusercontent.com/rvm/rvm/1.29.12/bi
RUN /bin/bash -l -c "rvm requirements"
RUN /bin/bash -l -c "rvm install 2.6 --with-openssl-dir=${CONDA_PATH} && rvm cleanup all"
RUN /bin/bash -l -c "gem install bundler --no-document"
RUN echo 'source /usr/local/rvm/scripts/rvm' >> /root/.bashrc

# Upgrade binutils
RUN curl -sL -O "https://ftp.gnu.org/gnu/binutils/binutils-${BINUTILS_VERSION}.tar.gz" \
Expand All @@ -177,13 +178,13 @@ RUN curl -sL -O "https://ftp.gnu.org/gnu/binutils/binutils-${BINUTILS_VERSION}.t
# Override updated linker system-wide
RUN ln -sf /usr/local/binutils/bin/ld /usr/bin/ld

# Gimme
RUN curl -sL -o /bin/gimme https://raw.githubusercontent.com/travis-ci/gimme/v1.5.4/gimme
RUN echo "03b295636d4e22870b6f6e9bc06a71d65311ae90d3d48cbc7071f82dd5837fbc /bin/gimme" | sha256sum --check
RUN chmod +x /bin/gimme
RUN gimme $GIMME_GO_VERSION
COPY ./gobin.sh /etc/profile.d/

# Go
RUN curl -sL -o /tmp/golang.tar.gz https://go.dev/dl/go$GO_VERSION.linux-amd64.tar.gz \
&& echo "e858173b489ec1ddbe2374894f52f53e748feed09dde61be5b4b4ba2d73ef34b /tmp/golang.tar.gz" | sha256sum --check \
&& tar -C /usr/local -xzf /tmp/golang.tar.gz \
&& rm -f /tmp/golang.tar.gz
ENV PATH="/usr/local/go/bin:${PATH}"
ENV PATH="${GOPATH}/bin:${PATH}"

# IBM MQ
RUN mkdir -p /opt/mqm \
Expand Down Expand Up @@ -280,14 +281,5 @@ RUN chmod +x /build-gcc.sh \
&& /build-gcc.sh \
&& rm /build-gcc.sh

# Entrypoint
COPY ./entrypoint.sh /
RUN chmod +x /entrypoint.sh

# create the agent build folder within $GOPATH
RUN mkdir -p /go/src/github.com/DataDog/datadog-agent

# Force umask to 0022
RUN echo "umask 0022" >> /root/.bashrc

ENTRYPOINT ["/entrypoint.sh"]
3 changes: 0 additions & 3 deletions setup_go.sh
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,6 @@ ln -sf /usr/local/binutils/bin/ld /usr/bin/ld

./all.bash

# Update PATH to include the built go binaries
echo 'export PATH="/goroot/bin:$PATH"' >> /root/.bashrc

# Remove gimme
rm -rf $HOME/.gimme
rm /bin/gimme
2 changes: 2 additions & 0 deletions setup_python.sh
Original file line number Diff line number Diff line change
Expand Up @@ -99,3 +99,5 @@ fi

# Add python3's invoke to the PATH even when ddpy3 is not active, since we want to use python3 invoke to run python2 tests
ln -s ${CONDA_PATH}/envs/ddpy3/bin/inv /usr/local/bin

echo "conda activate ddpy3" >> /root/.bashrc
28 changes: 10 additions & 18 deletions suse-x64/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ RUN echo "${CACERT_BUNDLE_SHA256} /cacert.pem" | sha256sum --check
FROM opensuse/archive:42.1

# Build Args
ARG GIMME_GO_VERSION=1.19.9
ARG GO_VERSION=1.19.9
ARG IBM_MQ_VERSION=9.2.4.0
ARG IBM_MQ_SHA256="d0d583eba72daf20b3762976f8831c2e23150ace90509520e12f8cda5b5bdb49"
ARG CMAKE_VERSION=3.23.0
Expand All @@ -31,7 +31,7 @@ ARG GCC_SHA256=ab1974017834430de27fd803ade4389602a7d6ca1362496c57bef384b2a4cb07

# Environment
ENV GOPATH /go
ENV GIMME_GO_VERSION $GIMME_GO_VERSION
ENV GO_VERSION $GO_VERSION
ENV IBM_MQ_VERSION $IBM_MQ_VERSION
ENV IBM_MQ_SHA256 $IBM_MQ_SHA256
ENV CMAKE_VERSION $CMAKE_VERSION
Expand All @@ -46,8 +46,6 @@ ENV GCC_VERSION $GCC_VERSION

ENV PATH="/opt/datadog/bin:${PATH}"

# Add our profile to the image that adds RVM and Conda to environment
COPY ./suse-x64/ddprofile.sh /etc/profile.d/
# Add /etc/profile to the image (taken from 42.3 image) -- it is missing from 42.1 the image
COPY ./suse-x64/profile /etc/profile

Expand Down Expand Up @@ -84,12 +82,13 @@ COPY --from=CURL_GETTER /cacert.pem /etc/ssl/certs/ca-certificates.crt
COPY --from=CURL_GETTER /curl-amd64 /opt/datadog/bin/curl
RUN chmod +x /opt/datadog/bin/curl

# Gimme
RUN curl -sL -o /bin/gimme https://raw.githubusercontent.com/travis-ci/gimme/v1.5.4/gimme
RUN echo "03b295636d4e22870b6f6e9bc06a71d65311ae90d3d48cbc7071f82dd5837fbc /bin/gimme" | sha256sum --check
RUN chmod +x /bin/gimme
RUN gimme $GIMME_GO_VERSION
COPY ./gobin.sh /etc/profile.d/
# Go
RUN curl -sL -o /tmp/golang.tar.gz https://go.dev/dl/go$GO_VERSION.linux-amd64.tar.gz \
&& echo "e858173b489ec1ddbe2374894f52f53e748feed09dde61be5b4b4ba2d73ef34b /tmp/golang.tar.gz" | sha256sum --check \
&& tar -C /usr/local -xzf /tmp/golang.tar.gz \
&& rm -f /tmp/golang.tar.gz
ENV PATH="/usr/local/go/bin:${PATH}"
ENV PATH="${GOPATH}/bin:${PATH}"

# Add systemd headers
COPY ./rpm-headers/systemd /usr/include/systemd
Expand Down Expand Up @@ -121,6 +120,7 @@ RUN curl -sSL -o get-rvm.sh https://raw.githubusercontent.com/rvm/rvm/1.29.12/bi
RUN bash -l -c "rvm autolibs disable" # do not try to fetch requirements from system repos
RUN bash -l -c "rvm install 2.7 --with-openssl-dir=${CONDA_PATH} && rvm cleanup all"
RUN bash -l -c "gem install bundler --no-document"
RUN echo 'source /usr/local/rvm/scripts/rvm' >> /root/.bashrc

# CMake
RUN set -ex \
Expand Down Expand Up @@ -172,14 +172,6 @@ RUN curl -sL -o /tmp/rpm-4.14.3.tar.bz2 http://ftp.rpm.org/releases/rpm-4.14.x/r
RUN mkdir -p /usr/local/var/lib/rpm \
&& cp /var/lib/rpm/Packages /usr/local/var/lib/rpm/Packages \
&& /usr/local/bin/rpm --rebuilddb
# Entrypoint
COPY ./entrypoint.sh /
RUN chmod +x /entrypoint.sh

# create the agent build folder within $GOPATH
RUN mkdir -p /go/src/github.com/DataDog/datadog-agent

# Force umask to 0022
RUN echo "umask 0022" >> /root/.bashrc

ENTRYPOINT ["/entrypoint.sh"]