Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes for ARM 32/64 #49

Merged
merged 22 commits into from
Jun 17, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
226403d
Fix 64 bit ARM builders
julien-lebot May 29, 2020
2aa2c0a
fix syntax error
julien-lebot May 29, 2020
49bb254
Add GIMME_ARCH in entrypoint
julien-lebot Jun 10, 2020
3c32698
Set GIMME_ARCH only for Arm builds
julien-lebot Jun 10, 2020
3f986df
Pass TARGET_ARCH to destination image
julien-lebot Jun 10, 2020
1e3965c
Test fake armv7l LD_PRELOAD trick
julien-lebot Jun 11, 2020
3e58734
Use Datadog fork of fakearmv7l
julien-lebot Jun 12, 2020
3b294e2
Replace TARGET_ARCH -> DD_TARGET_ARCH because it can conflict with so…
julien-lebot Jun 12, 2020
f0c61c4
Test fix RPM ARM builds
julien-lebot Jun 12, 2020
a85d6bf
Add newer gcc
julien-lebot Jun 12, 2020
3cc8dd5
Add conditional for ARM 64
julien-lebot Jun 12, 2020
11c8835
Separate RPM ARM64 / ARM32
julien-lebot Jun 15, 2020
ec5097b
Ensures processes in build image use fakearmv7l
julien-lebot Jun 15, 2020
794aa04
Keep DD_TARGET_ARCH env variable
julien-lebot Jun 16, 2020
ff9a575
Update rpm-arm/Dockerfile
julien-lebot Jun 16, 2020
8fdd209
Test passing variables from gitlab-ci directly
julien-lebot Jun 16, 2020
c21e5b6
Revert "Test passing variables from gitlab-ci directly"
julien-lebot Jun 16, 2020
fa0c26d
Update GO_VERSION
julien-lebot Jun 16, 2020
d645788
Add systemd dependency to RPM ARM64
julien-lebot Jun 16, 2020
93e3fed
Use correct RPM package for systemd
julien-lebot Jun 16, 2020
8f0a16a
Revert RPM ARM64 to this version https://github.com/DataDog/datadog-a…
julien-lebot Jun 16, 2020
46e965a
Update rpm-arm/Dockerfile
julien-lebot Jun 17, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,13 @@ build_rpm_x64:
image: arm64v8/docker:18.04
variables:
BASE_IMAGE: arm64v8/ubuntu:16.04
TARGET_ARCH: arm64v8
DD_TARGET_ARCH: arm64v8
script:
- apk add --update python py-pip
- pip install awscli
- aws --version
- eval "$(aws ecr get-login --region us-east-1 --no-include-email --registry-ids 486234852809)"
- docker build --build-arg BASE_IMAGE=$BASE_IMAGE --build-arg TARGET_ARCH=$TARGET_ARCH --tag 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/$IMAGE:v$CI_PIPELINE_ID-${CI_COMMIT_SHA:0:7} --file $DOCKERFILE .
- docker build --build-arg BASE_IMAGE=$BASE_IMAGE --build-arg DD_TARGET_ARCH=$DD_TARGET_ARCH --tag 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/$IMAGE:v$CI_PIPELINE_ID-${CI_COMMIT_SHA:0:7} --file $DOCKERFILE .
- docker push 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/$IMAGE:v$CI_PIPELINE_ID-${CI_COMMIT_SHA:0:7}
# For testing purposes
- docker tag 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/$IMAGE:v$CI_PIPELINE_ID-${CI_COMMIT_SHA:0:7} 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/$IMAGE:v$CI_PIPELINE_ID
Expand All @@ -61,22 +61,22 @@ build_deb_armhf:
DOCKERFILE: deb-arm/Dockerfile
IMAGE: deb_armhf
BASE_IMAGE: arm32v7/ubuntu:16.04
TARGET_ARCH: arm32v7
DD_TARGET_ARCH: arm32v7

build_rpm_arm64:
extends: .armbuild
variables:
DOCKERFILE: rpm-arm/Dockerfile
DOCKERFILE: rpm-arm64/Dockerfile
IMAGE: rpm_arm64
BASE_IMAGE: arm64v8/centos:7
BASE_IMAGE: amazonlinux:2.0.20181114
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did we change the base image? (just for future reference)

Copy link
Contributor Author

@julien-lebot julien-lebot Jun 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially it was amazonlinux, see #21. Amazon Linux however, doesn't provide 32 bit versions of their image, so we switched to CentOS 7 (which is what Amazon Linux is supposed to be based on I think). However, CentOS 7 is quite old and ships with GCC 4.8.5 - which causes issues when trying to build rtloader.
Hence we had two options:

  • Install a newer GCC on CentOS 7
  • Revert to the Amazon Linux image for 64 bit builds

We chose the latter because we don't build rtloader for 32 bit ARM, so it's simpler to revert to the 'known to work' image.


build_rpm_armhf:
extends: .armbuild
variables:
DOCKERFILE: rpm-arm/Dockerfile
IMAGE: rpm_armhf
BASE_IMAGE: arm32v7/centos:7
TARGET_ARCH: arm32v7
DD_TARGET_ARCH: arm32v7

build_system-probe_x64:
extends: .build
Expand Down Expand Up @@ -168,8 +168,8 @@ test_system-probe_arm64:
- $SHORT_CI_COMMIT_SHA = $($CI_COMMIT_SHA.Substring(0,7))
- $SRC_TAG = "v$CI_PIPELINE_ID-$SHORT_CI_COMMIT_SHA"
- $SRC_IMAGE = "486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/datadog-agent-buildimages/${IMAGE}:${SRC_TAG}"
- echo "build --build-arg BASE_IMAGE=$BASE_IMAGE --build-arg TARGET_ARCH=$TARGET_ARCH --build-arg WINDOWS_VERSION=$WINDOWS_VERSION --tag $SRC_IMAGE --file $DOCKERFILE ."
- powershell -Command "docker build --no-cache --build-arg BASE_IMAGE=$BASE_IMAGE --build-arg TARGET_ARCH=$TARGET_ARCH --build-arg WINDOWS_VERSION=$WINDOWS_VERSION --tag $SRC_IMAGE --file $DOCKERFILE ."
- echo "build --build-arg BASE_IMAGE=$BASE_IMAGE --build-arg DD_TARGET_ARCH=$DD_TARGET_ARCH --build-arg WINDOWS_VERSION=$WINDOWS_VERSION --tag $SRC_IMAGE --file $DOCKERFILE ."
- powershell -Command "docker build --no-cache --build-arg BASE_IMAGE=$BASE_IMAGE --build-arg DD_TARGET_ARCH=$DD_TARGET_ARCH --build-arg WINDOWS_VERSION=$WINDOWS_VERSION --tag $SRC_IMAGE --file $DOCKERFILE ."
- docker push $SRC_IMAGE
after_script:
- docker image prune -f
Expand All @@ -182,7 +182,7 @@ build_windows_1809_x64:
DOCKERFILE: windows/Dockerfile
IMAGE: windows_1809_x64
BASE_IMAGE: mcr.microsoft.com/dotnet/framework/runtime:4.8-20200114-windowsservercore-ltsc2019
TARGET_ARCH: x64
DD_TARGET_ARCH: x64
WINDOWS_VERSION: 1809
resource_group: windows_x64

Expand All @@ -193,7 +193,7 @@ build_windows_1809_x86:
DOCKERFILE: windows/Dockerfile
IMAGE: windows_1809_x86
BASE_IMAGE: mcr.microsoft.com/dotnet/framework/runtime:4.8-20200114-windowsservercore-ltsc2019
TARGET_ARCH: x86
DD_TARGET_ARCH: x86
WINDOWS_VERSION: 1809
resource_group: windows_x86

Expand All @@ -204,7 +204,7 @@ build_windows_1909_x64:
DOCKERFILE: windows/Dockerfile
IMAGE: windows_1909_x64
BASE_IMAGE: mcr.microsoft.com/dotnet/framework/runtime:4.8-20200310-windowsservercore-1909
TARGET_ARCH: x64
DD_TARGET_ARCH: x64
WINDOWS_VERSION: 1909
resource_group: windows_x64

Expand All @@ -215,7 +215,7 @@ build_windows_1909_x86:
DOCKERFILE: windows/Dockerfile
IMAGE: windows_1909_x86
BASE_IMAGE: mcr.microsoft.com/dotnet/framework/runtime:4.8-20200310-windowsservercore-1909
TARGET_ARCH: x86
DD_TARGET_ARCH: x86
WINDOWS_VERSION: 1909
resource_group: windows_x86

Expand Down
20 changes: 14 additions & 6 deletions deb-arm/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ ARG DD_PIP_VERSION=19.1
ARG DD_SETUPTOOLS_VERSION=41.0.1
ARG CMAKE_VERSION=3.14.4
ARG CLANG_VERSION=8.0.0
ARG TARGET_ARCH=arm64v8
ARG DD_TARGET_ARCH=arm64v8

# Environment
ENV GOPATH /go
Expand All @@ -18,8 +18,7 @@ ENV DD_PIP_VERSION $DD_PIP_VERSION
ENV DD_SETUPTOOLS_VERSION $DD_SETUPTOOLS_VERSION
ENV CMAKE_VERSION $CMAKE_VERSION
ENV CLANG_VERSION $CLANG_VERSION
ENV GIMME_ARCH $TARGET_ARCH

ENV DD_TARGET_ARCH $DD_TARGET_ARCH

RUN apt-get update && apt-get install -y fakeroot curl git procps bzip2 \
python2.7-dev build-essential pkg-config tar libsystemd-dev libkrb5-dev \
Expand All @@ -44,18 +43,27 @@ RUN pip install invoke distro==1.4.0 awscli==1.16.240
# Gimme
RUN curl -sL -o /bin/gimme https://raw.githubusercontent.com/travis-ci/gimme/master/gimme
RUN chmod +x /bin/gimme
RUN gimme $GIMME_GO_VERSION

# GIMME_ARCH = GOARCH, so must be a valid entry from `goarchlist` here:
# https://github.com/golang/go/blob/master/src/go/build/syslist.go
# Also see https://github.com/travis-ci/gimme/blob/master/gimme#L880
RUN if [ "$DD_TARGET_ARCH" = "arm64v8" ] ; then \
GIMME_ARCH=arm64 gimme $GIMME_GO_VERSION ; \
else \
GIMME_ARCH=arm gimme $GIMME_GO_VERSION ; \
fi

COPY ./gobin.sh /etc/profile.d/

# CMake. Pre-built using the build-cmake.sh script, to speed-up docker build.
RUN if [ "$TARGET_ARCH" = "arm64v8" ] ; then curl -sL -O https://dd-agent-omnibus.s3.amazonaws.com/cmake-${CMAKE_VERSION}-ubuntu-aarch64.tar.xz && \
RUN if [ "$DD_TARGET_ARCH" = "arm64v8" ] ; then curl -sL -O https://dd-agent-omnibus.s3.amazonaws.com/cmake-${CMAKE_VERSION}-ubuntu-aarch64.tar.xz && \
tar xf cmake-${CMAKE_VERSION}-ubuntu-aarch64.tar.xz --no-same-owner -kC / && \
rm cmake-${CMAKE_VERSION}-ubuntu-aarch64.tar.xz ; fi
ENV PATH="/opt/cmake/bin:$PATH"

# Install clang and llvm version 8. Pre-built because building takes ~4 hours.
# This was built from sources on centos 7, using the build-clang.sh script
RUN if [ "$TARGET_ARCH" = "arm64v8" ] ; then curl -sL -o clang_llvm.tar.xz https://dd-agent-omnibus.s3.amazonaws.com/clang%2Bllvm-${CLANG_VERSION}-aarch64-linux.tar.xz && \
RUN if [ "$DD_TARGET_ARCH" = "arm64v8" ] ; then curl -sL -o clang_llvm.tar.xz https://dd-agent-omnibus.s3.amazonaws.com/clang%2Bllvm-${CLANG_VERSION}-aarch64-linux.tar.xz && \
tar xf clang_llvm.tar.xz --no-same-owner -kC / && \
rm clang_llvm.tar.xz ; fi
ENV PATH="/opt/clang/bin:$PATH"
Expand Down
5 changes: 5 additions & 0 deletions entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ source /root/.bashrc

conda activate ddpy3 || true # We use system python3 on some images, allow this to fail

if [ "$DD_TARGET_ARCH" = "arm64v8" ] ; then
export GIMME_ARCH=arm64
elif [ "$DD_TARGET_ARCH" = "arm32v7" ] ; then
export GIMME_ARCH=arm
fi
eval "$(gimme)"

exec "$@"
44 changes: 22 additions & 22 deletions rpm-arm/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
## Valid archs are
# amd64, arm32v7, arm64v8, i386, ppc64le
ARG BASE_IMAGE=arm64v8/centos:7
ARG BASE_IMAGE=arm32v7/centos:7
FROM ${BASE_IMAGE}

# Build Args
Expand All @@ -9,7 +7,7 @@ ARG DD_PIP_VERSION=19.1
ARG DD_SETUPTOOLS_VERSION=41.0.1
ARG CMAKE_VERSION=3.14.4
ARG CLANG_VERSION=8.0.0
ARG TARGET_ARCH=arm64v8
ARG DD_TARGET_ARCH=arm32v7

# Environment
ENV GOPATH /go
Expand All @@ -18,21 +16,31 @@ ENV DD_PIP_VERSION $DD_PIP_VERSION
ENV DD_SETUPTOOLS_VERSION $DD_SETUPTOOLS_VERSION
ENV CMAKE_VERSION $CMAKE_VERSION
ENV CLANG_VERSION $CLANG_VERSION
ENV GIMME_ARCH $TARGET_ARCH
ENV DD_TARGET_ARCH $DD_TARGET_ARCH

# configure yum and rpm for running on non-armv7l architectures
RUN if [ "$TARGET_ARCH" = "arm32v7" ] ; then echo "armhfp" > /etc/yum/vars/basearch && \
RUN echo "armhfp" > /etc/yum/vars/basearch && \
echo "armv7hl" > /etc/yum/vars/arch && \
echo "armv7hl-redhat-linux-gnu" > /etc/rpm/platform ; fi
echo "armv7hl-redhat-linux-gnu" > /etc/rpm/platform

# Resolves a nasty NOKEY warning that appears when using yum
RUN if [ "$TARGET_ARCH" = "arm32v7" ] ; then rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 && \
rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-AltArch-Arm32 ; fi
RUN rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 && \
rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-AltArch-Arm32

RUN yum groupinstall -y development && yum -y install which perl-ExtUtils-MakeMaker ncurses-compat-libs git procps \
curl-devel expat-devel gettext-devel openssl-devel systemd-devel zlib-devel bzip2 glibc-static python-devel tar pkgconfig \
libtool autoconf policycoreutils-python

# This trick preloads a dynamic library that will replace the uname syscall to pretend the machine is armv7l
# The fakearmv7l library gets installed in "/usr/local/lib/libfakearmv7l.so"
RUN git clone --depth 1 https://github.com/DataDog/fakearmv7l ; \
cd fakearmv7l ; \
make ; \
make install ; \
cd .. ; \
rm -rf ./fakearmv7l ; \
export LD_PRELOAD="/usr/local/lib/libfakearmv7l.so"

# RVM
COPY ./rvm/gpg-keys /gpg-keys
RUN gpg --import /gpg-keys/*
Expand All @@ -49,21 +57,13 @@ RUN pip install invoke distro==1.4.0 awscli==1.16.240
# Gimme
RUN curl -sL -o /bin/gimme https://raw.githubusercontent.com/travis-ci/gimme/master/gimme
RUN chmod +x /bin/gimme
RUN gimme $GIMME_GO_VERSION
COPY ./gobin.sh /etc/profile.d/

# CMake. Pre-built using the build-cmake.sh script, to speed-up docker build.
RUN if [ "$TARGET_ARCH" = "arm64v8" ] ; then curl -sL -O https://dd-agent-omnibus.s3.amazonaws.com/cmake-${CMAKE_VERSION}-amzn-aarch64.tar.xz && \
tar xf cmake-${CMAKE_VERSION}-amzn-aarch64.tar.xz --no-same-owner -kC / && \
rm cmake-${CMAKE_VERSION}-amzn-aarch64.tar.xz ; fi
ENV PATH="/opt/cmake/bin:$PATH"
# GIMME_ARCH = GOARCH, so must be a valid entry from `goarchlist` here:
# https://github.com/golang/go/blob/master/src/go/build/syslist.go
# Also see https://github.com/travis-ci/gimme/blob/master/gimme#L880
RUN GIMME_ARCH=arm gimme $GIMME_GO_VERSION

# Install clang and llvm version 8. Pre-built because building takes ~4 hours.
# This was built from sources on centos 7, using the build-clang.sh script
RUN if [ "$TARGET_ARCH" = "arm64v8" ] ; then curl -sL -o clang_llvm.tar.xz https://dd-agent-omnibus.s3.amazonaws.com/clang%2Bllvm-${CLANG_VERSION}-aarch64-linux.tar.xz && \
tar xf clang_llvm.tar.xz --no-same-owner -kC / && \
rm clang_llvm.tar.xz ; fi
ENV PATH="/opt/clang/bin:$PATH"
COPY ./gobin.sh /etc/profile.d/

# Entrypoint
COPY ./entrypoint.sh /
Expand Down
67 changes: 67 additions & 0 deletions rpm-arm64/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
ARG BASE_IMAGE=amazonlinux:2.0.20181114
FROM ${BASE_IMAGE}

# Build Args
ARG GIMME_GO_VERSION=1.13.8
ARG DD_PIP_VERSION=19.1
ARG DD_SETUPTOOLS_VERSION=41.0.1
ARG CMAKE_VERSION=3.14.4
ARG CLANG_VERSION=8.0.0
ARG DD_TARGET_ARCH=arm64v8

# Environment
ENV GOPATH /go
ENV GIMME_GO_VERSION $GIMME_GO_VERSION
ENV DD_PIP_VERSION $DD_PIP_VERSION
ENV DD_SETUPTOOLS_VERSION $DD_SETUPTOOLS_VERSION
ENV CMAKE_VERSION $CMAKE_VERSION
ENV CLANG_VERSION $CLANG_VERSION
ENV DD_TARGET_ARCH $DD_TARGET_ARCH

RUN yum groupinstall -y development && yum -y install which perl-ExtUtils-MakeMaker ncurses-compat-libs git procps \
curl-devel expat-devel gettext-devel openssl-devel systemd-devel zlib-devel bzip2 glibc-static python-devel tar pkgconfig \
libtool autoconf policycoreutils-python

# RVM
COPY ./rvm/gpg-keys /gpg-keys
RUN gpg --import /gpg-keys/*
RUN rm -rf /gpg-keys
RUN curl -sSL https://get.rvm.io | bash -s stable
RUN /bin/bash -l -c "rvm requirements"
RUN /bin/bash -l -c "rvm install 2.3 && rvm cleanup all"
RUN /bin/bash -l -c "gem install bundler --no-document"

# Pip & Invoke
RUN curl "https://bootstrap.pypa.io/get-pip.py" | python2.7 - pip==${DD_PIP_VERSION} setuptools==${DD_SETUPTOOLS_VERSION}
RUN pip install invoke distro==1.4.0 awscli==1.16.240
Comment on lines +35 to +36
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use Python 3 here? Base image doesn't have it so it needs to be installed. Not a blocker but it would useful if we ever need a Python 3 only package.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I think we could have python 3 in this image, but I think this should be addressed as a separate PR


# Gimme
RUN curl -sL -o /bin/gimme https://raw.githubusercontent.com/travis-ci/gimme/master/gimme
RUN chmod +x /bin/gimme
RUN gimme $GIMME_GO_VERSION
COPY ./gobin.sh /etc/profile.d/

# CMake. Pre-built using the build-cmake.sh script, to speed-up docker build.
RUN curl -sL -O https://dd-agent-omnibus.s3.amazonaws.com/cmake-${CMAKE_VERSION}-amzn-aarch64.tar.xz && \
tar xf cmake-${CMAKE_VERSION}-amzn-aarch64.tar.xz --no-same-owner -kC / && \
rm cmake-${CMAKE_VERSION}-amzn-aarch64.tar.xz
ENV PATH="/opt/cmake/bin:$PATH"

# Install clang and llvm version 8. Pre-built because building takes ~4 hours.
# This was built from sources on centos 7, using the build-clang.sh script
RUN curl -sL -o clang_llvm.tar.xz https://dd-agent-omnibus.s3.amazonaws.com/clang%2Bllvm-${CLANG_VERSION}-aarch64-linux.tar.xz && \
tar xf clang_llvm.tar.xz --no-same-owner -kC / && \
rm clang_llvm.tar.xz
ENV PATH="/opt/clang/bin:$PATH"

# Entrypoint
COPY ./entrypoint.sh /
RUN chmod +x /entrypoint.sh

# create the agent build folder within $GOPATH
RUN mkdir -p /go/src/github.com/DataDog/datadog-agent

# Force umask to 0022
RUN echo "umask 0022" >> /root/.bashrc

ENTRYPOINT ["/entrypoint.sh"]