Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build cloud docker images for elastic-agent #28134

Merged
merged 21 commits into from
Oct 6, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .ci/packaging.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -321,10 +321,12 @@ def tagAndPush(Map args = [:]) {
}
// supported image flavours
def variants = ["", "-oss", "-ubi8"]
//

if(beatName == 'elastic-agent'){
variants.add("-complete")
variants.add("-cloud")
}

variants.each { variant ->
tags.each { tag ->
// TODO:
Expand Down
2 changes: 1 addition & 1 deletion Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -459,9 +459,9 @@ def tagAndPush(Map args = [:]) {
// supported image flavours
def variants = ["", "-oss", "-ubi8"]

// only add complete variant for the elastic-agent
if(beatName == 'elastic-agent'){
variants.add("-complete")
variants.add("-cloud")
}

variants.each { variant ->
Expand Down
32 changes: 32 additions & 0 deletions dev-tools/packaging/packages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -433,6 +433,7 @@ shared:
docker_entrypoint: 'docker-entrypoint.elastic-agent.tmpl'
user: '{{ .BeatName }}'
linux_capabilities: ''
image_name: ''
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forced to redefine this particular arg, otherwise:

 => => naming to docker.elastic.co/beats/elastic-agent-cloud:8.0.0                                                                                                                                             0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
package ran for 2m12.830616488s
Error: failed building elastic-agent type=docker for platform=linux/amd64: failed to prepare build: expanding template '/Users/vmartinez/work/src/github.com/elastic/beats/dev-tools/packaging/templates/docker/Dockerfile.elastic-agent.tmpl' to 'build/package/elastic-agent/elastic-agent-linux-amd64.docker/docker-build/Dockerfile': failed to expand template: template: /Users/vmartinez/work/src/github.com/elastic/beats/dev-tools/packaging/templates/docker/Dockerfile.elastic-agent.tmpl:26:16: executing "/Users/vmartinez/work/src/github.com/elastic/beats/dev-tools/packaging/templates/docker/Dockerfile.elastic-agent.tmpl" at <.image_name>: map has no entry for key "image_name"

files:
'elastic-agent.yml':
source: 'elastic-agent.docker.yml'
Expand All @@ -449,6 +450,11 @@ shared:
from: 'arm64v8/centos:7'
buildFrom: 'arm64v8/centos:7'

- &agent_docker_cloud_spec
<<: *agent_docker_spec
extra_vars:
image_name: '{{.BeatName}}-cloud'

# Deb/RPM spec for community beats.
- &deb_rpm_spec
<<: *common
Expand Down Expand Up @@ -1007,6 +1013,19 @@ specs:
'{{.BeatName}}{{.BinaryExt}}':
source: ./build/golang-crossbuild/{{.BeatName}}-{{.GOOS}}-{{.Platform.Arch}}{{.BinaryExt}}

# Cloud specific docker image
- os: linux
arch: amd64
types: [docker]
spec:
<<: *agent_docker_spec
<<: *agent_docker_cloud_spec
<<: *elastic_docker_spec
<<: *elastic_license_for_binaries
Comment on lines +1016 to +1024
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I used a previous defined spec (like the complete variant), then docker images were not correctly generated:


REPOSITORY                                                     TAG                                              IMAGE ID       CREATED             SIZE
docker.elastic.co/beats/elastic-agent-cloud                    8.0.0                                            32ed8b53650d   2 minutes ago       843MB
docker.elastic.co/beats/elastic-agent-ubi8                     8.0.0                                            a5a7dee0d5f2   2 minutes ago       299MB
docker.elastic.co/beats/elastic-agent-cloud-complete           8.0.0                                            efa622cf7b28   12 minutes ago      1.88GB

so the -cloud prefix was added to the complete one.

files:
'{{.BeatName}}{{.BinaryExt}}':
source: ./build/golang-crossbuild/{{.BeatName}}-{{.GOOS}}-{{.Platform.Arch}}{{.BinaryExt}}

- os: linux
arch: amd64
types: [docker]
Expand All @@ -1033,6 +1052,19 @@ specs:
'{{.BeatName}}{{.BinaryExt}}':
source: ./build/golang-crossbuild/{{.BeatName}}-{{.GOOS}}-{{.Platform.Arch}}{{.BinaryExt}}

# Cloud specific docker image
- os: linux
arch: arm64
types: [docker]
spec:
<<: *agent_docker_arm_spec
<<: *agent_docker_cloud_spec
<<: *elastic_docker_spec
<<: *elastic_license_for_binaries
files:
'{{.BeatName}}{{.BinaryExt}}':
source: ./build/golang-crossbuild/{{.BeatName}}-{{.GOOS}}-{{.Platform.Arch}}{{.BinaryExt}}

- os: linux
arch: arm64
types: [docker]
Expand Down
32 changes: 29 additions & 3 deletions dev-tools/packaging/templates/docker/Dockerfile.elastic-agent.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,11 @@ RUN mkdir -p {{ $beatHome }}/data {{ $beatHome }}/data/elastic-agent-{{ commit_s
{{- end }}
{{- range $i, $modulesd := .ModulesDirs }}
chmod 0775 {{ $beatHome}}/{{ $modulesd }} && \
{{- end }}
{{- if contains .image_name "-cloud" }}
mkdir -p /opt/filebeat /opt/metricbeat && \
tar xf {{ $beatHome }}/data/elastic-agent-*/downloads/metricbeat-*.tar.gz -C /opt/metricbeat --strip-components=1 && \
tar xf {{ $beatHome }}/data/elastic-agent-*/downloads/filebeat-*.tar.gz -C /opt/filebeat --strip-components=1 && \
{{- end }}
true

Expand All @@ -41,7 +46,6 @@ RUN case $(arch) in aarch64) YUM_FLAGS="-x bind-license";; esac; \
yum install -y epel-release && \
yum update -y $YUM_FLAGS && \
yum install -y jq && \

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated to this particular change, but let's keep it tidy

yum clean all && \
exit_code=0 && break || exit_code=$? && echo "yum error: retry $iter in 10s" && sleep 10; \
done; \
Expand Down Expand Up @@ -134,12 +138,26 @@ RUN mkdir /licenses
COPY --from=home {{ $beatHome }}/LICENSE.txt /licenses
COPY --from=home {{ $beatHome }}/NOTICE.txt /licenses

{{- if ne .user "root" }}
{{- if contains .image_name "-cloud" }}
COPY --from=home /opt /opt
{{- end }}

{{- if eq .user "root" }}
{{- if contains .image_name "-cloud" }}
# Generate folder for a stub command that will be overwritten at runtime
RUN mkdir /app
{{- end }}
{{- else }}
RUN groupadd --gid 1000 {{ .BeatName }}
RUN useradd -M --uid 1000 --gid 1000 --groups 0 --home {{ $beatHome }} {{ .user }}
{{- if (and (eq .Variant "complete") (not (contains .from "ubi-minimal"))) }}
RUN chown {{ .user }} $NODE_PATH
{{- end }}
{{- if contains .image_name "-cloud" }}
# Generate folder for a stub command that will be overwritten at runtime
RUN mkdir /app
RUN chown {{ .user }} /app
{{- end }}
{{- end }}
USER {{ .user }}

Expand Down Expand Up @@ -172,7 +190,6 @@ RUN cd {{$beatHome}}/.node \
{{- end }}



{{- range $i, $port := .ExposePorts }}
EXPOSE {{ $port }}
{{- end }}
Expand All @@ -183,4 +200,13 @@ EXPOSE {{ $port }}
ENV LIBBEAT_MONITORING_CGROUPS_HIERARCHY_OVERRIDE=/

WORKDIR {{ $beatHome }}

{{- if contains .image_name "-cloud" }}
ENTRYPOINT ["/usr/bin/tini", "--"]
CMD ["/app/apm.sh"]
# Generate a stub command that will be overwritten at runtime
RUN echo -e '#!/bin/sh\nexec /usr/local/bin/docker-entrypoint' > /app/apm.sh && \
chmod 0555 /app/apm.sh
{{- else }}
ENTRYPOINT ["/usr/bin/tini", "--", "/usr/local/bin/docker-entrypoint"]
{{- end }}