Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add hack for cgroup v2 to sandbox entrypoints #1175

Merged
merged 1 commit into from
Jun 24, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 9 additions & 12 deletions docker/sandbox/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM golang:1.16.0-alpine3.13 AS go_builder_
FROM golang:1.16.5-alpine3.13 AS go_builder_

# Install dependencies
RUN apk add --no-cache build-base git make
Expand All @@ -12,31 +12,28 @@ RUN git clone -b ${BUILDKIT_CLI_FOR_KUBECTL_VERSION} --single-branch --depth 1 h
&& make -C ${GOPATH}/src/github.com/vmware-tanzu/buildkit-cli-for-kubectl ${INSTALL_DIR}/linux/kubectl-build BIN_DIR=${INSTALL_DIR} VERSION=${BUILDKIT_CLI_FOR_KUBECTL_VERSION}


FROM alpine:3.13.2 AS base_

# Install dependencies
RUN apk add --no-cache curl
FROM alpine:3.13.5 AS base_

# Make directory to store artifacts
RUN mkdir -p /flyteorg/bin /flyteorg/share

# Install k3s
ARG K3S_VERSION="v1.20.2%2Bk3s1"
ARG K3S_VERSION="v1.21.1%2Bk3s1"
RUN wget -q -O /flyteorg/bin/k3s https://github.com/k3s-io/k3s/releases/download/${K3S_VERSION}/k3s \
&& chmod +x /flyteorg/bin/k3s

# Install flytectl
RUN wget -q -O - https://raw.githubusercontent.com/flyteorg/flytectl/master/install.sh | BINDIR=/flyteorg/bin sh -s

# Install buildkit-cli-for-kubectl
COPY --from=go_builder_ /install/linux/ /flyteorg/bin/

# Install flytectl
RUN curl https://raw.githubusercontent.com/flyteorg/flytectl/master/install.sh | sh

# Copy flyte manifest
COPY deployment/sandbox/flyte_generated.yaml /flyteorg/share/flyte_generated.yaml

# Copy scripts
RUN cp /bin/flytectl /flyteorg/bin/
COPY docker/sandbox/kubectl docker/sandbox/wait-for-flyte.sh /flyteorg/bin/
COPY docker/sandbox/kubectl docker/sandbox/cgroup-v2-hack.sh docker/sandbox/wait-for-flyte.sh /flyteorg/bin/


FROM base_ AS default

Expand All @@ -62,7 +59,7 @@ EXPOSE 30081 30082 30084
ENTRYPOINT ["tini", "flyte-entrypoint.sh"]


FROM docker:20.10.3-dind AS dind
FROM docker:20.10.7-dind AS dind

# Install dependencies
RUN apk add --no-cache bash git make tini
Expand Down
15 changes: 15 additions & 0 deletions docker/sandbox/cgroup-v2-hack.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/sh

###############################################################################################################
# DISCLAIMER #
# Copied from https://github.com/moby/moby/blob/ed89041433a031cafc0a0f19cfe573c31688d377/hack/dind#L28-L37 #
# Moby License Apache 2.0: https://github.com/moby/moby/blob/ed89041433a031cafc0a0f19cfe573c31688d377/LICENSE #
###############################################################################################################
if [ -f /sys/fs/cgroup/cgroup.controllers ]; then
# move the processes from the root group to the /init group,
# otherwise writing subtree_control fails with EBUSY.
mkdir -p /sys/fs/cgroup/init
busybox xargs -rn1 < /sys/fs/cgroup/cgroup.procs > /sys/fs/cgroup/init/cgroup.procs || :
# enable controllers
sed -e 's/ / +/g' -e 's/^/+/' <"/sys/fs/cgroup/cgroup.controllers" >"/sys/fs/cgroup/cgroup.subtree_control"
fi
5 changes: 4 additions & 1 deletion docker/sandbox/flyte-entrypoint-default.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
#!/bin/sh

set -e
set -euo pipefail

# Apply cgroup v2 hack
cgroup-v2-hack.sh

trap 'pkill -P $$' EXIT

Expand Down
5 changes: 4 additions & 1 deletion docker/sandbox/flyte-entrypoint-dind.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
#!/bin/sh

set -e
set -euo pipefail

# Apply cgroup v2 hack
cgroup-v2-hack.sh

trap 'pkill -P $$' EXIT

Expand Down