Bump base image

[aojea]

• Update the base image with PRs
• Added WSL2 fix for kubelet #2390
• mount node product_uuid and product_name in pod containers #2321
• upgrade pause image to 3.6 #2451
• Updated QEMU image and added cross build support for s390x kind binary and associated images #2494
• entrypoint: improve $userns support (alternative to #2492) #2498

Fixes: #2323, #2318, #2490

[aojea]

/assign @BenTheElder
I can't build the node image :(

[aojea]

/hold

https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kind/2465/pull-kind-conformance-parallel-1-17/1439353716925796352
pull-kind-conformance-parallel-1-17

I have the same problem locally

> runtime/cgo: pthread_create failed: Operation not permitted
SIGABRT: abort
PC=0x7f03710d6603 m=0 sigcode=18446744073709551610
goroutine 0 [idle]:
runtime: unknown pc 0x7f03710d6603
...
gs     0x0
runtime/cgo: pthread_create failed: Operation not permitted
SIGABRT: abort
PC=0x7fbde81fc603 m=0 sigcode=18446744073709551610
...
Image build Failed! Failed to load images [command "docker exec --privileged -i kind-build-1632004227-417728918 ctr --namespace=k8s.io images import --all-platforms --no-unpack -" failed with error: exit status 2, write |1: broken pipe]

it has to be related to cross building

[BenTheElder]

I get a different failure at HEAD now:

#17 0.108 Installing containerd ...
#17 0.162 curl: (6) getaddrinfo() thread failed to start
#17 0.162 
#17 1.164 curl: (6) getaddrinfo() thread failed to start
#17 1.164 
#17 3.166 curl: (6) getaddrinfo() thread failed to start
#17 3.166 
#17 7.170 curl: (6) getaddrinfo() thread failed to start
#17 7.170 
#17 15.18 curl: (6) getaddrinfo() thread failed to start
#17 15.18 
#17 31.19 curl: (6) getaddrinfo() thread failed to start

docker 20.10.2, kernel 5.10.40

[BenTheElder]

docker/buildx#772 (comment) ?

[BenTheElder]

Also reproduces on docker desktop currently, but I'm not at the latest version. Updating.

EDIT: not sure I can run docker desktop anymore at work actually ... forgot about that.

[qinqon]

Looks like the fix is in place ? docker/buildx#772 (comment)

[aojea]

🤕 https://www.mail-archive.com/[email protected]/msg5953123.html

[qinqon]

🤕 https://www.mail-archive.com/[email protected]/msg5953123.html

Another one?

[aojea]

/test all
something is wrong with 1.17

https://pastebin.com/wccujCtG

[aojea]

/test all

[aojea]

I think that the problem is that the krte image used in 1.17 is too old

[aojea]

@AkihiroSuda some of the building fails trying to use ctr , are you going to backport this in containerd?
containerd/containerd@55923da

[AkihiroSuda]

@AkihiroSuda some of the building fails trying to use ctr , are you going to backport this in containerd? containerd/containerd@55923da

That commit has been already backported in v1.5.6
https://github.com/containerd/containerd/commits/v1.5.6

[AkihiroSuda]

https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kind/2465/pull-kind-conformance-parallel-1-17/1448552573672886272/build-log.txt

Image build Failed! Failed to load images [command "docker exec --privileged -i kind-build-1634197355-636563126 ctr --namespace=k8s.io images import --all-platforms --no-unpack -" failed with error: exit status 2, write |1: broken pipe]
runtime/cgo: pthread_create failed: Operation not permitted

The error looks like an issue of the host Docker rather than ctr.

The Docker (Moby) PR (moby/moby#42681) was already cherry-picked into to v20.10 branch (moby/moby#42836) targeting at v20.10.10, but v20.10.10 is not released yet.

[aojea]

indeed, we have so many nested layers that is hard to know 😄
let me try this , I think that the problem is in the image used to build the node

diff --git a/pkg/build/nodeimage/buildcontext.go b/pkg/build/nodeimage/buildcontext.go
index 5b409df0..f07ca217 100644
--- a/pkg/build/nodeimage/buildcontext.go
+++ b/pkg/build/nodeimage/buildcontext.go
@@ -360,6 +360,7 @@ func (c *buildContext) createBuildContainer() (id string, err error) {
                        "--entrypoint=sleep",
                        "--name=" + id,
                        "--platform=" + dockerBuildOsAndArch(c.arch),
+                       "--security-opt", "seccomp=unconfined", // ignore seccomp
                },
                []string{
                        "infinity", // sleep infinitely to keep the container around

[aojea]

The Docker (Moby) PR (moby/moby#42681) was already cherry-picked into to v20.10 branch (moby/moby#42836) targeting at v20.10.10, but v20.10.10 is not released yet.

I think most distros has already patched it, but I expect users hitting it in kind and reporting issues, we have to add this to the release notes

[aojea]

Awesome, it was that, now everything works, the job that was failing here pass now
And it can builds node images for all kubernetes versions since 1.14
https://github.com/aojea/kind-images/runs/3892473052?check_suite_focus=true
/hold cancel
@BenTheElder we are ready for release

[BenTheElder]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aojea, BenTheElder

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [BenTheElder,aojea]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[BenTheElder]

@aojea please bump the default node image to one built with this base image X the latest k8s patch release

[rodnymolina]

And it can builds node images for all kubernetes versions since 1.14 ...

@BenTheElder, @aojea, could you please help me understand the typical procedure followed by KinD to have new node images being re-created when the base one is bumped? Can't find any info about this topic ...

I see that a new kindest/node:v1.22.2 image has been published, but what about other tags also supported by KinD's current v0.11.1 release (e.g. v1.21.x, v1.20.x, etc)? Are those images created only during release cycles? Sure, I can create them myself, but I was hoping to be able to make use of KinD's official images once that this PR got merged ...

Thanks.

[BenTheElder]

I see that a new kindest/node:v1.22.2 image has been published, but what about other tags also supported by KinD's current v0.11.1 release (e.g. v1.21.x, v1.20.x, etc)? Are those images created only during release cycles? Sure, I can create them myself, but I was hoping to be able to make use of KinD's official images once that this PR got merged ...

There's a few issues tracking this. Currently pre-built images are primarily published upon releasing the binary.

[aojea]

@rodnymolina if is for testing you can use the ones I've created for testing, they are published in
https://quay.io/repository/aojea/kindnode?tab=tags , the tag is the stable version 1.19, 1.20, ...

[rodnymolina]

Thanks @BenTheElder and @aojea. More than testing I'm interested in having this issue fixed for the existing KinD-in-Sysbox users. As I said, I can build the images myself, but it's best for everyone to rely on the official ones.

[felipecrs]

Is there any ETA for the next version of KinD which will include this?

[BenTheElder]

Probably later this week. We don't have any outstanding known regressions now, but Antonio and I have some other obligations early this week.

[felipecrs]

That would be awesome. Thank you!