Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pre-load CNI #331

Merged
merged 2 commits into from
Feb 23, 2019
Merged

pre-load CNI #331

merged 2 commits into from
Feb 23, 2019

Conversation

BenTheElder
Copy link
Member

@BenTheElder BenTheElder commented Feb 23, 2019 •
edited
Loading

With this patch a cluster can be created fully offline. Fixes #200

A test image is at: gcr.io/bentheelder-kind-dev/kindest/node:v1.13.3

How this works:

  • pkg/build/node now contains some private constants for the CNI, with instructions for udpating them
  • we write a fixed CNI manifest to a well-known-location when building the node image
  • we pull a fixed set of CNI images and pre-load them when building the node image
  • when creating a cluster we detect if the manifest exists in the container, if it does we will load that, if not we will fall back to the previous CNI install mechanism

Along the way I cleaned up the image build slightly, and improved the ergonomics of the exec.Command interface to allow method chaining.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Feb 23, 2019
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BenTheElder

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 23, 2019
This was referenced Feb 23, 2019
Closed
Closed
neolit123
neolit123 reviewed Feb 23, 2019
View reviewed changes
pkg/cluster/internal/create/kubeadm-init.go
// we found the default manifest, install that
// the images should already be loaded along with kubernetes
if err := node.Command(
"kubectl", "create", "--kubeconfig=/etc/kubernetes/admin.conf",
Copy link
Member

@neolit123 neolit123 Feb 23, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bellow we are using the "/bin/sh", "-c", ... pattern for kubectl.
this seems better.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we still pass the kubeconfig via the flag? this is the most explicit way to pass kubeconfig, I don't think using bash is better, we just need to easily exactly mimic the command they specify in their docs

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would've strongly preferred to not invoke bash, but the upstream preferred installation from a non-fixed manifest is:

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

https://www.weave.works/docs/net/latest/kubernetes/kube-addon/#-installation

pkg/cluster/internal/create/kubeadm-init.go
).Run(); err != nil {
return errors.Wrap(err, "failed to apply overlay network")
}
} else {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

given now the images are preloaded and manifest is written on the node, shouldn't we exit with an error instead of going for the download from web method as a fallback?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the command below does download from the web?
kind gracefully degrades in the presence of older images without this to the behavior expected for those images

this is in line with our principles of:

  • graceful degredation
  • avoiding breaking changes
  • avoiding assumptions

neolit123
neolit123 reviewed Feb 23, 2019
View reviewed changes
Copy link
Member

@neolit123 neolit123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/hold

@k8s-ci-robot k8s-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Feb 23, 2019
@BenTheElder
Copy link
Member Author

/hold cancel
Will move that image and file a follow up to default to it.

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 23, 2019
@k8s-ci-robot k8s-ci-robot merged commit 4119e0b into kubernetes-sigs:master Feb 23, 2019
This was referenced Feb 23, 2019
Merged
Closed
@0xmichalis
Copy link

@BenTheElder a howto in the docs would be nice for this - it's not clear what are the exact deps/tools we need to have offline to deploy a cluster.

@BenTheElder
Copy link
Member Author

@Kargakis noted -- right now you just need to install from HEAD and pull the node image. docker pull kindest/node:v1.13.3, kind create cluster should work

@0xmichalis
Copy link

Yeap, I confirm I can create offline clusters now. Thanks!

🎉 🍾 🎉 🍾

@BenTheElder
Copy link
Member Author

Awesome! We'll definitely document this and ways to work with it 👍

@BenTheElder BenTheElder mentioned this pull request Feb 24, 2019
Closed
@BenTheElder BenTheElder deleted the prepull-cni branch February 24, 2019 22:21
@tao12345666333 tao12345666333 mentioned this pull request Nov 6, 2019
Merged
stg-0 added a commit to stg-0/kind that referenced this pull request Sep 29, 2023
@stg-0
Bump cluster-operator prerrelease (kubernetes-sigs#331)
32203d2 
* Bump cluster-operator prerrelease

* Update DEPENDENCIES

* Update pkg/cluster/internal/create/actions/createworker/provider.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@neolit123 neolit123 neolit123 left review comments

@krzyzacy krzyzacy Awaiting requested review from krzyzacy

Assignees

@neolit123 neolit123

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

pre-load CNI
4 participants
@BenTheElder @k8s-ci-robot @0xmichalis @neolit123