Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build images from tarballs #396

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# ignore the kind binary at the root of the project
/kind

# build outputs
/_output
/_artifacts
Expand Down
2 changes: 1 addition & 1 deletion cmd/kind/build/nodeimage/nodeimage.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ func NewCommand() *cobra.Command {
}
cmd.Flags().StringVar(
&flags.BuildType, "type",
"docker", "build type, one of [bazel, docker, apt]",
"docker", "build type, one of [bazel, docker, apt, tar]",
)
cmd.Flags().StringVar(
&flags.Image, "image",
Expand Down
3 changes: 3 additions & 0 deletions pkg/build/kube/aptbits.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,9 @@ func (b *AptBits) Build() error {
return nil
}

// Clean implements the Bits.Clean interface.
func (b *AptBits) Clean() {}

// Paths implements Bits.Paths
func (b *AptBits) Paths() map[string]string {
return map[string]string{}
Expand Down
3 changes: 3 additions & 0 deletions pkg/build/kube/bazelbuildbits.go
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,9 @@ func (b *BazelBuildBits) Build() error {
return buildVersionFile(b.kubeRoot)
}

// Clean implements the Bits.Clean interface.
func (b *BazelBuildBits) Clean() {}

func (b *BazelBuildBits) findPaths(bazelGoosGoarch string) map[string]string {
// https://docs.bazel.build/versions/master/output_directories.html
binDir := filepath.Join(b.kubeRoot, "bazel-bin")
Expand Down
4 changes: 4 additions & 0 deletions pkg/build/kube/bits.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@ type Bits interface {
// TODO(bentheelder): eliminate install, make install file-copies only,
// support cross-building
Install(InstallContext) error
// Clean is called after the image has been built and gives a Bits
// implementation an opportunity to clean up after itself.
Clean()
}

// InstallContext should be implemented by users of Bits
Expand All @@ -59,6 +62,7 @@ type InstallContext interface {
// "apt" -> NewAptBits(kubeRoot)
// "bazel" -> NewBazelBuildBits(kubeRoot)
// "docker" or "make" -> NewDockerBuildBits(kubeRoot)
// "tar" -> NewTarBits(kubeRoot)
func NewNamedBits(name string, kubeRoot string) (bits Bits, err error) {
bitsImpls.Lock()
fn, ok := bitsImpls.impls[name]
Expand Down
6 changes: 5 additions & 1 deletion pkg/build/kube/dockerbuildbits.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,10 @@ import (
"os"
"path"
"path/filepath"
"sigs.k8s.io/kind/pkg/util"
"strings"

"sigs.k8s.io/kind/pkg/util"

"github.com/pkg/errors"

"sigs.k8s.io/kind/pkg/exec"
Expand Down Expand Up @@ -86,6 +87,9 @@ func (b *DockerBuildBits) Build() error {
return buildVersionFile(b.kubeRoot)
}

// Clean implements the Bits.Clean interface.
func (b *DockerBuildBits) Clean() {}

// binary and image build when we have `make quick-release-images` support
func (b *DockerBuildBits) build() error {
// build binaries
Expand Down
84 changes: 84 additions & 0 deletions pkg/build/kube/files.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
/*
Copyright 2019 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package kube

import (
"io/ioutil"
"net/http"

"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
)

const kubeletServiceURI = `https://raw.githubusercontent.com/kubernetes/release/master/debian/xenial/kubelet/lib/systemd/system/kubelet.service`

const kubeletService = `[Unit]
Description=kubelet: The Kubernetes Node Agent
Documentation=http://kubernetes.io/docs/

[Service]
ExecStart=/usr/bin/kubelet
Restart=always
StartLimitInterval=0
RestartSec=10

[Install]
WantedBy=multi-user.target
`

const tenKubeadmConfURI = `https://raw.githubusercontent.com/kubernetes/release/master/debian/xenial/kubeadm/channel/stable/etc/systemd/system/kubelet.service.d/post-1.10/10-kubeadm.conf`

const tenKubeadmConf = `# Note: This dropin only works with kubeadm and kubelet v1.11+
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use
# the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/default/kubelet
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

baking in the unit/drop-in is not ideal. because kind now has to maintain and sync with the upstream systemd files...
we have deb/rpm specs scattered in k/k and k/release. kind probably shouldn't manage it's own copies.

i would HTTP download them from github:
https://github.com/kubernetes/release/blob/master/debian/xenial/kubelet/lib/systemd/system/kubelet.service
https://github.com/kubernetes/release/blob/master/debian/xenial/kubeadm/channel/stable/etc/systemd/system/kubelet.service.d/post-1.10/10-kubeadm.conf

(1.15 is hopefully going to nuke the above, though).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @neolit123,

I was just following @BenTheElder's // TODO from the BazelBits code. Still, I've no issue downloading them either. Ben?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @neolit123,

I've addressed your feedback here. I first try to get the files at the remote URIs, and if that fails I default back to the constant values.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

waiting on Ben's comments on this one. pulling from the release repo is not ideal too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @neolit123,

Sounds good. I'm happy to take this in any direction. I figured the constants were what @BenTheElder was thinking since Kind already embeds the Weave manifest as a constant.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The weave constant is not per-kubernetes version. This one is a bit more problematic.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Downloading it separately is also not super ideal, will have to poke around the tarballs a bit. There should be a way to consistently obtain this 🤔

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The TODO about our own config is stale and wrong but the overall situation for these has not been good :(

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @BenTheElder,

The weave constant is not per-kubernetes version. This one is a bit more problematic.

And yet I also cannot find the affected files in any of the k/k repos except for release. So while they may be specific to the k/k version, they're not seemingly available per build. I initially wanted to use the version grokked from kubernetes.tar.gz and get the service and kubadm conf file specific for that version. Then I realized those were not available.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And yet I also cannot find the affected files in any of the k/k repos except for release.

there are systemd unit files in kubernetes/kubernetes in the build directory, the non-versioned k/release files are considered a bug and have caused many problems (and they may change inbetween releases happening, so checking in a static version here may not be right)

`

func getKubeletServiceBytes() []byte {
return getRemoteOrDefaultBytes(kubeletServiceURI, kubeletService)
}

func getTenKubeadmConfBytes() []byte {
return getRemoteOrDefaultBytes(tenKubeadmConfURI, tenKubeadmConf)
}

func getRemoteOrDefaultBytes(uri, defaultContent string) []byte {
resp, err := http.Get(uri)
if err == nil && resp.StatusCode != 200 {
err = errors.Errorf("%s", resp.Status)
}
if err != nil {
log.WithError(err).Debugf(
"error reading %s; using default content", uri)
return []byte(defaultContent)
}
defer resp.Body.Close()
buf, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.WithError(err).Debugf(
"error reading %s; using default content", uri)
return []byte(defaultContent)
}
return buf
}
Loading