Skip to content

Commit

Permalink
Attempting to make kind a little less Docker Centric
Browse files Browse the repository at this point in the history
I believe that kind should be able to run with Podman as
well as Docker.

Signed-off-by: Daniel J Walsh <[email protected]>
  • Loading branch information
rhatdan committed Dec 5, 2018
1 parent 4d6990e commit 967498f
Show file tree
Hide file tree
Showing 27 changed files with 182 additions and 128 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
</a> <a href="https://goreportcard.com/report/sigs.k8s.io/kind"><img alt="Go Report Card" src="https://goreportcard.com/badge/sigs.k8s.io/kind" /></a>


`kind` is a tool for running local Kubernetes clusters using Docker container "nodes".
`kind` is a tool for running local Kubernetes clusters using container "nodes".
`kind` is primarily designed for testing Kubernetes 1.11+, initially targeting the [conformance tests].

If you have [go] and [docker] installed `go get sigs.k8s.io/kind && kind create cluster` is all you need!
Expand All @@ -17,7 +17,7 @@ If you have [go] and [docker] installed `go get sigs.k8s.io/kind && kind create
`kind` consists of:
- Go [packages][packages] implementing [cluster creation][cluster package], [image build][build package], etc.
- A command line interface ([`kind`][kind cli]) built on these packages.
- Docker [image(s)][images] written to run systemd, Kubernetes, etc.
- Container [image(s)][images] written to run systemd, Kubernetes, etc.
- [`kubetest`][kubetest] integration also built on these packages (WIP)

`kind` bootstraps each "node" with [kubeadm][kubeadm]. For more details see [the design documentation][design doc].
Expand Down
6 changes: 3 additions & 3 deletions docs/design/base-image.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# The Base Image

The ["base" image][base image] is a small-ish Docker image for running
The ["base" image][base image] is a small-ish container image for running
nested containers, systemd, and kubernetes components.

To do this we need to set up an environment that will meet the CRI
(currently just docker) and systemd's particular needs. Documentation for each
step we take is inline to the image's [Dockerfile][dockerfile]),
but essentially:

- we preinstall tools / packages expected by systemd / Docker / Kubernetes other
- we preinstall tools / packages expected by systemd / Docker | Podman / Kubernetes other
than Kubernetes itself

- we install a custom entrypoint that allows us to perform some actions before
Expand All @@ -17,7 +17,7 @@ the container truly boots
- we set up a systemd service to forward journal logs to the container tty

- we do a few tricks to minimize unnecessary services and inform systemd that it
is in docker (see the [Dockerfile][dockerfile])
is in a container (see the [Dockerfile][dockerfile])

This image is based on a minimal debian image (currently `k8s.gcr.io/debian-base`)
due to high availability of tooling.
Expand Down
16 changes: 8 additions & 8 deletions docs/design/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This is the root design documentation for `kind`. See also the project
## Overview

`kind` or **k**ubernetes **in** **d**ocker is a suite of tooling for local
Kubernetes "clusters" where each "node" is a Docker container.
Kubernetes "clusters" where each "node" is a container.
`kind` is targeted at testing Kubernetes.

`kind` is divided into go packages implementing most of the functionality, a
Expand Down Expand Up @@ -35,10 +35,10 @@ In practice kind looks something like this:
Clusters are managed by logic in [`pkg/cluster`][pkg/cluster], which the
`kind` cli wraps.

Each "cluster" is identified by an internal but well-known [docker object label](https://docs.docker.com/config/labels-custom-metadata/) key, with the cluster
Each "cluster" is identified by an internal but well-known [container object label](https://docs.docker.com/config/labels-custom-metadata/) key, with the cluster
name / ID as the value on each "node" container.

We initially offload this type of state into the containers and Docker.
We initially offload this type of state into the containers.
Similarly the container names are automatically managed by `kind`, though
we will select over labels instead of names because these are less brittle and
are properly namespaced. Doing this also avoids us needing to manage anything
Expand All @@ -63,17 +63,17 @@ For more see [node-image.md][node-image.md].

### Cluster Creation

Each "node" runs as a docker container. Each container initially boots to a
Each "node" runs as a container. Each container initially boots to a
pseudo "paused" state, with [the entrypoint][entrypoint] waiting for `SIGUSR1`.
This allows us to manipulate and inspect the container with `docker exec ...`
This allows us to manipulate and inspect the container with `containerEngine exec ...`
and other tools prior to starting systemd and all of the components.

This setup includes fixing mounts and pre-loading saved docker images.
This setup includes fixing mounts and pre-loading saved container images.

Once the nodes are sufficiently prepared, we signal the entrypoint to actually
"boot" the node.

We then wait for the Docker service to be ready on the node before running
We then wait for the Docker service to be ready if we are using Docker on the node before running
`kubeadm` to initialize the node.

Once kubeadm has booted, we export the [KUBECONFIG][kubeconfig], then apply
Expand All @@ -84,7 +84,7 @@ At this point users can test Kubernetes by using the exported kubeconfig.

### Cluster Deletion

All "node" containers in the cluster are tagged with docker labels identifying
All "node" containers in the cluster are tagged with container labels identifying
the cluster by the chosen cluster name (default is "1"), to delete a cluster
we can simply list and delete containers with this label.

Expand Down
6 changes: 3 additions & 3 deletions docs/design/node-image.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# The Node Image

The ["node" image][node image] is a Docker image for running
The ["node" image][node image] is a container image for running
nested containers, systemd, and Kubernetes components.

This image is built on top of the ["base" image][base image].
Expand All @@ -15,7 +15,7 @@ provides most of the tools statically needed for a kubernetes deployment
(eg `systemd`), variants of this image have the following properties:

- `/kind/images/` contains various `*.tar` files which are
[Docker image archives][docker image archives],
[Container image archives][container image archives],
these images will be loaded by the cluster tooling prior to running `kubeadm`

- `kubeadm`, `kubectl`, `kubelet` are in the path
Expand All @@ -35,6 +35,6 @@ each "node" container with [kubeadm][kubeadm].
[base image]: ./base-image.md
[build package]: ./../../pkg/build
[cluster package]: ./../../pkg/cluster
[docker image archives]: https://docs.docker.com/engine/reference/commandline/save/
[container image archives]: https://docs.docker.com/engine/reference/commandline/save/
[systemd service]: https://www.freedesktop.org/software/systemd/man/systemd.service.html
[kubeadm]: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/
6 changes: 3 additions & 3 deletions docs/design/overview.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Overview
`kind` is a command-line application for building Kubernetes clusters using
Docker containers as "nodes."
containers as "nodes."


## CLI
Expand All @@ -27,7 +27,7 @@ and setup logging.
├── pkg
│   ├── build # Build and manage images
│   ├── cluster # Build and manage clusters
│   ├── docker # Interact with Docker
│   ├── container # Interact with container engines
│   ├── exec # Execute commands
│   ├── fs # Interact with the host file system
│   ├── kustomize # Work with embedded kustomize commands
Expand All @@ -36,7 +36,7 @@ and setup logging.
```
`kind` commands rely on the functionality of the [packages directory][pkg].
Here, you will find everything needed to build container images for `kind`;
create clusters from these images; interact with the Docker engine and file system; customize configuration files; and logging.
create clusters from these images; interact with the container engine and file system; customize configuration files; and logging.



Expand Down
15 changes: 12 additions & 3 deletions docs/devel/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Before you being you must have:
* [GitHub account][github]
* `git`
* `go`
* `Docker`
* `Docker` | `Podman`

## Task 1. Read the Kubernetes community guidelines
Make sure to read you read the [Kubernetes community guidelines][community].
Expand All @@ -32,22 +32,31 @@ $ go version
```
This documentation is written using Go version 1.11+.

## Task 4. Install or upgrade Docker
## Task 4. Install or upgrade Docker or Podman
If you haven't already, install the
[Docker software using the instructions for your operating system][docker].
or
[Podman software using the instructions for your operating system][podman].
If you have an existing installation, check your version and make sure you have
the latest Docker.
the latest Docker or Podman

To check if `docker` is has been installed:
```
$ docker --version
```
This documentation is written using Docker version 18.09.0.

To check if `podman` is has been installed:
```
$ podman --version
```
This documentation is written using Podman version 0.12.1.



[github]: https://github.com/
[community]: https://github.com/kubernetes/community
[contributor]: https://github.com/kubernetes/community/blob/master/contributors/guide/README.md
[golang]: https://golang.org/doc/install
[docker]: https://docs.docker.com/install/#supported-platforms
[podman]: https://github.com/containers/libpod/install
2 changes: 1 addition & 1 deletion docs/todo.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ A non-exhaustive list of tasks (in no-particular order) includes:
- [x] node lifecycle hooks
- [ ] more advanced network configuration (not docker0)
- [ ] support for other CRI within the "node" containers (containerd, cri-o)
- [ ] move all docker functionality into a common package (`pkg/docker`) [WIP]
- [ ] move all container engine functionality into a common package (`pkg/container`) [WIP]
- [ ] potentially move this to using the docker client library
- [x] log dumping functionality / cli commands
- [ ] support audit logging
Expand Down
3 changes: 2 additions & 1 deletion hack/build/push-base.sh
Original file line number Diff line number Diff line change
Expand Up @@ -43,5 +43,6 @@ IMAGE="kindest/base:${TAG}"
(set -x; "${KIND}" build base-image --image="${IMAGE}")

# push
docker push "${IMAGE}"
ENGINE=$(command -v podman 2> /dev/null || echo docker)
${ENGINE} push "${IMAGE}"

7 changes: 4 additions & 3 deletions hack/build/push-node.sh
Original file line number Diff line number Diff line change
Expand Up @@ -44,10 +44,11 @@ set -x

# re-tag with kubernetes version
IMG="kindest/node:${TAG}"
KUBE_VERSION="$(docker run --rm --entrypoint=cat "${IMG}" /kind/version)"
docker tag "${IMG}" "kindest/node:${KUBE_VERSION}"
ENGINE=$(command -v podman 2> /dev/null || echo docker)
KUBE_VERSION="$(${ENGINE} run --rm --entrypoint=cat "${IMG}" /kind/version)"
${ENGINE} tag "${IMG}" "kindest/node:${KUBE_VERSION}"

# push
docker push kindest/node:"${KUBE_VERSION}"
${ENGINE} push kindest/node:"${KUBE_VERSION}"


13 changes: 7 additions & 6 deletions pkg/build/base/base.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import (
log "github.com/sirupsen/logrus"

"sigs.k8s.io/kind/pkg/build/base/sources"
"sigs.k8s.io/kind/pkg/container"
"sigs.k8s.io/kind/pkg/exec"
"sigs.k8s.io/kind/pkg/fs"
)
Expand Down Expand Up @@ -109,7 +110,7 @@ func (c *BuildContext) Build() (err error) {
return err
}

// then the actual docker image
// then the actual container image
return c.buildImage(buildDir)
}

Expand All @@ -120,7 +121,7 @@ func (c *BuildContext) buildEntrypoint(dir string) error {
entrypointDest := filepath.Join(dir, "entrypoint", "entrypoint")

cmd := exec.Command(c.goCmd, "build", "-o", entrypointDest, entrypointSrc)
// TODO(bentheelder): we may need to map between docker image arch and GOARCH
// TODO(bentheelder): we may need to map between container image arch and GOARCH
cmd.SetEnv("GOOS=linux", "GOARCH="+c.arch)

// actually build
Expand All @@ -136,14 +137,14 @@ func (c *BuildContext) buildEntrypoint(dir string) error {

func (c *BuildContext) buildImage(dir string) error {
// build the image, tagged as tagImageAs, using the our tempdir as the context
cmd := exec.Command("docker", "build", "-t", c.image, dir)
log.Info("Starting Docker build ...")
cmd := exec.Command(container.Engine, "build", "-t", c.image, dir)
log.Info("Starting container image build ...")
exec.InheritOutput(cmd)
err := cmd.Run()
if err != nil {
log.Errorf("Docker build Failed! %v", err)
log.Errorf("Container image build Failed! %v", err)
return err
}
log.Info("Docker build completed.")
log.Info("Container image build completed.")
return nil
}
Loading

0 comments on commit 967498f

Please sign in to comment.