Attempting to make kind a little less Docker Centric

README.md

@@ -6,17 +6,17 @@
 
 # `kind` - `k`ubernetes `in` `d`ocker
 
-`kind` is a tool for running local Kubernetes clusters using Docker container "nodes".  
+`kind` is a tool for running local Kubernetes clusters using container "nodes".
 `kind` is primarily designed for testing Kubernetes 1.11+, initially targeting the [conformance tests].
 
-If you have [go] and [docker] installed `go get sigs.k8s.io/kind && kind create cluster` is all you need!
+If you have [go] and [docker] or [podman] installed `go get sigs.k8s.io/kind && kind create cluster` is all you need!
 
 <img src="https://gist.githubusercontent.com/BenTheElder/621bc321fc6d9506fd936feb36d32dd0/raw/7fe14e9d0929cab428929ca6c501abc990c07359/kind-create-cluster.gif" alt="2x speed `kind create cluster` demo" />
 
 `kind` consists of:
  - Go [packages][packages] implementing [cluster creation][cluster package], [image build][build package], etc.
  - A command line interface ([`kind`][kind cli]) built on these packages.
- - Docker [image(s)][images] written to run systemd, Kubernetes, etc.
+ - Container [image(s)][images] written to run systemd, Kubernetes, etc.
  - [`kubetest`][kubetest] integration also built on these packages (WIP)
 
 `kind` bootstraps each "node" with [kubeadm][kubeadm]. For more details see [the design documentation][design doc].  
@@ -30,8 +30,8 @@ You can install `kind` with `go get sigs.k8s.io/kind`. This will put `kind` in
 shown [here](https://golang.org/doc/code.html#GOPATH) if you encounter the error
 `kind: command not found` after installation.
 
-To use `kind`, you will need to [install docker].  
-Once you have docker running you can create a cluster with `kind create cluster`  
+To use `kind`, you will need to [install docker] or [install podman].
+Once you have installed podman or have the docker service running you can create a cluster with `kind create cluster`
 To delete your cluster use `kind delete cluster`
 
 <!--TODO(bentheelder): improve this part of the guide-->
@@ -84,6 +84,7 @@ Participation in the Kubernetes community is governed by the [Kubernetes Code of
 <!--links-->
 [go]: https://golang.org/
 [docker]: https://www.docker.com/
+[podman]: https://podman.io/
 [community page]: http://kubernetes.io/community/
 [Kubernetes Code of Conduct]: code-of-conduct.md
 [Go Report Card Badge]: https://goreportcard.com/badge/sigs.k8s.io/kind
@@ -106,5 +107,6 @@ Participation in the Kubernetes community is governed by the [Kubernetes Code of
 [#kind]: https://kubernetes.slack.com/messages/CEKK1KTN2/
 [docs/roadmap.md]: ./docs/roadmap.md
 [install docker]: https://docs.docker.com/install/
+[install podman]: https://github.com/containers/libpod/install.md
 [@BenTheElder]: https://github.com/BenTheElder
 [@munnerz]: https://github.com/munnerz

docs/design/base-image.md

@@ -1,14 +1,14 @@
 # The Base Image
 
-The ["base" image][base image] is a small-ish Docker image for running
+The ["base" image][base image] is a small-ish container image for running
 nested containers, systemd, and kubernetes components.
 
 To do this we need to set up an environment that will meet the CRI 
 (currently just docker) and systemd's particular needs. Documentation for each
 step we take is inline to the image's [Dockerfile][dockerfile]),
 but essentially:
 
-- we preinstall tools / packages expected by systemd / Docker / Kubernetes other
+- we preinstall tools / packages expected by systemd / container engine / Kubernetes other
 than Kubernetes itself
 
 - we install a custom entrypoint that allows us to perform some actions before
@@ -17,7 +17,7 @@ the container truly boots
 - we set up a systemd service to forward journal logs to the container tty
 
 - we do a few tricks to minimize unnecessary services and inform systemd that it
-is in docker (see the [Dockerfile][dockerfile])
+is in a container (see the [Dockerfile][dockerfile])
 
 This image is based on a minimal debian image (currently `k8s.gcr.io/debian-base`)
 due to high availability of tooling.  

docs/design/design.md

@@ -6,7 +6,7 @@ This is the root design documentation for `kind`. See also the project
 ## Overview
 
 `kind` or **k**ubernetes **in** **d**ocker is a suite of tooling for local 
-Kubernetes "clusters" where each "node" is a Docker container.
+Kubernetes "clusters" where each "node" is a container.
 `kind` is targeted at testing Kubernetes.
 
 `kind` is divided into go packages implementing most of the functionality, a
@@ -35,10 +35,10 @@ In practice kind looks something like this:
 Clusters are managed by logic in [`pkg/cluster`][pkg/cluster], which the
 `kind` cli wraps.
 
-Each "cluster" is identified by an internal but well-known [docker object label](https://docs.docker.com/config/labels-custom-metadata/) key, with the cluster
+Each "cluster" is identified by an internal but well-known [container object label](https://docs.docker.com/config/labels-custom-metadata/) key, with the cluster
 name / ID as the value on each "node" container.
 
-We initially offload this type of state into the containers and Docker. 
+We initially offload this type of state into the containers.
 Similarly the container names are automatically managed by `kind`, though
 we will select over labels instead of names because these are less brittle and
 are properly namespaced. Doing this also avoids us needing to manage anything
@@ -63,17 +63,17 @@ For more see [node-image.md][node-image.md].
 
 ### Cluster Creation
 
-Each "node" runs as a docker container. Each container initially boots to a
+Each "node" runs as a container. Each container initially boots to a
 pseudo "paused" state, with [the entrypoint][entrypoint] waiting for `SIGUSR1`.
-This allows us to manipulate and inspect the container with `docker exec ...`
+This allows us to manipulate and inspect the container with eg `docker exec ...`
 and other tools prior to starting systemd and all of the components.
 
-This setup includes fixing mounts and pre-loading saved docker images.
+This setup includes fixing mounts and pre-loading saved container images.
 
 Once the nodes are sufficiently prepared, we signal the entrypoint to actually
 "boot" the node.
 
-We then wait for the Docker service to be ready on the node before running
+We then wait for the Docker service to be ready if we are using Docker on the node before running
 `kubeadm` to initialize the node.
 
 Once kubeadm has booted, we export the [KUBECONFIG][kubeconfig], then apply
@@ -84,7 +84,7 @@ At this point users can test Kubernetes by using the exported kubeconfig.
 
 ### Cluster Deletion
 
-All "node" containers in the cluster are tagged with docker labels identifying
+All "node" containers in the cluster are tagged with container labels identifying
 the cluster by the chosen cluster name (default is "1"), to delete a cluster
 we can simply list and delete containers with this label.
 

docs/design/node-image.md

@@ -1,6 +1,6 @@
 # The Node Image
 
-The ["node" image][node image] is a Docker image for running
+The ["node" image][node image] is a container image for running
 nested containers, systemd, and Kubernetes components.
 
 This image is built on top of the ["base" image][base image].
@@ -15,7 +15,7 @@ provides most of the tools statically needed for a kubernetes deployment
 (eg `systemd`), variants of this image have the following properties:
 
 - `/kind/images/` contains various `*.tar` files which are 
-[Docker image archives][docker image archives],
+[Container image archives][container image archives],
 these images will be loaded by the cluster tooling prior to running `kubeadm`
 
 - `kubeadm`, `kubectl`, `kubelet` are in the path
@@ -35,6 +35,6 @@ each "node" container with [kubeadm][kubeadm].
 [base image]: ./base-image.md
 [build package]: ./../../pkg/build
 [cluster package]: ./../../pkg/cluster
-[docker image archives]: https://docs.docker.com/engine/reference/commandline/save/
+[container image archives]: https://docs.docker.com/engine/reference/commandline/save/
 [systemd service]: https://www.freedesktop.org/software/systemd/man/systemd.service.html
 [kubeadm]: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/

docs/design/overview.md

@@ -1,6 +1,6 @@
 # Overview
 `kind` is a command-line application for building Kubernetes clusters using
-Docker containers as "nodes."
+ containers as "nodes."
 
 
 ## CLI
@@ -27,7 +27,7 @@ and setup logging.
 ├── pkg
 │   ├── build     # Build and manage images
 │   ├── cluster   # Build and manage clusters
-│   ├── docker    # Interact with Docker
+│   ├── container # Interact with container engines
 │   ├── exec      # Execute commands
 │   ├── fs        # Interact with the host file system
 │   ├── kustomize # Work with embedded kustomize commands
@@ -36,7 +36,7 @@ and setup logging.
 ```
 `kind` commands rely on the functionality of the [packages directory][pkg].
 Here, you will find everything needed to build container images for `kind`;
-create clusters from these images; interact with the Docker engine and file system; customize configuration files; and logging.
+create clusters from these images; interact with the container engine and file system; customize configuration files; and logging.
 
 
 

docs/devel/README.md

@@ -6,7 +6,7 @@ Before you being you must have:
 * [GitHub account][github]
 * `git`
 * `go`
-* `Docker`
+* `Container Engine`
 
 ## Task 1. Read the Kubernetes community guidelines
 Make sure to read you read the [Kubernetes community guidelines][community].
@@ -32,22 +32,31 @@ $ go version
 ```
 This documentation is written using Go version 1.11+.
 
-## Task 4. Install or upgrade Docker
+## Task 4. Install or upgrade Docker or Podman
 If you haven't already, install the 
 [Docker software using the instructions for your operating system][docker].
+or
+[Podman software using the instructions for your operating system][podman].
 If you have an existing installation, check your version and make sure you have
-the latest Docker.
+the latest Docker or Podman
 
 To check if `docker` is has been installed:
 ```
 $ docker --version
 ```
 This documentation is written using Docker version 18.09.0.
 
+To check if `podman` is has been installed:
+```
+$ podman --version
+```
+This documentation is written using Podman version 0.12.1.
+
 
 
 [github]: https://github.com/
 [community]: https://github.com/kubernetes/community
 [contributor]: https://github.com/kubernetes/community/blob/master/contributors/guide/README.md
 [golang]: https://golang.org/doc/install
 [docker]: https://docs.docker.com/install/#supported-platforms
+[podman]: https://github.com/containers/libpod/install.md

docs/todo.md

@@ -0,0 +1,49 @@
+# TODO
+
+A non-exhaustive list of tasks (in no-particular order) includes:
+- [x] basic single "node" clusters
+- [x] multiple clusters per host / named clusters
+- [x] user guide(s)
+  - [x] more detailed user guides for common usage
+  - [x] more detailed user guides for more advanced usage
+- [ ] preflight checks [WIP]
+- [ ] multi-node clusters
+- [x] cli command to list clusters
+- [x] support for multiple kubernetes builds:
+  - [x] bazel build from source
+  - [x] docker / make build from source
+  - [x] apt (upstream / official release packages)
+    - [ ] support for selecting a non-default package version
+- [ ] kubetest ingregration [WIP]
+- [ ] improved logging and error handling [WIP]
+- [x] continuous integration
+- [x] publish pre-built images to a registry
+- [x] pre-load images that are not from the build / possibly build more images
+  - [x] etcd
+  - [x] coreDNS
+  - [x] Kubernetes images for released builds
+  - [ ] overlay network images?
+- [ ] support multiple overlay networks
+- [x] support advanced configuration via config file
+  - [x] kubeadm config template override
+  - [x] node lifecycle hooks
+- [ ] more advanced network configuration (not docker0)
+- [ ] support for other CRI within the "node" containers (containerd, cri-o)
+- [ ] move all container engine functionality into a common package (`pkg/container`) [WIP]
+ - [ ] potentially move this to using the docker client library
+- [x] log dumping functionality / cli commands
+  - [ ] support audit logging
+- [ ] fake out all internals and unit test [WIP]
+- [ ] support for local registries
+- [ ] support for mounting extra directories
+
+# Wishlist
+
+Longer term / continually appealing items:
+
+- Improved documentation
+- Support for architectures / platforms other than linux / amd64 for the node images
+- Support for client platforms other than docker on linux / docker for mac
+- Less privileged containers or sharing a CRI via something like [containerd namespaces](https://github.com/containerd/containerd/blob/master/docs/namespaces.md), generally
+ better isolation
+- HA kubeadm / multiple control plane nodes

hack/build/push-base.sh

@@ -43,5 +43,6 @@ IMAGE="kindest/base:${TAG}"
 (set -x; "${KIND}" build base-image --image="${IMAGE}")
 
 # push
-docker push "${IMAGE}"
+ENGINE=$(command -v podman 2> /dev/null || echo docker)
+${ENGINE} push "${IMAGE}"
 

hack/build/push-node.sh

@@ -44,10 +44,11 @@ set -x
 
 # re-tag with kubernetes version
 IMG="kindest/node:${TAG}"
-KUBE_VERSION="$(docker run --rm --entrypoint=cat "${IMG}" /kind/version)"
-docker tag "${IMG}" "kindest/node:${KUBE_VERSION}"
+ENGINE=$(command -v podman 2> /dev/null || echo docker)
+KUBE_VERSION="$(${ENGINE} run --rm --entrypoint=cat "${IMG}" /kind/version)"
+${ENGINE} tag "${IMG}" "kindest/node:${KUBE_VERSION}"
 
 # push
-docker push kindest/node:"${KUBE_VERSION}"
+${ENGINE} push kindest/node:"${KUBE_VERSION}"
 
 

pkg/build/base/base.go

@@ -24,6 +24,7 @@ import (
 	log "github.com/sirupsen/logrus"
 
 	"sigs.k8s.io/kind/pkg/build/base/sources"
+	"sigs.k8s.io/kind/pkg/container"
 	"sigs.k8s.io/kind/pkg/exec"
 	"sigs.k8s.io/kind/pkg/fs"
 )
@@ -109,7 +110,7 @@ func (c *BuildContext) Build() (err error) {
 		return err
 	}
 
-	// then the actual docker image
+	// then the actual container image
 	return c.buildImage(buildDir)
 }
 
@@ -120,7 +121,7 @@ func (c *BuildContext) buildEntrypoint(dir string) error {
 	entrypointDest := filepath.Join(dir, "entrypoint", "entrypoint")
 
 	cmd := exec.Command(c.goCmd, "build", "-o", entrypointDest, entrypointSrc)
-	// TODO(bentheelder): we may need to map between docker image arch and GOARCH
+	// TODO(bentheelder): we may need to map between container image arch and GOARCH
 	cmd.SetEnv("GOOS=linux", "GOARCH="+c.arch)
 
 	// actually build
@@ -136,14 +137,14 @@ func (c *BuildContext) buildEntrypoint(dir string) error {
 
 func (c *BuildContext) buildImage(dir string) error {
 	// build the image, tagged as tagImageAs, using the our tempdir as the context
-	cmd := exec.Command("docker", "build", "-t", c.image, dir)
-	log.Info("Starting Docker build ...")
+	cmd := exec.Command(container.Engine, "build", "-t", c.image, dir)
+	log.Info("Starting container image build ...")
 	exec.InheritOutput(cmd)
 	err := cmd.Run()
 	if err != nil {
-		log.Errorf("Docker build Failed! %v", err)
+		log.Errorf("Container image build Failed! %v", err)
 		return err
 	}
-	log.Info("Docker build completed.")
+	log.Info("Container image build completed.")
 	return nil
 }