Merge pull request #995 from ktock/integration-docs

Add document about integrations of eStargz with other tools

README.md

@@ -47,6 +47,7 @@ Stargz Snapshotter is a **non-core** sub-project of containerd.
 
 - For more details about stargz snapshotter plugin and its configuration, refer to [Containerd Stargz Snapshotter Plugin Overview](/docs/overview.md).
 - For more details about setup lazy pulling of eStargz with containerd, CRI-O, Podman, systemd, etc., refer to [Install Stargz Snapshotter and Stargz Store](./docs/INSTALL.md).
+- For more details about integration status of eStargz with tools in commuinty, refer to [Integration of eStargz with other tools](./docs/integration.md)
 
 For using stargz snapshotter on kubernetes nodes, you need the following configuration to containerd as well as run stargz snapshotter daemon on the node.
 We assume that you are using containerd (> v1.4.2) as a CRI runtime.

docs/integration.md

@@ -0,0 +1,184 @@
+# Integration of eStargz with other tools
+
+This document lists links and information about integrations of stargz-snapshotter with tools in commuinty.
+
+You can refer to [issue #258 "Tracker issue for adoption status"](https://github.com/containerd/stargz-snapshotter/issues/258) for the list of the latest status of these integrations.
+
+## Kubernetes
+
+To use stargz snapshotter on Kubernetes nodes, you need to use containerd as the CRI runtime.
+You also need to run stargz snapshotter on the node.
+
+### Kind
+
+See [`/README.md#quick-start-with-kubernetes`](/README.md#quick-start-with-kubernetes).
+
+### k3s
+
+k3s >= v1.22 supports stagz-snapshotter as an experimental feature.
+`--snapshotter=stargz` for k3s server and agent enables this feature.
+
+```
+k3s server --snapshotter=stargz
+```
+
+Refer to [k3s docs](https://docs.k3s.io/advanced#enabling-lazy-pulling-of-estargz-experimental) for more details.
+
+The following is a quick demo using [k3d](https://github.com/k3d-io/k3d) (k3s in Docker).
+
+```console
+$ k3d cluster create mycluster --k3s-arg='--snapshotter=stargz@server:*;agent:*'
+$ cat <<'EOF' | kubectl --context=k3d-mycluster apply -f -
+apiVersion: v1
+kind: Pod
+metadata:
+  name: nodejs
+spec:
+  containers:
+  - name: nodejs-stargz
+    image: ghcr.io/stargz-containers/node:17.8.0-esgz
+    command: ["node"]
+    args:
+    - -e
+    - var http = require('http');
+      http.createServer(function(req, res) {
+        res.writeHead(200);
+        res.end('Hello World!\n');
+      }).listen(80);
+    ports:
+    - containerPort: 80
+EOF
+$ kubectl --context=k3d-mycluster get po nodejs -w
+$ kubectl --context=k3d-mycluster port-forward nodejs 8080:80 &
+$ curl 127.0.0.1:8080
+Hello World!
+$ k3d cluster delete mycluster
+```
+
+### Google Kubernetes Engine
+
+There is no node image includes stargz snapshotter by default as of now so you need to manually customize the nodes.
+
+A brief instrcution of enabling stargz snapshotter is the following:
+
+- Create a Kubernetes cluster using containerd-supported Linux node images like `ubuntu_containerd`. containerd must be >= v1.4.2.
+- SSH into each node and install stargz snapshotter following [`./INSTALL.md`](./INSTALL.md#install-stargz-snapshotter-for-containerd-with-systemd). You need this installation on all worker nodes.
+- Optionally apply configuration to allow stargz-snapshotter to access private registries following [`./overview.md`](./overview.md#authentication).
+
+### Amazon Elastic Kubernetes Service
+
+There is no AMI includes stargz snapshotter by default as of now so you need to manually customize the nodes.
+
+A brief instrcution of enabling stargz snapshotter is the following:
+
+- Create a Kubernetes cluster using containerd-supported Linux AMIs. containerd must be >= v1.4.2. e.g. Amazon EKS optimized Amazon Linux AMIs with [containerd runtime bootstrap flag](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html).
+- SSH into each node and install stargz snapshotter following [`./INSTALL.md`](./INSTALL.md#install-stargz-snapshotter-for-containerd-with-systemd). You need this installation on all worker nodes.
+- Optionally apply configuration to allow stargz-snapshotter to access private registries following [`./overview.md`](./overview.md#authentication).
+
+## CRI runtimes
+
+### containerd
+
+See [`./INSTALL.md`](./INSTALL.md#install-stargz-snapshotter-for-containerd-with-systemd)
+
+> :information_source: There is also a doc for [integration with firecracker-containerd](https://github.com/firecracker-microvm/firecracker-containerd/blob/24f1fcf99ebf6edcb94edd71a2affbcdae6b08e7/docs/remote-snapshotter-getting-started.md).
+
+### CRI-O
+
+See [`./INSTALL.md`](./INSTALL.md#install-stargz-store-for-cri-opodman-with-systemd).
+
+## High-level container engines
+
+### Docker
+
+Docker Desktop 4.12.0 "Containerd Image Store (Beta)" uses stargz-snapshotter.
+Refer to [Docker documentation](https://docs.docker.com/desktop/containerd/).
+
+### nerdctl
+
+See the [docs in nerdctl](https://github.com/containerd/nerdctl/blob/main/docs/stargz.md).
+
+### Podman
+
+See [`./INSTALL.md`](./INSTALL.md#install-stargz-store-for-cri-opodman-with-systemd).
+
+## Image builders
+
+### BuildKit
+
+#### Building eStargz
+
+BuildKit >= v0.10 supports creating eStargz images.
+See [`README.md`](/README.md#building-estargz-images-using-buildkit) for details.
+
+#### Lazy pulling of eStargz
+
+BuildKit >= v0.8 supports stargz-snapshotter and can perform lazy pulling of eStargz-formatted base images during build.
+`--oci-worker-snapshotter=stargz` flag enables this feature.
+
+You can try this feature using Docker Buildx as the following.
+
+```
+$ docker buildx create --use --name lazy-builder --buildkitd-flags '--oci-worker-snapshotter=stargz'
+$ docker buildx inspect --bootstrap lazy-builder
+```
+
+The following is a sample Dockerfile that uses eStargz-formatted golang image (`ghcr.io/stargz-containers/golang:1.18-esgz`) as the base image.
+
+```Dockerfile
+FROM ghcr.io/stargz-containers/golang:1.18-esgz AS dev
+COPY ./hello.go /hello.go
+RUN go build -o /hello /hello.go
+
+FROM scratch
+COPY --from=dev /hello /
+ENTRYPOINT [ "/hello" ]
+```
+
+Put the following Go source code in the context directory with naming it `hello.go`.
+
+```golang
+package main
+
+import "fmt"
+
+func main() {
+	fmt.Println("Hello, world!")
+}
+```
+
+The following build performs lazy pulling of the eStargz-formatted golang base image.
+
+```console
+$ docker buildx build --load -t hello /tmp/ctx/
+$ docker run --rm hello
+Hello, world!
+```
+
+### Kaniko
+
+#### Building eStargz
+
+Kaniko >= v1.5.0 creates eStargz images when `GGCR_EXPERIMENT_ESTARGZ=1` is specified.
+See [`README.md`](/README.md#building-estargz-images-using-kaniko) for details.
+
+### ko
+
+ko >= v0.7.0 creates eStargz images when `GGCR_EXPERIMENT_ESTARGZ=1` is specified.
+Please see also [the docs in ko](https://github.com/ko-build/ko/blob/f70e3cad38c3bbd232f51604d922b8baff31144e/docs/advanced/faq.md#can-i-optimize-images-for-estargz-support).
+
+## P2P image distribution
+
+### IPFS
+
+See [`./ipfs.md`](./ipfs.md)
+
+### Dragonfly
+
+See [the document in Dragonfly project for how to use dragonfly with eStargz](https://d7y.io/docs/setup/integration/stargz/).
+
+## Registry-side conversion of eStargz
+
+### Harbor
+
+See the docs in Harbor: https://github.com/goharbor/acceleration-service

docs/ipfs.md

@@ -1,5 +1,7 @@
 # Running containers on IPFS (experimental)
 
+:information_source: This document isn't for Kubernetes environemnt. For information about node-to-node image sharing on Kubernetes, please refer to [the docs in nerdctl project](https://github.com/containerd/nerdctl/tree/main/examples/nerdctl-ipfs-registry-kubernetes).
+
 You can run OCI-compatible container images on IPFS with lazy pulling.
 
 To enable this feature, add the following configuration to `config.toml` of Stargz Snapsohtter (typically located at `/etc/containerd-stargz-grpc/config.toml`).

docs/overview.md

@@ -123,7 +123,7 @@ Stargz snapshotter doesn't share credentials with containerd so credentials spec
 
 #### CRI-based authentication
 
-Following configuration enables stargz snapshotter to pull private images on Kubernetes.
+Following configuration (typically located at `/etc/containerd-stargz-grpc/config.toml`) enables stargz snapshotter to pull private images on Kubernetes.
 The snapshotter works as a proxy of CRI Image Service and exposes CRI Image Service API on the snapshotter's unix socket (i.e. `/run/containerd-stargz-grpc/containerd-stargz-grpc.sock`).
 The snapshotter acquires registry creds by scanning requests.
 
@@ -140,7 +140,7 @@ image_service_path = "/run/containerd/containerd.sock"
 
 This is another way to enable lazy pulling of private images on Kubernetes.
 
-Following configuration enables stargz snapshotter to access to private registries using kubernetes secrets (type = `kubernetes.io/dockerconfigjson`) in the cluster using kubeconfig files.
+Following configuration (typically located at `/etc/containerd-stargz-grpc/config.toml`) enables stargz snapshotter to access to private registries using kubernetes secrets (type = `kubernetes.io/dockerconfigjson`) in the cluster using kubeconfig files.
 You can specify the path of kubeconfig file using `kubeconfig_path` option.
 It's no problem that the specified file doesn't exist when this snapshotter starts.
 In this case, snapsohtter polls the file until actually provided.