diff --git a/docs/guides/remote-driver.md b/docs/guides/remote-driver.md new file mode 100644 index 000000000000..2e03f94a87ef --- /dev/null +++ b/docs/guides/remote-driver.md @@ -0,0 +1,127 @@ +# Remote driver + +The buildx remote driver allows for more complex custom build workloads that +allow users to connect to external buildkit instances. This is useful for +scenarios that require manual management of the buildkit daemon, or where a +buildkit daemon is exposed from another source. + +To connect to a running buildkitd instance: + + $ docker buildx create \ + --name remote \ + --driver remote \ + tcp://localhost:1234 + +## Remote Buildkit over unix sockets + +In this scenario, we'll create a setup with buildkitd listening on a unix +socket, and have buildx connect through it. + +Firstly, ensure that [buildkit](https://github.com/moby/buildkit) is installed. +Then launch an instance of buildkitd: + + $ sudo ./buildkitd --group $(id -gn) --addr unix://$HOME/buildkitd.sock + +This will setup a socket in your home directory that you have read-write +permissions to. + + $ ls -lh + srw-rw---- 1 root user 0 May 5 11:04 /home/user/buildkitd.sock + +You can then connect buildx to it with the remote driver: + + $ buildx create \ + --name remote-unix \ + --driver remote \ + unix://$HOME/buildkitd.sock + +If you list available builders, you should then see `remote-unix` among them: + + $ buildx ls + NAME/NODE DRIVER/ENDPOINT STATUS PLATFORMS + test remote + test0 unix:///home/.../buildkitd.sock running linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386 + default * docker + default default running linux/amd64, linux/386 + +We can switch to this new builder as the default using `docker buildx use remote-unix`, +or specify it per build: + + $ docker buildx build --builder=remote-unix -t test --load . + +(remember that `--load` is neccessary when not using the default `docker` +driver, to load the build result into the docker daemon) + +## Remote Buildkit in Docker container + +In this scenario, we'll create a similar setup to the `docker-container` +driver, by manually booting a buildkit docker container and connecting to it +using the buildx remote driver. In most cases you'd probably just use the +`docker-container` driver directly, but imagine that for some reason you want +to manually manage the container instead of having buildx manage it for you. + +First, we need to generate certificates for buildkit - you can use the +[create-certs.sh](https://github.com/moby/buildkit/blob/master/examples/kubernetes/create-certs.sh) +script as a starting point. Note, that while it is *possible* to expose +buildkit over TCP without using TLS, it is **not recommened**, since this will +allow arbitrary access to buildkit without credentials. + +With our certificates generated in `.certs/`, we startup the container: + + $ docker run -d --rm \ + --name=remote-buildkitd \ + --privileged \ + -p 1234:1234 \ + -v $PWD/.certs:/certs \ + moby/buildkit:latest \ + --addr tcp://0.0.0.0:1234 \ + --tlscacert /certs/ca.pem \ + --tlscert /certs/daemon-cert.pem \ + --tlskey /certs/daemon-key.pem + +The above command starts a buildkit container and exposes the daemon's port +1234 to localhost. + +We can now connect to this running container using buildx: + + $ docker buildx create \ + --name remote-kubernetes \ + --driver remote \ + --driver-opt cacert=.certs/ca.pem,cert=.certs/client-cert.pem,key=.certs/client-key.pem,servername=... \ + tcp://buildkitd.default.svc:1234 + +# Remote Buildkit in Kubernetes + +In this scenario, we'll create a similar setup to the `kubernetes` driver by +manually creating a buildkit `Deployment`. While the `kubernetes` driver will +do this under-the-hood, it might sometimes be desirable to scale buildkit +manually. Additionally, when executing builds from inside Kubernetes pods, +the buildx builder will need to be recreated from within each pod or copied +between them. + +Firstly, we can create a kubernetes deployment of buildkitd, as per the +instructions [here](https://github.com/moby/buildkit/tree/master/examples/kubernetes). +Following the guide, we setup certificates for the buildkit daemon and client +(as above using [create-certs.sh](https://github.com/moby/buildkit/blob/master/examples/kubernetes/create-certs.sh)) +and create a `Deployment` of buildkit pods with a service that connects to +them. + +Assuming that the service is called `buildkitd`, we can create a remote builder +in buildx, ensuring that the listed certificate files are present: + + $ docker buildx create \ + --name remote-kubernetes \ + --driver remote \ + --driver-opt cacert=.certs/ca.pem,cert=.certs/client-cert.pem,key=.certs/client-key.pem \ + tcp://buildkitd.default.svc:1234 + +Note that the above will only work in-cluster (since the buildkit setup guide +only creates a ClusterIP service). To configure the builder to be accessible +remotely, you can use an appropriately configured Ingress, which is outside the +scope of this guide. + +Alternatively, to access remotely use the port forwarding mechanism in kubectl: + + $ kubectl port-forward svc/buildkitd 1234:1234 + +Then you can simply point the remote driver at `tcp://localhost:1234`.