QEMU on docker root-less runners

[grzesuav]

Description

I am using github runners in root-less mode - set up like described here -

When I try to use this action with following setup

      - name: Set up QEMU
        uses: docker/setup-qemu-action@v3
        with:
          platforms: linux/amd64,linux/arm64
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
        with:
          platforms: "linux/amd64,linux/arm64"
      - name: 'build image' 
        uses: docker/build-push-action@v6
        with:
          tags: ${{ env.ACR_REPO }}/actions-runner:${{ github.sha }}
          platforms: "linux/amd64,linux/arm64"
          context: provisioning/actions-runner
          push: true

it does not work, error as follows:

...
2024-09-19T15:03:11.5041547Z ##[endgroup]
2024-09-19T15:03:11.5266721Z Warning: DOCKER_HOST environment variable overrides the active context. To use a context, either set the global --context flag, or unset DOCKER_HOST environment variable.
2024-09-19T15:03:11.5268399Z NAME        DESCRIPTION                               DOCKER ENDPOINT                           ERROR
2024-09-19T15:03:11.5269541Z default *   Current DOCKER_HOST based configuration   unix:///home/runner/var/run/docker.sock   
2024-09-19T15:03:11.5359192Z ##[group]Run docker/setup-qemu-action@v3
2024-09-19T15:03:11.5360057Z with:
2024-09-19T15:03:11.5360617Z   platforms: linux/amd64,linux/arm64
2024-09-19T15:03:11.5361256Z   image: docker.io/tonistiigi/binfmt:latest
2024-09-19T15:03:11.5361891Z env:

...

2024-09-19T15:03:11.5375344Z ##[endgroup]
2024-09-19T15:03:11.7878431Z ##[group]Docker info
2024-09-19T15:03:11.7913507Z [command]/usr/bin/docker version
2024-09-19T15:03:11.8238346Z Client:
2024-09-19T15:03:11.8241184Z  Version:           27.1.1
2024-09-19T15:03:11.8241920Z  API version:       1.46
2024-09-19T15:03:11.8242618Z  Go version:        go1.21.12
2024-09-19T15:03:11.8243568Z  Git commit:        6312585
2024-09-19T15:03:11.8244323Z  Built:             Tue Jul 23 19:55:52 2024
2024-09-19T15:03:11.8245415Z  OS/Arch:           linux/amd64
2024-09-19T15:03:11.8246014Z  Context:           default
2024-09-19T15:03:11.8246321Z 
2024-09-19T15:03:11.8246823Z Server: Docker Engine - Community
2024-09-19T15:03:11.8247358Z  Engine:
2024-09-19T15:03:11.8247873Z   Version:          27.1.1
2024-09-19T15:03:11.8248506Z   API version:      1.46 (minimum version 1.24)
2024-09-19T15:03:11.8249254Z   Go version:       go1.21.12
2024-09-19T15:03:11.8249848Z   Git commit:       cc13f95
2024-09-19T15:03:11.8250439Z   Built:            Tue Jul 23 19:57:10 2024
2024-09-19T15:03:11.8251078Z   OS/Arch:          linux/amd64
2024-09-19T15:03:11.8251662Z   Experimental:     false
2024-09-19T15:03:11.8252196Z  containerd:
2024-09-19T15:03:11.8252703Z   Version:          v1.7.20
2024-09-19T15:03:11.8253741Z   GitCommit:        8fc6bcff51318944179630522a095cc9dbf9f353
2024-09-19T15:03:11.8254434Z  runc:
2024-09-19T15:03:11.8255178Z   Version:          1.1.13
2024-09-19T15:03:11.8255824Z   GitCommit:        v1.1.13-0-g58aa920
2024-09-19T15:03:11.8256478Z  docker-init:
2024-09-19T15:03:11.8257010Z   Version:          0.19.0
2024-09-19T15:03:11.8257518Z   GitCommit:        de40ad0
2024-09-19T15:03:11.8258066Z  rootlesskit:
2024-09-19T15:03:11.8258564Z   Version:          2.0.2
2024-09-19T15:03:11.8259136Z   ApiVersion:       1.1.1
2024-09-19T15:03:11.8259683Z   NetworkDriver:    vpnkit
2024-09-19T15:03:11.8260234Z   PortDriver:       builtin
2024-09-19T15:03:11.8260853Z   StateDir:         /tmp/rootlesskit3136240777
2024-09-19T15:03:11.8261478Z  vpnkit:
2024-09-19T15:03:11.8262057Z   Version:          7f0eff0dd99b576c5474de53b4454a157c642834
2024-09-19T15:03:11.8285775Z [command]/usr/bin/docker info
2024-09-19T15:03:11.8664218Z Client:
2024-09-19T15:03:11.8665493Z  Version:    27.1.1
2024-09-19T15:03:11.8666448Z  Context:    default
2024-09-19T15:03:11.8667450Z  Debug Mode: false
2024-09-19T15:03:11.8668285Z  Plugins:
2024-09-19T15:03:11.8669146Z   buildx: Docker Buildx (Docker Inc.)
2024-09-19T15:03:11.8670125Z     Version:  v0.16.2
2024-09-19T15:03:11.8671716Z     Path:     /usr/local/lib/docker/cli-plugins/docker-buildx
2024-09-19T15:03:11.8672526Z 
2024-09-19T15:03:11.8672886Z Server:
2024-09-19T15:03:11.8673660Z  Containers: 0
2024-09-19T15:03:11.8674450Z   Running: 0
2024-09-19T15:03:11.8675253Z   Paused: 0
2024-09-19T15:03:11.8676148Z   Stopped: 0
2024-09-19T15:03:11.8676902Z  Images: 0
2024-09-19T15:03:11.8677795Z  Server Version: 27.1.1
2024-09-19T15:03:11.8678797Z  Storage Driver: overlay2
2024-09-19T15:03:11.8679689Z   Backing Filesystem: extfs
2024-09-19T15:03:11.8680709Z   Supports d_type: true
2024-09-19T15:03:11.8681674Z   Using metacopy: false
2024-09-19T15:03:11.8682617Z   Native Overlay Diff: true
2024-09-19T15:03:11.8683673Z   userxattr: true
2024-09-19T15:03:11.8685394Z  Logging Driver: json-file
2024-09-19T15:03:11.8686421Z  Cgroup Driver: none
2024-09-19T15:03:11.8687323Z  Cgroup Version: 2
2024-09-19T15:03:11.8688171Z  Plugins:
2024-09-19T15:03:11.8689031Z   Volume: local
2024-09-19T15:03:11.8690433Z   Network: bridge host ipvlan macvlan null overlay
2024-09-19T15:03:11.8692301Z   Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
2024-09-19T15:03:11.8693661Z  Swarm: inactive
2024-09-19T15:03:11.8694627Z  Runtimes: io.containerd.runc.v2 runc
2024-09-19T15:03:11.8695346Z  Default Runtime: runc
2024-09-19T15:03:11.8695969Z  Init Binary: docker-init
2024-09-19T15:03:11.8696680Z  containerd version: 8fc6bcff51318944179630522a095cc9dbf9f353
2024-09-19T15:03:11.8697633Z  runc version: v1.1.13-0-g58aa920
2024-09-19T15:03:11.8698606Z  init version: de40ad0
2024-09-19T15:03:11.8699340Z  Security Options:
2024-09-19T15:03:11.8699868Z   seccomp
2024-09-19T15:03:11.8700388Z    Profile: builtin
2024-09-19T15:03:11.8700949Z   rootless
2024-09-19T15:03:11.8701437Z   cgroupns
2024-09-19T15:03:11.8702015Z  Kernel Version: 5.15.0-1068-azure
2024-09-19T15:03:11.8702701Z  Operating System: Alpine Linux v3.20 (containerized)
2024-09-19T15:03:11.8703377Z  OSType: linux
2024-09-19T15:03:11.8703921Z  Architecture: x86_64
2024-09-19T15:03:11.8704459Z  CPUs: 32
2024-09-19T15:03:11.8704989Z  Total Memory: 251.7GiB
2024-09-19T15:03:11.8705606Z  Name: compute-runners-ltp9l-runner-97gwl
2024-09-19T15:03:11.8706458Z  ID: c5653b70-eddc-4ba2-b0b0-6dc24d4f23db
2024-09-19T15:03:11.8707195Z  Docker Root Dir: /home/runner/.local/share/docker
2024-09-19T15:03:11.8707854Z  Debug Mode: false
2024-09-19T15:03:11.8708405Z  Experimental: false
2024-09-19T15:03:11.8708979Z  Insecure Registries:
2024-09-19T15:03:11.8709487Z   127.0.0.0/8
2024-09-19T15:03:11.8710034Z  Live Restore Enabled: false
2024-09-19T15:03:11.8710615Z  Product License: Community Engine
2024-09-19T15:03:11.8710988Z 
2024-09-19T15:03:11.8711672Z WARNING: Running in rootless-mode without cgroups. Systemd is required to enable cgroups in rootless-mode.
2024-09-19T15:03:11.8713342Z ##[endgroup]

Prepare

2024-09-19T15:03:11.8714267Z ##[group]Pulling binfmt Docker image
2024-09-19T15:03:11.8715363Z [command]/usr/bin/docker pull docker.io/tonistiigi/binfmt:latest
2024-09-19T15:03:12.4442982Z latest: Pulling from tonistiigi/binfmt
2024-09-19T15:03:12.5919641Z 8d4d64c318a5: Pulling fs layer
2024-09-19T15:03:12.9438646Z e9c608ddc3cb: Pulling fs layer
2024-09-19T15:03:12.9439517Z e9c608ddc3cb: Verifying Checksum
2024-09-19T15:03:12.9440311Z e9c608ddc3cb: Download complete
2024-09-19T15:03:13.6138947Z 8d4d64c318a5: Verifying Checksum
2024-09-19T15:03:13.6139828Z 8d4d64c318a5: Download complete
2024-09-19T15:03:13.8251820Z 8d4d64c318a5: Pull complete
2024-09-19T15:03:13.8541699Z e9c608ddc3cb: Pull complete
2024-09-19T15:03:13.8576312Z Digest: sha256:66e11bea77a5ea9d6f0fe79b57cd2b189b5d15b93a2bdb925be22949232e4e55
2024-09-19T15:03:13.8585832Z Status: Downloaded newer image for tonistiigi/binfmt:latest
2024-09-19T15:03:13.8602686Z docker.io/tonistiigi/binfmt:latest
2024-09-19T15:03:13.8626048Z ##[endgroup]
2024-09-19T15:03:13.8627121Z ##[group]Image info
2024-09-19T15:03:13.8637670Z [command]/usr/bin/docker image inspect docker.io/tonistiigi/binfmt:latest
2024-09-19T15:03:13.8768409Z [
2024-09-19T15:03:13.8769910Z     {
2024-09-19T15:03:13.8770827Z         "Id": "sha256:354472a378935adfe74a19600b89bd9ada7bb058306fff23b3d6613405852faf",
2024-09-19T15:03:13.8771966Z         "RepoTags": [
2024-09-19T15:03:13.8772674Z             "tonistiigi/binfmt:latest"
2024-09-19T15:03:13.8773330Z         ],
2024-09-19T15:03:13.8773898Z         "RepoDigests": [
2024-09-19T15:03:13.8774742Z             "tonistiigi/binfmt@sha256:66e11bea77a5ea9d6f0fe79b57cd2b189b5d15b93a2bdb925be22949232e4e55"
2024-09-19T15:03:13.8775636Z         ],
2024-09-19T15:03:13.8776160Z         "Parent": "",
2024-09-19T15:03:13.8776734Z         "Comment": "buildkit.dockerfile.v0",
2024-09-19T15:03:13.8777645Z         "Created": "2022-08-02T19:13:20.178433831Z",
2024-09-19T15:03:13.8778296Z         "DockerVersion": "",
2024-09-19T15:03:13.8778918Z         "Author": "",
2024-09-19T15:03:13.8779441Z         "Config": {
2024-09-19T15:03:13.8779950Z             "Hostname": "",
2024-09-19T15:03:13.8780762Z             "Domainname": "",
2024-09-19T15:03:13.8781354Z             "User": "",
2024-09-19T15:03:13.8781891Z             "AttachStdin": false,
2024-09-19T15:03:13.8782642Z             "AttachStdout": false,
2024-09-19T15:03:13.8783227Z             "AttachStderr": false,
2024-09-19T15:03:13.8783839Z             "Tty": false,
2024-09-19T15:03:13.8784404Z             "OpenStdin": false,
2024-09-19T15:03:13.8784950Z             "StdinOnce": false,
2024-09-19T15:03:13.8785518Z             "Env": [
2024-09-19T15:03:13.8786229Z                 "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
2024-09-19T15:03:13.8787046Z                 "QEMU_PRESERVE_ARGV0=1"
2024-09-19T15:03:13.8787664Z             ],
2024-09-19T15:03:13.8788160Z             "Cmd": null,
2024-09-19T15:03:13.8788698Z             "Image": "",
2024-09-19T15:03:13.8789242Z             "Volumes": {
2024-09-19T15:03:13.8789773Z                 "/tmp": {}
2024-09-19T15:03:13.8790346Z             },
2024-09-19T15:03:13.8790851Z             "WorkingDir": "/",
2024-09-19T15:03:13.8791429Z             "Entrypoint": [
2024-09-19T15:03:13.8791983Z                 "/usr/bin/binfmt"
2024-09-19T15:03:13.8792534Z             ],
2024-09-19T15:03:13.8793002Z             "OnBuild": null,
2024-09-19T15:03:13.8793588Z             "Labels": {
2024-09-19T15:03:13.8794391Z                 "org.opencontainers.image.created": "2022-08-02T18:32:39.936Z",
2024-09-19T15:03:13.8795634Z                 "org.opencontainers.image.description": "Cross-platform emulator collection distributed with Docker images",
2024-09-19T15:03:13.8796698Z                 "org.opencontainers.image.licenses": "MIT",
2024-09-19T15:03:13.8797665Z                 "org.opencontainers.image.revision": "a161c41c7aeaf3ef1c5b97f91aa02a12cca73432",
2024-09-19T15:03:13.8799016Z                 "org.opencontainers.image.source": "https://github.com/tonistiigi/binfmt",
2024-09-19T15:03:13.8799938Z                 "org.opencontainers.image.title": "Binfmt",
2024-09-19T15:03:13.8801077Z                 "org.opencontainers.image.url": "https://github.com/tonistiigi/binfmt",
2024-09-19T15:03:13.8802079Z                 "org.opencontainers.image.version": "qemu-v7.0.0-28"
2024-09-19T15:03:13.8802812Z             }
2024-09-19T15:03:13.8803332Z         },
2024-09-19T15:03:13.8803803Z         "Architecture": "amd64",
2024-09-19T15:03:13.8804397Z         "Os": "linux",
2024-09-19T15:03:13.8805602Z         "Size": 60182964,
2024-09-19T15:03:13.8806136Z         "GraphDriver": {
2024-09-19T15:03:13.8806695Z             "Data": {
2024-09-19T15:03:13.8807733Z                 "LowerDir": "/home/runner/.local/share/docker/overlay2/89b4dee1346a6ca1033dcbbb12e1f47fbeb743168bd565124d8f6dc35ea71ec8/diff",
2024-09-19T15:03:13.8809331Z                 "MergedDir": "/home/runner/.local/share/docker/overlay2/c3926b1dd20bef4599648ca99d17516a2440f3f488a934b74af3214963a55e36/merged",
2024-09-19T15:03:13.8810919Z                 "UpperDir": "/home/runner/.local/share/docker/overlay2/c3926b1dd20bef4599648ca99d17516a2440f3f488a934b74af3214963a55e36/diff",
2024-09-19T15:03:13.8812962Z                 "WorkDir": "/home/runner/.local/share/docker/overlay2/c3926b1dd20bef4599648ca99d17516a2440f3f488a934b74af3214963a55e36/work"
2024-09-19T15:03:13.8814221Z             },
2024-09-19T15:03:13.8814751Z             "Name": "overlay2"
2024-09-19T15:03:13.8815297Z         },
2024-09-19T15:03:13.8815803Z         "RootFS": {
2024-09-19T15:03:13.8816305Z             "Type": "layers",
2024-09-19T15:03:13.8816929Z             "Layers": [
2024-09-19T15:03:13.8817701Z                 "sha256:4c67e4044f8c0fe3e3efaf76f2a3d5d3d866f8ef2e8a9da756949d90e576baa0",
2024-09-19T15:03:13.8818687Z                 "sha256:949acf1cb73a60306e050836deb85a26fe23e226f6bcc499872b057efbf22dd1"
2024-09-19T15:03:13.8819502Z             ]
2024-09-19T15:03:13.8819934Z         },
2024-09-19T15:03:13.8820448Z         "Metadata": {
2024-09-19T15:03:13.8821121Z             "LastTagTime": "0001-01-01T00:00:00Z"
2024-09-19T15:03:13.8821773Z         }
2024-09-19T15:03:13.8822393Z     }
2024-09-19T15:03:13.8823180Z ]
2024-09-19T15:03:13.8824388Z ##[endgroup]

Error

2024-09-19T15:03:13.8825239Z ##[group]Installing QEMU static binaries
2024-09-19T15:03:13.8826792Z [command]/usr/bin/docker run --rm --privileged docker.io/tonistiigi/binfmt:latest --install linux/amd64,linux/arm64
2024-09-19T15:03:14.0907721Z docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "sysfs" to rootfs at "/sys": mount sysfs:/sys (via /proc/self/fd/6), flags: 0xe: operation not permitted: unknown.
2024-09-19T15:03:14.0943497Z ##[endgroup]
2024-09-19T15:03:14.0987533Z ##[error]docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "sysfs" to rootfs at "/sys": mount sysfs:/sys (via /proc/self/fd/6), flags: 0xe: operation not permitted: unknown.
2024-09-19T15:03:14.1202103Z Post job cleanup.
2024-09-19T15:03:14.4331024Z [command]/usr/bin/docker logout cssacrstg.azurecr.io
2024-09-19T15:03:14.4473910Z Removing login credentials for cssacrstg.azurecr.io
2024-09-19T15:03:14.4525797Z ##[group]Post cache
2024-09-19T15:03:14.4526737Z State not set
2024-09-19T15:03:14.4528747Z ##[endgroup]
2024-09-19T15:03:19.4858605Z Post job cleanup.
...

I suspect the issue is because QEMU need some special setup in root-less mode (or it does not support it), I found related thread in runc - opencontainers/runc#3672 however I wonder :

1. it its possible to run qemu/docker buildx with DinD root-less mode
2. Can it be supported/work-arounded somehow ?

[k4kratik]

+1
I am also using dind:rootless and not able to use QEMU.