Qemu instruction translation for ARM

[computermouth]

• This is a feature request

Dockerfile

FROM arm32v7/debian:stretch-slim
RUN apt-get -yqq update

Expected behavior

user@box:~/tmp$ docker build .
Sending build context to Docker daemon  12.29kB
Step 1/2 : FROM arm32v7/debian:stretch-slim
 ---> a50ff4bddff0
Step 2/2 : RUN ls
 ---> Running in 7cf0295440e8
bin
[...]
usr
var
 ---> 7dbfaa72cc6f
Removing intermediate container 7cf0295440e8
Successfully built 7dbfaa72cc6f

Actual behavior

user@box:~/tmp$ docker build .
Sending build context to Docker daemon  12.29kB
Step 1/2 : FROM arm32v7/debian:stretch-slim
 ---> a50ff4bddff0
Step 2/2 : RUN ls
 ---> Running in 61398d123a67
standard_init_linux.go:187: exec user process caused "no such file or directory"
The command '/bin/sh -c ls' returned a non-zero code: 1

Notes

This Dockerfile will build on DfW CE and DfM CE, I believe that both are running a Linux VM in a hypervisor, and that each is shipped with qemu translation. Regardless, it seems odd to have a non-uniform experience across the platforms. I'm working on a solution for embedded Linux developers to be able to build products with their software running in containers, as such, size is of the essence, and expecting users to add/remove qemu-arm-static to/from containers before production would be an awkward practice to push. It'd be lovely if arm32v7 on Linux "Just Worked"™ the way it does on the other platforms.

Output of docker version:

Client:
 Version:      17.06.0-ce
 API version:  1.30
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:17:22 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.06.0-ce
 API version:  1.30 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:16:12 2017
 OS/Arch:      linux/amd64
 Experimental: false

Output of docker info:

Containers: 5
 Running: 0
 Paused: 0
 Stopped: 5
Images: 34
Server Version: 17.06.0-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: cfb82a876ecc11b5ca0977d1733adbe58599088a
runc version: 2d41c047c83e09a6d61d464906feb2a2f3c52aa4
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.0-3-amd64
Operating System: Debian GNU/Linux 9 (stretch)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.746GiB
Name: box
ID: CVDF:ACOE:ABI4:PMSG:5D6H:ZCPL:JXAX:Z5CA:SZRP:IV4M:M6KZ:KHC3
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

WARNING: No swap limit support

[computermouth]

Solution found, filed a bug report with Debian [also applies to Ubuntu]. Details here:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=868217

Closing issue

[ruffsl]

Any progress on that debian bug report? Looks like it hasn't been active since 2017.

I'm now using Ubuntu 18.04, I also tried moving the config file to get things working:

sudo mkdir -p /lib/binfmt.d
sudo cp /var/lib/binfmts/* /lib/binfmt.d/
sudo systemctl restart systemd-binfmt.service

$ systemctl status systemd-binfmt
● systemd-binfmt.service - Set Up Additional Binary Formats
   Loaded: loaded (/lib/systemd/system/systemd-binfmt.service; static; vendor preset: enabled)
   Active: active (exited) since Fri 2018-08-17 11:03:50 PDT; 5s ago
     Docs: man:systemd-binfmt.service(8)
           man:binfmt.d(5)
           https://www.kernel.org/doc/Documentation/binfmt_misc.txt
  Process: 11217 ExecStart=/lib/systemd/systemd-binfmt (code=exited, status=0/SUCCESS)
 Main PID: 11217 (code=exited, status=0/SUCCESS)

Aug 17 11:03:50 cog-rw07 systemd[1]: Starting Set Up Additional Binary Formats...
Aug 17 11:03:50 cog-rw07 systemd[1]: Started Set Up Additional Binary Formats.

But still fail when attempting to run arm64v8 containers with docker, as it looks like they are still missing the "F - fix binary" flag for binfmt entries.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=868030

$ docker run -it arm64v8/ubuntu:18.04 uname -a
standard_init_linux.go:190: exec user process caused "exec format error"

If I switch to using the configs provided by fedora however, like here it now works.
https://github.com/computermouth/qemu-static-conf/blame/75cc31c05af38177dbba94d09568322fdddf14fd/README.md#L36-L39

[thavlik]

@ruffsl that worked for me on Debian stretch.

For reference, the commands were:

git clone https://github.com/computermouth/qemu-static-conf.git
sudo mkdir -p /lib/binfmt.d
sudo cp qemu-static-conf/*.conf /lib/binfmt.d/
sudo systemctl restart systemd-binfmt.service