Package request: Docker engine

[danielmeneses]

Great job for all that Termux is. I know that this is probably a "no can do", but would be a for a nobel award!

;)

[samos123]

+1 this would also make it easier to support building termux packages on Android via docker. I guess the x86 architecture makes most sense, although ARM is also supported: https://github.com/umiddelb/armhf/wiki/Installing,-running,-using-docker-on-armhf-(ARMv7)-devices

[casept]

I have tried building docker engine on android. The major issues seem to be the craptastic, outdated kernels shipped with most devices, missing support for storage drivers in these kernels and the need for root to use docker. I can't see a sane way to make this happen.

[Serkan-devel]

Maybe it would work on lineage os

[casept]

It might on some extremely new devices. Most still package 3.x series kernels, which are too old for docker and can't be updated by ROM maintainers because most are raging tire fires of hackish vendor patches stacked one onto another.

[phonicmouse]

+1

[T-vK]

+1

[gut5]

My device (Nexus 6) has a 3.10 kernel, I think it would be enough for Docker.

[T-vK]

Requirements seem to be:

• A 64-bit installation
• Version 3.10 or higher of the Linux kernel. The latest version of the kernel available for you platform is recommended.
• iptables version 1.4 or higher
• git version 1.7 or higher
• A ps executable, usually provided by procps or a similar package.
• XZ Utils 4.9 or higher
• a properly mounted cgroupfs hierarchy; a single, all-encompassing cgroup mount point is not sufficient.

[gut5]

Damn, 64-bit only. That will rule it out for a lot of people.

[TheDiamondYT1]

I'm sure docker could be patched to support 32 bit, I could be wrong

[Serkan-devel]

I have docker running on my 32-bit raspberry pi 2

[T-vK]

@casept I just noticed my 2014 tablet has a 3.10 kernel. So I think the number of devices that would support Docker might have risen quite a bit. Also, I don't think root is required. On the desktop you can simply add a non-root user to the docker group and then that user can run docker without root privileges.

[danielmeneses]

@T-vK that's nice and I don't quite know the docker implementation but if it makes use of the chroot command or similar I think we will need root access anyway. But it's somehow possible it would be great.

[gut5]

https://docs.docker.com/engine/installation/linux/docker-ce/debian/
https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/

Debian and Ubuntu both have armhf Docker packages. And I don't know if there's anything to be gained from looking at the Moby project but it looks like this is where Docker system integration development is taking place now.

https://github.com/moby/moby
https://mobyproject.org/

Hoping to get some time to look into this soon.

[smiller171]

@T-vK the docker client can run without root, but the docker engine requires root access afaik.

Also, it doesn't just need a 3.10 kernel, it needs kernel modules that aren't built into Android kernels. You would need to compile a custom kernel

[pwdonald]

I see that CrossOver runs Windows apps using emulation, is something similar possible with Linux on Android so that a more appropriate Kernel version could be utilized to run Docker?

[smiller171]

@pwdonald Nope. You either need to compile a custom Android kernel, or somehow run a VM on Android. Would be better just to run a remote server and SSH in.

[sneak]

The engine and all its associated architecture requirements aside, I would love to see just the docker client packaged. One can use it with socat to redirect a local socket via an SSH tunnel to a remote host that is running the docker engine, or with the docker TCP/TLS rpc system (e.g. as set up by docker-machine).

[onlurking]

I don't think this is easy since Docker uses a lot of the linux kernel features, and Android ships a stripped down version of the kernel.

[smiller171]

The Docker client can be done. There are already Android apps that can talk to a remote Docker Engine if you expose the API

[T-vK]

This doesn't change anything. The PRoot would still use the kernel of your device. The only way around it would be virtualization which would probably make everything painfully slow and I'm not even sure if this can be done on any phone.

[smiller171]

Yeah, the only way to do this is to build a custom kernel for your phone unless Google decides to build container support into Android.

[T-vK]

And I doubt the latter will happen. There has been something in the news recently about Chrome OS potentially adding some kind of container support, though..

[Serkan-devel]

Or it will be implemented on Fuchisa

[smiller171]

@T-vK It actually might happen. Google is heavily invested in containers so I could see a future version of Android using container sandboxing as an alternative to the current JVM sandboxing. They wouldn't expose it as Docker natively, but it could open a path to compiling docker-engine for stuff like Termux.

[smiller171]

Also, you're thinking of this: https://chromeunboxed.com/news/chrome-os-chromebook-containers-developer which seems to be designed for improving Chromebooks for development. I'm hoping for an announcement at I/O

[emahuni]

Ok here is a guy who ran a docker image on termux. It was a
https://www.reddit.com/r/docker/comments/7r7t6b/is_docker_possible_on_mobile/
using this: https://github.com/nmilosev/anyfed

So can't we take lessons from there and create an actual docker image running package?

[rodrigoaguilera]

I think the scope of this issue is to be able to have a docker engine running and be able to use the docker CLI client.

Using docker images as a base for chroot is nice but I feel it does not get any closer to the goal

[T-vK]

I agree with rodrigoaguilera.

[ghost]

Were you able to run any Linux container?

Ubuntu worked for me.

Or maybe, just bootstrap a Linux, chroot inside it and try to run Docker from there. @xeffyr , have you ever tried any of that?

Yes, Docker works under chroot. You can go with something like http://dl-cdn.alpinelinux.org/alpine/v3.12/releases/aarch64/alpine-minirootfs-3.12.0-aarch64.tar.gz as base for Docker installation.

[FreddieOliveira]

Yes, Docker works under chroot. You can go with something like http://dl-cdn.alpinelinux.org/alpine/v3.12/releases/aarch64/alpine-minirootfs-3.12.0-aarch64.tar.gz as base for Docker installation.

Indeed, I was able to run docker inside a chrooted rootfs of alpine and ubuntu. Things were a bit unstable and my phone often reboots after running the containers (not sure if it's my kernel, or whatever), but it works.

Another option for who's interested, is to emulate a GNU/Linux distro using qemu and run docker inside the VM. This has the advantage of not needing a rooted device nor a custom kernel. But as it's been running under an emulated environment, it will be slower than running the chrooted alternative.

Those are the only two ways of running docker containers under Android, as far as I'm concerned.

[FreddieOliveira]

@xeffyr I noted that the kernel, by default, indeed creates the cgroups hierarchy with the cpuset prefix, but the Android, while booting, renames the files by striping the prefix, thus changing cpuset.cpus and cpuset.mems to just cpus and mems (among others files).

In Android versions bellow 10, this happens when the init process is executing the /init.rc script. If desired, one can edit this script to prevent the renaming and even to automate the cgroup mounting under the default /sys/fs/cgroup dir.

In Android 10, things were changed and the init process creates the cgroups by itself, using the cgroups.json just to check the mount location and the /init.rc to do some additional setups. So, to prevent the renaming, one should patch the Android source code and compile it, or simply delete the cgroups.json file, so the Android won't even try to setup cgroups. After the phone boots it's still necessary to mount cgroups manually.

[ghost]

my phone often reboots after running the containers

I had same and that was caused by null pointer dereference in xt_qtaguid subsystem which is Android specific. That bug exists for a long time and I'm not sure if it was fixed in Android 10.

I patched it manually, that broke its functionality but Docker ran perfectly after that.

Kernel log:

Unable to handle kernel NULL pointer dereberence at virtual address 00000078
  pgd = ffffffc850c3f000
  [00000078] *pgd=00000008cf156003, *pud=00000008cf156003, *pmd=0000000000000000
  ------------[ cut here ]---,--------
  Kernel BUG at ffffffc000ae6f38 [verbose debug info unavailable]
  Internal error: Oops - BUG: 96000006 [#1] PREEMPT SMP
  exynos-snapshot: exynos_ss_get_reason 0x0 (CPU:2)
  exynos-snapshot: core register saved(CPU:2)
  CPUMERRSR: 00000000081c06c5, L2MERRSR: 0000000010302808
  SCTLR_EL1: 04C5591D,
  exynos-snapshot: context saved(CPU:2)
  exynos-snapshot: item - log_kevents is disabled
  CPU: 2 PID: 3873 Comm: WifiService Tainted: G        W      3.18.140-by_heffyr-g7aa873f92eb #1
  Hardware name: Samsung UNIVERSAL8890 board based on EXYNOS8890 (DT)
  task: ffffffc848e70cc0 ti: ffffffc8487bc000 task.ti: ffffffc8487bc000
  PC is at dev_get_stats+0x1c/0xa0
  LR is at ifabe[stat_fmt_proc_show+0x78/0x178

[ghost]

And here is an applied patch:

--- orig/net/netfilter/xt_qtaguid.c     2020-05-12 12:13:14.000000000 +0300
+++ my/net/netfilter/xt_qtaguid.c       2019-09-15 23:56:45.000000000 +0300
@@ -737,7 +737,7 @@
 {
        struct proc_iface_stat_fmt_info *p = m->private;
        struct iface_stat *iface_entry;
-       struct rtnl_link_stats64 dev_stats, *stats;
+       struct rtnl_link_stats64 *stats;
        struct rtnl_link_stats64 no_dev_stats = {0};


@@ -745,13 +745,8 @@
                 current->pid, current->tgid, from_kuid(&init_user_ns, current_fsuid()));

        iface_entry = list_entry(v, struct iface_stat, list);
+       stats = &no_dev_stats;

-       if (iface_entry->active) {
-               stats = dev_get_stats(iface_entry->net_dev,
-                                     &dev_stats);
-       } else {
-               stats = &no_dev_stats;
-       }
        /*
         * If the meaning of the data changes, then update the fmtX
         * string.

Breaks xt_qtaguid functionality, however no impact on Android OS net traffic accounting was observed.

---

Another patch for my kernel was fixing prefixed cgroup files:

--- orig/kernel/cgroup.c        2020-05-12 12:13:14.000000000 +0300
+++ my/kernel/cgroup.c  2019-09-15 23:56:45.000000000 +0300
@@ -3025,6 +3025,12 @@

        if (cft->seq_show == cgroup_populated_show)
                cgrp->populated_kn = kn;
+
+       if (cft->ss && (cgrp->root->flags & CGRP_ROOT_NOPREFIX) && !(cft->flags & CFTYPE_NO_PREFIX)) {
+               snprintf(name, CGROUP_FILE_NAME_MAX, "%s.%s", cft->ss->name, cft->name);
+               kernfs_create_link(cgrp->kn, name, kn);
+       }
+
        return 0;
 }

[FreddieOliveira]

That bug exists for a long time and I'm not sure if it was fixed in Android 10

I can confirm this bug is still present in Android 10. The source code is exactly the same and applying your patch solves the problem! Thanks for sharing the patch, btw 👍 I would've spent days figuring it out by myself.

[FreddieOliveira]

Finally, it worked!

This is docker successfully running containers directly in Android. No chroot inside GNU/Linux rootfs and no VM emulation with qemu. This is pure Android.

As showed here, docker runs fine in Android, it just can't pull any container since the Android OS isn't supported by them. Trying to politely ask a Linux version of the container with the flag --platform=linux didn't work either. So I lost my patience debugging the seg fault I was getting and had the idea to spoof my operating system by replacing every occurrence of runtime.GOOS (which returns the string android) by "linux". And guess what, it worked.

I opened a feature request in docker repo yesterday: moby/moby#41111, but I'm not sure if they'll take it seriously.

[mmBesar]

Docker on android would be great

[FreddieOliveira]

For whoever is interested in get docker running on its phone, I've written a step by step gist

After 7 months using it I can say it's working pretty stable on my phone. The only thing I couldn't get working 100% is the network isolation for the containers. For now, to give the containers internet access you have to expose its network by using the --net=host flag.

@xeffyr @Grimler91 A PR for docker in termux-unstable would be accepted?

[ghost]

@FreddieOliveira Yes, PR would be nice.

As for networking, I guess here is something to do with Android firewall or routing tables, if relevant drivers are already enabled.

[FreddieOliveira]

if relevant drivers are already enabled

I believe they are.

As for networking, I guess here is something to do with Android firewall or routing tables

That's what I think. Since I use AdAway and AFWall+ apps, my routing table is pretty messed up. Probably it's something silly I haven't figured out yet. When the PRs are done, I'll open an issue specific for this internet problem, so this issue here can be closed and the relevant discussion continues there.

[FreddieOliveira]

I'll open an issue specific for this internet problem

This isn't needed anymore. The problem was already solved and was indeed related with the routing table.

[FreddieOliveira]

It took almost 5 years since this issue was opened, but Docker is finally part of the root repo now. Run pkg install docker to install it.

Note that you still need a rooted phone with a compatible kernel to run it.

[softyn]

Could you describe how to install and what are final requirements? I downloaded termux on my (non rooted) android device, run 'pkg install docker' and got 'unable to locate package docker' :(

[FreddieOliveira]

Unfortunately you won't be able to run docker (or any program from termux-root repository) on non-rooted devices. You need to root your phone, compile your kernel with the necessary features, subscribe to termux-root repo with pkg install root-repo and install docker with pkg install docker. Only then you'll be able to run it.

[thebestnom]

I get mount: bad /etc/fstab: No such file or directory when running dockerd, what do you meant by relevant kernel features?

Edit: my bad.. didn't run as sudo 😓

[xy815661276]

We open the Android container, which can run the docker container on the Android operating system. Not only can run Docker containers, but also other containers, such as podman. In addition, we have also enabled the container to support checkpoint and restore functions, and support the migration of containers across architectures and operating systems. See https://github.com/CGCL-codes/Android-Container for details.

[Grimler91]

Closing since docker is available for several months already