Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No module ip_vs in Docker4Mac's kernel #4754

Closed
2 tasks done
dduportal opened this issue Jul 8, 2020 · 8 comments
Closed
2 tasks done

No module ip_vs in Docker4Mac's kernel #4754

dduportal opened this issue Jul 8, 2020 · 8 comments

Comments

@dduportal
Copy link

dduportal commented Jul 8, 2020

  • I have tried with the latest version of my channel (Stable or Edge)
  • I have uploaded Diagnostics
  • Diagnostics ID: 17EC4488-B01D-4BA8-ABCC-DD188197C8D5/20200708141647

Expected behavior

When loading the kernel module ip_vs from a (privileged) container with access to the VM's kernel modules,
Then I expect the module to be loaded at least until the VM is stopped or restarted

$ docker run --rm -ti -v /lib/modules:/lib/modules --privileged alpine:3.12 sh -c "modprobe ip_vs && lsmod"
# ...
ip_vs                 151552  0 
# ...

Actual behavior

The kernel module absent from the VM's modules kernel config, and cannot be loaded.
An error module ip_vs not found in modules.dep is thrown when loading the module:

$ docker run --rm -ti -v /lib/modules:/lib/modules --privileged alpine:3.12 sh -c "modprobe ip_vs && lsmod | grep ip_vs"
modprobe: module ip_vs not found in modules.dep

Information

I want to execute containers running keepalived services with a virtual IP in a docker private network.
It utilizes the "IP Virtual Server" (ip_vs) for loadbalancing the virtual IP through vrrp protocol.

This is something which used to work in earlier versions of Docker4Mac and docker-machine,
but not sure when this issue started to happen (not more than 1 year ago).

It works on most of the Docker on Linux (as soon as the host kernel has the module of course :) ).

Some uses case:

  • Demonstrate HA architectures (1 private network with 1 vip, 2 webserver containers in this network, 2 keepalived services for instance)
  • Reproduce HA environments

My configuration:

  • macOS Version: 10.15.5
  • Docker4Mac 2.3.2.0 (latest Edge)

This issue is related to other "missing" kernel modules: #4560, #4549, #4660, #4556.

This (old) comment from Justin - #719 (comment) might give some clues but I was not able to try the "compile module" solution and switched to multipass for this use case.

Sounds like a non trivial issue as there is a balance between not embedding too much modules to ensure fast VM, and usages.
=> In my case, it would totally be acceptable to NOT have the module loaded or even present by default, but if I have an option to retrieve & build the module on demand, and then I can load it, even on each reboot.
(A docker file with the module as "arg" ? or an advanced panel in the GUI where to list the module I want, a bit like the daemon.json so linuxkit would be tuned ? Or even a procedure to "rebuild" my own docker4mac).

Diagnostic logs

Not sure what to put here as I don't have the menu "Diagnose and feedback" & a Diagnostic bundle had been uploaded?

Steps to reproduce the behavior

  1. Start Docker4Mac
  2. Run the command docker run --rm -ti -v /lib/modules:/lib/modules --privileged alpine:3.12 sh -c "modprobe ip_vs && lsmod | grep ip_vs" as described in the section Actual Behavior below.
@docker-robott
Copy link
Collaborator

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

@dduportal
Copy link
Author

Hello,

A tiny bump to update the issue: it is still the same with the latest edge version (2.4.2.0):

➜  ~ docker run --rm -ti -v /lib/modules:/lib/modules --privileged alpine:3.12 sh -c "modprobe ip_vs && lsmod | grep ip_vs"
modprobe: module ip_vs not found in modules.dep
➜  ~ docker version
Client: Docker Engine - Community
 Cloud integration: 0.1.22
 Version:           20.10.0-beta1
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        ac365d7
 Built:             Tue Oct 13 18:13:53 2020
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.0-beta1
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       9c15e82
  Built:            Tue Oct 13 18:17:18 2020
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          v1.4.1
  GitCommit:        c623d1b36f09f8ef6536a057bd658b3aa8632828
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

@stephen-turner
Copy link
Contributor

@dduportal As of 2.4.2.0 we are up to date with the latest version of LinuxKit, so any modules that are present upstream should be present in Docker Desktop.

@stephen-turner
Copy link
Contributor

stephen-turner commented Nov 19, 2020

If the module is still missing, you should probably raise it in https://github.com/linuxkit/linuxkit.

@dduportal
Copy link
Author

Hello @stephen-turner thanks a lot for the answer. I'm going to check with the latest version, and with LinuxKit part if the issue is still there.

If you don't mind, I'll report here the results and close once it will be ok (either with the latest Docker Desktop, or when LinuxKit and the downstream Docker Desktop will be updated). Sounds good ?

Thanks for all this awesome work!

@dduportal
Copy link
Author

dduportal commented Nov 26, 2020

Closing the issue, as I can confirme that the module ip_vs can be loaded on the latest Docker Desktop Edge (version 2.5.0.1).

Reproduction using Ubuntu (my initial command with Alpine is not working due to Alpine's):

# Inspiration from https://dev.to/douglasmakey/how-to-setup-simple-load-balancing-with-ipvs-demo-with-docker-4j1d
# Start 2 containers to be loadbalanced
$ docker run -d --name first -t jwilder/whoami # Retrieve IP with docker inspect: 172.17.0.3
$ docker run -d --name second -t jwilder/whoami # Retrieve IP with docker inspect: 172.17.0.4
$ docker run --rm -ti -v /lib/modules:/lib/modules --privileged ubuntu:20.04 bash
> apt-get update
> apt-get install -y kmod curl ipvsadm
> modprobe ip_vs
> ipvsadm -A -t 100.100.100.100:80 -s rr
> curl 172.17.0.3:8000 # Should answer "I'm <container id>"
> ipvsadm -a -t 100.100.100.100:80 -r 172.17.0.3:8000 -m
> curl 172.17.0.4:8000 # Should answer "I'm <container id>"
> ipvsadm -a -t 100.100.100.100:80 -r 172.17.0.4:8000 -m
> ipvsadm -l # Check load balancing config
> curl 100.100.100.100; curl 100.100.100.100; curl 100.100.100.100 # You should see "I'm <container id>" with the id changing

@stephen-turner
Copy link
Contributor

Great, thanks for checking.

@docker-robott
Copy link
Collaborator

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked

@docker docker locked and limited conversation to collaborators Dec 26, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants