How to setup MTU for tunnel - guide in docs does not work?

[ieugen]

How do I set the MTU? I followed the guide on [1] and I still get tunnel MTU at 1440 (larger than my wireguard MTU 1420) .

Expected Behavior

The tunnel MTU to be the one I set via ipipMTU - 1400 .

Current Behavior

When I display MTU for all interfaces on all nodes I get 1440 for tunnel.

salt-ssh  '*' -r 'ip addr' | grep mtu
        1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
        2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        3: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1
        4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
        5: cali217c693643a@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default 
        6: cali893367b5351@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default 
        7: tunl0@NONE: <NOARP,UP,LOWER_UP> mtu 1440 qdisc noqueue state UNKNOWN group default qlen 1
        8: cali12d4a061371@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default 
        1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
        2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        3: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1
        4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
        1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
        2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        3: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1
        4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
        5: tunl0@NONE: <NOARP,UP,LOWER_UP> mtu 1440 qdisc noqueue state UNKNOWN group default qlen 1
        6: cali3aa079d1c1a@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default 
        7: cali91193ebe10d@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default 
        8: calid14c9d9c505@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default 
        9: calidcc9b422d75@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default 
        10: cali92ed41d3a22@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default 
        1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
        2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        3: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1
        4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
        5: cali66c0e5aef8d@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default 
        6: cali71d64841ae9@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default 
        7: tunl0@NONE: <NOARP,UP,LOWER_UP> mtu 1440 qdisc noqueue state UNKNOWN group default qlen 1
        8: cali9cafa0a893e@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default 
        9: caliad17b2e6582@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default 
        1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
        2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        3: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1
        4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
        5: calicbf4a3084d5@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default 
        6: calicaf8383f0e4@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UP group default 
        7: tunl0@NONE: <NOARP,UP,LOWER_UP> mtu 1440 qdisc noqueue state UNKNOWN group default qlen 1

Possible Solution

??

Steps to Reproduce (for bugs)

Steps I took:

• updated calico config mtu to 1400 with kubectl -n kube-system edit configmap calico-config
• added felix ipipMTU with 1400 via calicoctl replace -f felix.yaml
• restared all cluster nodes at once with salt-ssh

I'm trying to get tunnel to match wg0 MTU size of 1420 - my example shows with 1400, it's not a typo, this is what I tried in my second approach.

[1] https://docs.projectcalico.org/v3.1/usage/configuration/mtu

Context

I'm trying to fix issue described here #1709 (comment) . In summary: responses larger than 1868 bytes time out in my ingress-nginx running with host networking.

I confirm it's caused by MTU - I have tcpdump captures and I've reproduced the issue + fixed it by lowering the MTU to 1400.

I wish to align the MTU's according to the guide, just to be ok.

Your Environment

• Calico version
  calicoctl version
  Client Version: v3.1.3
  Build date: 2018-05-30T17:15:59+0000
  Git commit: 231083c
  Cluster Version: v3.1.3
  Cluster Type: k8s,bgp,kdd

• Orchestrator version (e.g. kubernetes, mesos, rkt):
  kubeadm 1+3 node cluster of k8s 1.11

• Operating System and version:
  NAME="Debian GNU/Linux"
  VERSION_ID="9"
  VERSION="9 (stretch)"

Linux m01 4.9.0-6-amd64 #1 SMP Debian 4.9.88-1+deb9u1 (2018-05-07) x86_64 GNU/Linux

[tmjd]

Could you take a look at the start of the felix logs (the calico-node container)? I believe it logs the MTU values it reads in for configuration, maybe that will shed some light on why it isn't behaving as expected.