"Operation not permitted" messages logged after initialization

[zim68]

I want to establish an MPTCP connection between two servers with multiple subflows on each side.

Configuration of both servers

• multiple CPUs
• 4-port 1 Gbps Ethernet interface
• Linux Fedora 38 Server (up-to-date)
• Linux Kernel 6.6.2-101.fc38.x86_64
• mptcpd-0.11-2.fc38 installed
• sudo ip mptcp limit show
  add_addr_accepted 3 subflows 3

Configuration of server 1

• sudo ip mptcp endpoint show
  192.168.3.191 id 1 signal subflow dev eno1
  192.168.3.192 id 2 signal subflow dev eno2
  192.168.3.193 id 3 signal subflow dev eno3
  192.168.3.194 id 4 signal subflow dev eno4

Configuration of server 2

• sudo ip mptcp endpoint show
  192.168.3.161 id 1 signal subflow dev eno0
  192.168.3.162 id 2 signal subflow dev eno1
  192.168.3.163 id 3 signal subflow dev eno2
  192.168.3.164 id 4 signal subflow dev eno3

On both servers the mptcpd service could be started but logs "Operation not permitted" messages.

Service logs (reversed order) - mptcpd

12:11 PM add_addr: Operation not permitted
12:11 PM set_limits: Operation not permitted
12:11 PM add_addr: Operation not permitted
12:11 PM set_limits: Operation not permitted
12:11 PM add_addr: Operation not permitted
12:11 PM set_limits: Operation not permitted
12:11 PM add_addr: Operation not permitted
12:11 PM set_limits: Operation not permitted (2 occurrences)
12:11 PM MPTCP single-subflow-per-interface path manager initialized.
12:11 PM MPTCP address advertiser path manager initialized.
12:11 PM Started mptcp.service - Multipath TCP service.

Log message details

set_limits: Operation not permitted
CODE_FILE
commands.c
CODE_FUNC
mptcpd_check_genl_error
CODE_LINE
71
PRIORITY
3
_BOOT_ID
271ac867969843c09c09ed3c84cfc958
_CAP_EFFECTIVE
1000
_CMDLINE
/usr/libexec/mptcpd --log=journal
_COMM
mptcpd
_EXE
/usr/libexec/mptcpd
_GID
62966
_HOSTNAME
HP-DL380-1
_MACHINE_ID
fbfba620f3184e6fb75ce66a4090bfe8
_PID
1784
_RUNTIME_SCOPE
system
_SELINUX_CONTEXT
system_u:system_r:mptcpd_t:s0
_SOURCE_REALTIME_TIMESTAMP
1701343190958054
_SYSTEMD_CGROUP
/system.slice/mptcp.service
_SYSTEMD_INVOCATION_ID
9d552c1842244fda9d963265648ca149
_SYSTEMD_SLICE
system.slice
_SYSTEMD_UNIT
mptcp.service
_TRANSPORT
journal
_UID
62966
__CURSOR
s=c3ce83a68e9e4cc99c5dd7f4f91e3a23;i=77b46;b=271ac867969843c09c09ed3c84cfc958;m=17a557b;t=60b5cd489c412;x=80846d23a838ac4f
__MONOTONIC_TIMESTAMP
24794491
__REALTIME_TIMESTAMP
1701343190959122

add_addr: Operation not permitted
CODE_FILE
commands.c
CODE_FUNC
mptcpd_check_genl_error
CODE_LINE
71
PRIORITY
3
_BOOT_ID
271ac867969843c09c09ed3c84cfc958
_CAP_EFFECTIVE
1000
_CMDLINE
/usr/libexec/mptcpd --log=journal
_COMM
mptcpd
_EXE
/usr/libexec/mptcpd
_GID
62966
_HOSTNAME
HP-DL380-1
_MACHINE_ID
fbfba620f3184e6fb75ce66a4090bfe8
_PID
1784
_RUNTIME_SCOPE
system
_SELINUX_CONTEXT
system_u:system_r:mptcpd_t:s0
_SOURCE_REALTIME_TIMESTAMP
1701343199804620
_SYSTEMD_CGROUP
/system.slice/mptcp.service
_SYSTEMD_INVOCATION_ID
9d552c1842244fda9d963265648ca149
_SYSTEMD_SLICE
system.slice
_SYSTEMD_UNIT
mptcp.service
_TRANSPORT
journal
_UID
62966
__CURSOR
s=c3ce83a68e9e4cc99c5dd7f4f91e3a23;i=77c51;b=271ac867969843c09c09ed3c84cfc958;m=201533a;t=60b5cd510c1d1;x=ebc3b2a80e6ec26f
__MONOTONIC_TIMESTAMP
33641274
__REALTIME_TIMESTAMP
1701343199805905

Expected behavior

There are no errors/warnings after service mptcpd has been started and initialized.

[zim68]

UPDATE:
After deactivation of the plugin "addr_adv" in /etc/mptcp/mptcp.conf and restarting the service "mptcpd" the error messages will not print anymore.

log=stderr plugin-dir=/usr/lib64/mptcpd path-manager=sspi addr-flags=signal,subflow notify-flags=existing,skip_link_local,skip_loopback load-plugins=sspi

But that's not the goal to be achieved.
So it is clear that this plugin triggers these error messages.

What did I configure incorrectly?

[ivanbaldo]

Happening to me in an up to date RHEL 9.4 server...

[ossama-othman]

The set_limits and add_addr operations executed by the addr_adv plugin require the CAP_NET_ADMIN capability if not running as root. Otherwise those operations will result in the "Operation not permitted" (EPERM) error that you're seeing.

The mptcpd.service systemd unit file used to start mptcpd should already set up the necessary capabilities for the mptcp dynamic user under which mptcpd runs. If you're running mptcpd without systemd and want to use the addr_adv plugin without running as root, you'll have to explicitly set the CAP_NET_ADMIN capability, as described in the README.

[ivanbaldo]

With the mptcpd-0.8-3.el9 package installed it doesn't work.
If I run it as root manually with /usr/libexec/mptcpd then it does work.
Maybe some other capability is needed too?

# systemctl cat mptcp
# /usr/lib/systemd/system/mptcp.service
# SPDX-License-Identifier: BSD-3-Clause
#
# Copyright (c) 2017-2019, Intel Corporation

[Unit]
Description=Multipath TCP service
Documentation=man:mptcpd(8)

[Service]
Type=simple
DynamicUser=yes
ExecStart=/usr/libexec/mptcpd --log=journal
CapabilityBoundingSet=CAP_NET_ADMIN
AmbientCapabilities=CAP_NET_ADMIN
LimitNPROC=1

[Install]
WantedBy=sockets.target

Thanks!!