Which capabilities are recommended to set?

[ljkiraly]

During the implementation of security hardening we found some settings which should be recommended for NSM users and maybe implemented by NSM.

The aim was to run containers and pods with least privileges.
When deploying NSM images the containers should run as non root user and with minimal set of capabilities. It is also preferred to prevent writing the root file system (readOnlyRootFilesystem: true).

Most of the elements does not need any extra capability. In some cases it is hard to find the minimum set of capabilities.

Using this example Dockerfile an image can be built based on forwarder-vpp. In this new image a user is created and set as default. Also a minimum set of capability was added to the binaries:

FROM cmd-forwarder-vpp:v1.4.0
ARG user=nsm-user
ARG group=nsm-user
ARG uid=10001
ARG gid=10001
RUN groupadd -g ${gid} ${user} && useradd -g ${gid} -l -M -u ${uid} ${user}
RUN setcap cap_dac_override,cap_sys_admin,cap_net_admin=eip /bin/forwarder
RUN setcap cap_ipc_lock,cap_net_raw,cap_sys_ptrace,cap_dac_override,cap_sys_admin,cap_net_admin,cap_setgid=eip /usr/bin/vpp
USER ${uid}
ENTRYPOINT [ "/bin/forwarder" ]

In example manifest files the security context properties should specify the user and the capabilities.
If no default user is set in the image the user and/or group can be configured by runAsUser/runAsGroup.
If the default user is specified in the container image this setting can be overridden with runAsUser/runAsGroup properties.

(forwarder.yaml)
  containers:
    - image: cmd-forwarder-vpp:latest
      securityContext:
             privileged: true
             runAsNonRoot: true
             capabilities:
               drop:
                 - ALL
               add: ["DAC_OVERRIDE", "SYS_ADMIN", "NET_ADMIN", "IPC_LOCK", "NET_RAW","SYS_PTRACE","SETGID"]

Note that these capabilities set in forwarder-vpp image were not fully tested with SR-IOV configuration (some help needed in this area to figure out if additional capabilities are required).

• Can you recommend capability settings for NSM binaries?
• Can these settings be added to Dockerfiles used to build NSM images?
• Can the Dockerfiles contain the user creation?
• Can the example manifests extended with security context settings to run as non-root user, with minimal set of privileges and capabilities?

--
Laszlo

[ljkiraly]

Based on latest test result the example security settings applied in forwarder-vpp image and test manifest file works with SR-IOV configuration also:
https://github.com/networkservicemesh/integration-k8s-packet/actions/runs/2705228841

[denis-tingaikin]

@ljkiraly Is this still actual?

[ljkiraly]

Capability settings for binaries added to Dockerfiles used to build NSM images:
cmd-forwarder-vpp/681
cmd-nsmgr/547

Example manifests (forwarder-vpp, nsmgr) extended with security context settings to run as non-root user, with minimal set of privileges and capabilities:
deployments-k8s/7526

The latest PR depends on first two that's why in draft state. One more step is missing to completion: changing any example manifest in deployments-k8s/apps to test with restricted privileges.
/Laszlo