The exec driver is mishandling capabilities on Linux

[elprans]

Nomad version

Nomad v1.5.3-dev
Revision b84c455

Operating system and Environment details

Ubuntu Linux 22.04

Issue

Capabilities configured in allow_caps (and cap_add) have no effect on the task process as Nomad only configures the Bounding capability set.

Reproduction steps

cap-test.hcl:

job "cap-test" {
  type = "service"

  group "cap-test" {
    count = 1
    network {
      port "https" {
        static = 443
      }
    }

    service {
      name     = "cap-test"
      port     = "https"
      provider = "nomad"
    }

    task "cap-test-task" {
      driver = "exec"

      config {
        cap_drop = ["all"]
        cap_add  = ["net_bind_service"]
        command  = "/usr/bin/nc"
        args     = ["-l", "-p", "${NOMAD_PORT_https}"]
      }
    }
  }
}

nomad job run cap-test.hcl

Expected Result

Job runs successfully, nc can bind to port 443 because net_bind_service is effective.

Actual Result

Job fails with

nc: Permission denied

Root Cause

The issue stems from how Nomad configures the capability sets in libcontainer config:

nomad/drivers/shared/executor/executor_linux.go

Lines 528 to 531 in b84c455

	// otherwise apply the plugin + task capability configuration
	cfg.Capabilities = &lconfigs.Capabilities{
	Bounding: command.Capabilities,
	}

Since the task is unprivileged (i.e. not run as "root"), the privileges set up by libcontainer will not survive execve unless they are specified in the Ambient set.

Per CAPABILITIES(7):

Ambient (since Linux 4.3)

This is a set of capabilities that are preserved across an execve(2) of a program that is not privileged. The ambient capability set obeys the invariant that no capability can ever be ambient if it is not both permitted and inheritable.

[tgross]

Hi @elprans! I did some code spelunking and it looks like we split out this capabilities set when we fixed CVE-2019-12618 a while back, but then when we added the add_caps feature in #10600 we didn't configure the the Ambient set, as you've noted. I'm reviewing and testing your PR #16643 and that looks roughly correctly. I'll get back to you if I find any issues. Thanks!