Nomad Actions

[philrenaud]

Introduces the concept of Actions to Nomad: operator-defined commands and workflows that Nomad users can run on an allocation without having to sweat the details.

These take a similar form to task config blocks and live at task level in the jobspec. To demonstrate:

job "actions-demo" {
  type = "system"

  group "group" {
    task "task" {
      driver = "raw_exec"
      config {
        command = "node"
        args    = ["-e",
        <<EOT
console.log('Howdy!');
// sleep every 3 seconds
setInterval((a) => {
  console.log('Howdy after 3 seconds');
}, 3000);
EOT
        ]
      }

      action "weather" {
        command = "/usr/bin/curl"
        args = ["-s", "wttr.in/Toronto?format=3"]
      }
      action "hello" {
        command = "say"
        args = ["howdy"]
      }
      action "get job info" {
        command = "/bin/sh"
        args    = ["-c",
          <<EOT
          nomad job status ${NOMAD_JOB_NAME}
          EOT
        ]
      }
      action "get alloc info" {
        command = "/bin/sh"
        args    = ["-c",
          <<EOT
          nomad alloc status ${NOMAD_ALLOC_ID}
          EOT
        ]
      }

      action "echo time" {
        command = "/bin/sh"
        args    = ["-c", "counter=0; while true; do echo \"Running for ${counter} seconds\"; counter=$((counter + 1)); sleep 1; done"]
      }

    }
  }
}

These actions can be run from the CLI with nomad action by passing in either the allocation ID, or if none is provided, the task and group names and an allocation will be randomly selected:

nomad action \
  -allocation=cafee3ae \
  -job=actions-demo \
weather

nomad action \
  -group=group \
  -task=task \
  -job=actions-demo \
"weather"

An endpoint at v1/job/:id/actions will GET return a list of actions the job has access to, and a GET at v1/job/:id/action?action=NAME&allocID=ID&group=GROUP&task=TASK&tty=true will open a websocket with base64-encoded output (be sure to use Upgrade: websocket and related headers)

Future work:

When #18793 lands, the Nomad web UI will display interactive elements to run actions and view their output in various places:

Further, part of the intent behind actions is to give operators greater ability to specify which sorts of things a Nomad user can perform. With this in mind, ACL changes with actions in mind are planned in the near future.

Resolves #18627

[github-actions]

Ember Test Audit comparison

	main	1fe0e11	change
passes	1533	1534	+1
failures	1	1	0
flaky	0	0	0
duration	000ms	000ms	-000ms

[tgross]

LGTM once the ACL check comment is fixed.

[kaspergrubbe]

This is amazing, does it support giving the actions any form of input?

[philrenaud]

@kaspergrubbe We've been toying around with the idea internally, and I'd be interested in hearing your use-case. My own feeling on the matter is that allowing operators to provide variable input for actions would create some degree of risk and make the action workflows/output less "guaranteed".

Certainly it would expand functionality and open up some cool new stuff, but we want to weigh that against the potential of providing a foot-gun here.

[benvanstaveren]

A late comment, but @philrenaud, I want my foot gun. I've got a few cases where being able to enter arguments to an action would be a godsend (even if triggered through the API only); perhaps cage it behind a configuration flag that needs to be explicitly enabled to allow it so that it can be decorated with the usual "warning, foot gun" disclaimers in the docs?