Add manage roles privilege

[jfreden]

This PR adds functionality to limit the resources and privileges an Elasticsearch user can grant permissions to when creating a role. This is achieved using a new global (configurable/request aware) cluster privilege , named role, with a sub-key called manage/indices which is an array where each entry is a pair of index patterns and index privileges.

Definition

• Using a role with this privilege to create, update or delete roles with privileges on indices outside of the indices matched by the index pattern in the indices array, will fail.
• Using a role with this privilege to try to create, update or delete roles with cluster, run_as, etc. privileges will fail.
• Using a role with this privilege with restricted indices will fail.
• Other broader privileges (such as manage_security) will nullify this privilege.

Example

Create test-manage role:

POST _security/role/test-manage
{
    "global": {
        "role": {
            "manage": {
                "indices": [
                    {
                        "names": ["allowed-index-prefix-*"],
                        "privileges":["read"]
                    }
                ]
            }
        }
    }
}

And then a user with that role creates a role:

POST _security/role/a-test-role
{
    "indices": [
        {
            "names": [
                "allowed-index-prefix-some-index"
            ],
            "privileges": [
                "read"
            ]}]
}

But this would fail for:

POST _security/role/a-test-role
{
    "indices": [
        {
            "names": [
                "not-allowed-index-prefix-some-index"
            ],
            "privileges": [
                "read"
            ]}]
}

Backwards compatibility and mixed cluster concerns

• A new mapping version has been added to the security index to store the new privilege.
• If the new mapping version is not applied and a role descriptor with the new global privilege is written, the write will fail causing an exception.
• When sending role descriptors over the transport layer in a mixed cluster, the new global privilege needs to be excluded for older versions. This is hanled with a new transport version.
• If a role descriptor is serialized for API keys on one node in a mixed cluster and read from another, an older node might not be able to deserialize it, so it needs to be removed before being written in mixed cluster with old nodes. This is handled in the API key service.
• If a role descriptor containing a global privilege is in a put role request in a mixed cluster where it's not supported on all nodes, fail request to create role.
• RCS is not applicable here since RCS only considers cluster privileges and index privileges (not global cluster privileges).
• This doesn't include remote privileges, since the current use case with connectors doesn't need roles to be created on a cluster separate from the cluster where the search data resides.

Follow up work

• Create a docs PR
• Error handling for actions that use manage roles. Should configurable cluster privileges that grant restricted usage of actions be listed in error authorization error messages?

[elasticsearchmachine]

Hi @jfreden, I've created a changelog YAML for you.

[elasticsearchmachine]

Pinging @elastic/es-security (Team:Security)

[jfreden]

The tests in ManageRolesPrivilegesTests::testPrivilegeIntersectionPutRoleRequest makes this easier to reason about.

[slobodanadamovic]

The high level changes are looking good. I'm leaving a few comments just to give you the initial feedback, but I still need more time to complete review of all changes.

[jfreden]

Added an integration test for an API Key with manage_roles to make sure we cover that code path.

[jfreden]

Extended the new API key test to also include a limited-by-role test.

[slobodanadamovic]

LGTM Thanks for iterations. 👍

[jfreden]

Thanks for the great review!