Changing from manage_api_key to manage_own_api_key can cause an error and non-deleted key

[FrankHassanabad]

If you create an alert with a role that has permissions of manage_api_key or manage_security and then later change that role to have the permissions of manage_own_api_key when you next update an alert by enabling/disabling it you get this error message in your console:

server    log   [15:25:03.213] [error][api-key][plugins][security] Failed to invalidate API key: [security_exception] action [cluster:admin/xpack/security/api_key/invalidate] is unauthorized for user [ReadOnlyUser]
server    log   [15:25:03.214] [error][alerting][plugins] Failed to invalidate API Key: [security_exception] action [cluster:admin/xpack/security/api_key/invalidate] is unauthorized for user [ReadOnlyUser]
server    log   [15:25:04.462] [error][api-key][plugins][security] Failed to invalidate API key: [security_exception] action [cluster:admin/xpack/security/api_key/invalidate] is unauthorized for user [ReadOnlyUser]
server    log   [15:25:04.463] [error][alerting][plugins] Failed to invalidate API Key: [security_exception] action [cluster:admin/xpack/security/api_key/invalidate] is unauthorized for user [ReadOnlyUser]

Everything still runs as expected as the new key is replaced but the existing/old key is not deleted from the console logs and you are left with a dangling key.

[elasticmachine]

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

[peterschretlen]

I was able to reproduce using just the elasticsearch apis. I think the issue is manage_own_api_key privilege does not allow invalidating the keys. That is consistent with the docs for invalidating API keys:

To use this API, you must have at least the manage_api_key cluster privilege.

Here are the same steps done with the API

Step 1: create user and two roles - one with manage_own_api_key and the other with manage_api_key

curl -X POST -u elastic:changeme localhost:9200/_security/user/keyuser -H 'Content-Type: application/json' -d '{ "password" : "asdfsadfa", "roles" : [ "apm_user", "manage_api_key"], "full_name" : "Api Key User", "email" : "[email protected]" }'

curl -X POST -u elastic:changeme localhost:9200/ POST /_security/role/own_api_key_manager -d '{ "cluster": [ "manage_own_api_key" ] }'
curl -X POST -u elastic:changeme localhost:9200/ POST /_security/role/api_key_manager -d '{ "cluster": [ "manage_api_key" ] }'

curl -k -X POST -u elastic:changeme https://localhost:9200/_security/role/own_api_key_manager -H 'Content-Type: application/json' -d '{ "cluster": [ "manage_own_api_key" ] }'
curl -k -X POST -u elastic:changeme https://localhost:9200/_security/role/api_key_manager -H 'Content-Type: application/json' -d '{ "cluster": [ "manage_api_key" ] }'

Step 2: assign the api_key_manager role to the user, and create an API key:

curl -k -X POST -u elastic:changeme https://localhost:9200/_security/user/keyuser -H 'Content-Type: application/json' -d '{ "password" : "asdfsadfa", "roles" : [ "apm_user", "api_key_manager"], "full_name" : "Api Key User", "email" : "[email protected]" }'
curl -k -X POST -u keyuser:asdfsadfa https://localhost:9200/_security/api_key -H 'Content-Type: application/json' -d '{ "name": "my-api-key" }'

Step 3: change the role to own_api_key_manager, and then invalidate the key:

curl -k -X PUT -u elastic:changeme https://localhost:9200/_security/user/keyuser -H 'Content-Type: application/json' -d '{ "roles" : [ "apm_user", "own_api_key_manager"] }'
curl -k -X  DELETE -u keyuser:asdfsadfa https://localhost:9200/_security/api_key -H 'Content-Type: application/json' -d' { "name" : "my-api-key" }'

Result:

{
  "error": {
    "root_cause": [
      {
        "type": "security_exception",
        "reason": "action [cluster:admin/xpack/security/api_key/invalidate] is unauthorized for user [keyuser]"
      }
    ],
    "type": "security_exception",
    "reason": "action [cluster:admin/xpack/security/api_key/invalidate] is unauthorized for user [keyuser]"
  },
  "status": 403
}

cc @pmuellr

[peterschretlen]

So alerts should probably only be managed in 7.6 using roles having the manage_api_key privilege.

If you have the manange_own_api_key privilege, you'll be able to create alerts but not update them, because you can't invalidate keys:

curl -k -X POST -u keyuser:asdfsadfa https://localhost:9200/_security/api_key -H 'Content-Type: application/json' -d '{ "name": "new-api-key" }'
{"id":"NnQSoG8BBRffyjk3G4kd","name":"new-api-key","api_key":"qXJuBjCWQWK9eTx_cN3how"}

curl -k -X  DELETE -u keyuser:asdfsadfa https://localhost:9200/_security/api_key -H 'Content-Type: application/json' -d' { "name" : "new-api-key" }'
{"error":{"root_cause":[{"type":"security_exception","reason":"action [cluster:admin/xpack/security/api_key/invalidate] is unauthorized for user [keyuser]"}],"type":"security_exception","reason":"action [cluster:admin/xpack/security/api_key/invalidate] is unauthorized for user [keyuser]"},"status":403}

[peterschretlen]

Related to: elastic/elasticsearch#48716 which will allow kibana system to create API keys on behalf of users and we can avoid these issues.

[pmuellr]

We should certainly doc this - in the README I guess.

Note that Frank says things work as expected, the new key is actually used, the only technical problem is that the old api key is not invalidated.

I wonder if there is some chance we could have the Kibana system user invalidate these, instead of using the user's permission?

[peterschretlen]

I wonder if there is some chance we could have the Kibana system user invalidate these, instead of using the user's permission?

it's good point - elastic/elasticsearch#48716 as it's currently scoped won't be enough, since it only covers creation. We'll need create and delete

[tvernum]

It is possible for a user with manage_own_api_key access to delete their own API keys.

The options are:

1. Pass the owner flag in the DELETE body

   curl -X  DELETE -u keyuser:asdfsadfa http://localhost:9200/_security/api_key -H 'Content-Type: application/json' -d' { "name" : "my-api-key" , "owner": "true" }'
   

2. Authenticate using the API key and then delete it by id

   curl -X  DELETE -H "Authorization: ApiKey MFVidm9XOEI1QjQ0bDhhRHRVaWk6bnUwWTFpQUxRai1aMHFlRDBQcDFNZw==" http://localhost:9200/_security/api_key -H 'Content-Type: application/json' -d' { "id":"0UbvoW8B5B44l8aDtUii" }'
   

[pmuellr]

I just merged the README note, but am wondering whether we should apply a fix suggested by @tvernum ^^^ to get it to work with 7.6, or a 7.6.x release, so re-opened.