Document permissions and/or role required for user to run X-Pack features

[dedemorton]

Currently we document how to set up users for Logstash (here), but we don't document which roles are required when using specific features, like the pipeline viewer and monitoring.

(One exception is centralized config management, where we do document the required roles: https://www.elastic.co/guide/en/logstash/master/logstash-centralized-pipeline-management.html)

@karenzone You'll need to investigate which roles are required for using the pipeline viewer and monitoring.

[jakelandis]

I'm going to hijack this issue for my opinion on the current state of https://www.elastic.co/guide/en/logstash/current/ls-security.html and suggestions for how to fix.

Logstash Internal

 _xpack/security/role/logstash_writer -> ["manage_index_templates", "monitor"]

_xpack/security/user/logstash_internal ->  "roles" : [ "logstash_writer"]

input {
  elasticsearch {
    ...
    user => logstash_internal

• I don't think this user needs the monitor privilge. (they do need it for the health check)
• (opinion) Logstash Internal is kinda weird name, Internal is a relative term and can mean different things based on context.

Logstash User

_xpack/security/role/logstash_reader ->  ["read","view_index_metadata"]

_xpack/security/user/logstash_user -> [ "logstash_reader", "logstash_admin"],

• No examples of where to use this user is intended to be used.
• The monitor user vs. central config is hinted at, but not explicit nor in the examples here.
• We shouldn't advocate a "user" to have "admin" level privileges.

Logstash System

_xpack/security/user/logstash_system/_password

xpack.monitoring.elasticsearch.username: logstash_system

_xpack/security/user/logstash_system/_enable

• This is not a pre-defined user. Running this either the password or enable command will result in an error "user must exist in order to change password;"
• This is a pre-defined role for the monitoring, but should it also be used for central management ?
• There is no example for how to create this user.

Logstash admin user

xpack.management.elasticsearch.username: logstash_admin_user

• Where did this user come from ? (there are no example for this)
• This user seems to overlap with the recommedations from both logstash user with the admin role as well the definition (but lacking the role) for the logstash_system user.

Suggested fixes:

To discuss ...

[jakelandis]

We also need to reconcile any changes/documentation with this: https://www.elastic.co/guide/en/x-pack/current/built-in-roles.html

[dedemorton]

Looping @lcawl in on this discussion since it pertains to security. TBH, I'm not sure who wrote the original text here because the content has moved around a lot and lost its history. She might have more detail on specific users/roles and why we suggest specific privileges.

I did test this content last year. If you change the user names/roles/privileges, you'll need to make sure you update (and test) any topics that refer to them.

[lcawl]

The original home of that page was https://www.elastic.co/guide/en/x-pack/5.0/logstash.html, but it hasn't had too many substantive updates since that time, so there's definitely room for improvement.

With respect to the logstash_system built-in user, it's one of the ones that's created when you have X-Pack security enabled and its initial password is set by using the setup-password command, per https://www.elastic.co/guide/en/x-pack/master/setting-up-authentication.html#built-in-users
I believe the statement "By default, the logstash_system user does not have a password" in https://www.elastic.co/guide/en/logstash/master/ls-security.html is incorrect, so that will need to be corrected too.

I'm happy to help with these updates.

[jakelandis]

I removed the suggested fixes as they were too aggressive w/r/t passivity. We can discuss and I will take another stab at a suggestion.

[jakelandis]

There are basically three index patterns needed for Logstash default.

• logstash* - the events ingested
• .logstash* - the data that backs management (x-pack)
• .monitoring-logstash* - the data that backs monitoring (x-pack)

For logstash* , there are primary two personas:

• the Logstash system writing events and managing the time based index
• the Human reading these events, presumably from Kibaba

For .logstash*, there are two personas:

• the Logstash system reading the management data
• the Human changing the central config (management data)
  (note we don't to support reading the central config as standalone persona due to security reasons)

For .monitoring-logstash*

• the Logstash system writing the monitoring data
• the Human reading the monitoring data

@ycombinator @andrewvc - would you agree that these are personas that should back suggestions for RBAC ?

[jakelandis]

Out of the box, it appears that we have 2 pre-defined roles (and 0 pre-defined users):

Logstash admin role:

and Logstash system role:

It seems that the Logstash admin role is target at "the Human changing the central config " , which seems correct.

The documentation states that Logstash system role "Grants access necessary for the Logstash system user to send system-level data (such as monitoring) to Elasticsearch.' , which seems to be targeting the "the Logstash system writing the monitoring data, and reading the management data" persona. However, the privileges don't appear be correct since there are no index level privs and the existing documentation refers to enabling a user that does not work. I think persona needs the most attention, and may require changes to the default roles.

The other personas seem to be documented as logstash reader/writer roles which also seem correct, but may benefit from some updates for clarification.

[jakelandis]

I think this proposal covers the personas and is a minimal difference to what is currently there.

Human / System	User	Role
System	logstash_system_output	logstash_system (for cluster health)+ logstash_writer (for logstash)
System	logstash_system	logstash_system (for cluster health) + logstash_admin (for .logstash) + ?? role (for .monitoring-logstash)
Human	logastash_user	logstash_reader (for logstash)
Human	logstash_user_admin	logstash_reader (for logstash) + logstash_admin (for .logstash) + monitoring_user (for .monitoring-logstash)

[jakelandis]

bin/x-pack/setup-passwords interactive

^^ is needed to bootstrap the logstash_system user.

logstash_system
The user Logstash uses when storing monitoring information in Elasticsearch.

so I think that bootstrapping step is needed to give the .monitoring-logstash* write privs ... but it is unclear how to do that for cloud deployment.

• Note - since the logstash_system user is a special reserved user, we will want to confirm it's exact permissions. I do believe however, that both management and monitoring should be serviced by this user.

[jakelandis]

?? role (for .monitoring-logstash)

I confirmed that the logstash_system role has implicit privs to write to the monitoring index.

Per the proposal above I suggest we should (by default) give the logstash_system user the logstash_admin role to as well to allow (by default) the logstash_system user to be used for both monitoring and management.

is needed to bootstrap the logstash_system user.

Also confirmed that this user is not currently available via cloud, so we will likely want to explicitly document how to set this user up. (note the role is present , just not the user)

[ycombinator]

For .logstash*, there are two personas:

the Logstash system reading the management data
the Human changing the central config (management data)

I agree with the personas.

(note we don't to support reading the central config as standalone persona due to security reasons)

Can you elaborate on this? I've always wondered about this: why the Logstash system doesn't get just read access to .logstash, because it doesn't need to write to it. From a security POV, I'd think we'd want to provide as restrictive access as necessary, no?

[jakelandis]

Can you elaborate on this? I've always wondered about this: why the Logstash system doesn't get just read access to .logstash, because it doesn't need to write to it

The comment was in reference to the human use case, where we should never give a human the ability to read but not write. Doing that would open the door to accidentally give a non-admin user the ability see potentially sensitive information.

I wasn't part of the part the decision w/r/t the definition of the role, but it does make sense for the human use case.

From a security POV, I'd think we'd want to provide as restrictive access as necessary, no?

Agreed for the system use case. Perhaps we should try address that with a change to the logstash_system role to provide that role with the read access .logstash ? (that would mean in a minor change to the proposed logstash_system_output user above)

[ycombinator]

The comment was in reference to the human use case, where we should never give a human the ability to read but not write. Doing that would open the door to accidentally give a non-admin user the ability see potentially sensitive information.

The comment was in reference to the human use case, where we should never give a human the ability to read but not write. Doing that would open the door to accidentally give a non-admin user the ability see potentially sensitive information.

I wasn't part of the part the decision w/r/t the definition of the role, but it does make sense for the human use case.

Agreed. Makes sense. Thanks for the explanation.

Perhaps we should try address that with a change to the logstash_system role to provide that role with the read access .logstash ? (that would mean in a minor change to the proposed logstash_system_output user above)

Yeah, this makes sense to me. It does mean that all existing users of Logstash would've given their Logstash (system) read and write privileges to .logstash but at least new users would have the option of using a more restricted user (logstash_system_output).

[dedemorton]

For the Beats -> LS -> ES (with secured cluster) use case, we also need to document the privileges that are required to write/read to beats indexes for users who want to use the beats dashboards--that is, users who have this config:

output {
  elasticsearch {
    hosts => "localhost:9200"
    manage_template => false
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" 
    document_type => "%{[@metadata][type]}"
    user => "logstash_internal"
    password => "x-pack-test-password"
  }
}

You get the following security exception if the user doesn't have the correct privileges:

[2018-05-17T18:35:26,780][INFO ][logstash.outputs.elasticsearch] retrying failed action with 
response code: 403 ({"type"=>"security_exception", "reason"=>"action [indices:admin/create] is 
unauthorized for user [logstash_internal]"})

[ashokbellur]

Hello, I am exactly running into the above issue using stack 6.3.1.

I have a secured cluster with almost similar config.

Is this issue resolved now?
][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"security_exception", "reason"=>"action [indices:admin/create] is unauthorized for user [logstash_internal]"})

[dedemorton]

@ashokbellur Make sure your logasth_internal user has write and create_index privileges on the Beats index you are trying to send events to. Note that this GitHub issue is for tracking updates required to the documentation. If you have questions about configuring Logstash and/or Beats, please ask in the discussion forums. See:

https://discuss.elastic.co/c/logstash
https://discuss.elastic.co/c/beats

[dedemorton]

@karenzone Is this issue still valid, or should I close it?

[dedemorton]

closing because this issue is soooo old.