_cat/indices with Security, hide names when wildcard #38824
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
albertzaharovits
added
>bug
:Security/Authorization
|
Pinging @elastic/es-security |
| assert concreteIndices.length == state.metaData().getIndices().size(); | ||
|
|
||
| final ClusterState clusterState = clusterStateResponse.getState(); | ||
| final IndexMetaData[] indicesMetaData = getOrderedIndexMetaData(indices, clusterState, strictExpandIndicesOptions); |
There was a problem hiding this comment.
As the code suggests, indexNameExpressionResolver.concreteIndices should probably return the resolved indices expression in the order that they were requested. It does not appear to be the case now, but I nevertheless kept the logic based on this assumption in the spirit of introducing a single change in a PR.
There was a problem hiding this comment.
we put indices in a set I think, then ordering is not deterministic
javanna
reviewed
Feb 13, 2019
| @@ -170,4 +188,19 @@ private IndicesStatsResponse randomIndicesStatsResponse(final Index[] indices) { | |||
| shardStats.toArray(new ShardStats[shardStats.size()]), shardStats.size(), shardStats.size(), 0, emptyList() | |||
| ); | |||
| } | |||
|
|
|||
| private <T> T[] removeRandomElement(T[] array) { | |||
There was a problem hiding this comment.
maybe you can use a List and avoid all the ceremony need to remove an element from an array?
| assert concreteIndices.length == state.metaData().getIndices().size(); | ||
|
|
||
| final ClusterState clusterState = clusterStateResponse.getState(); | ||
| final IndexMetaData[] indicesMetaData = getOrderedIndexMetaData(indices, clusterState, strictExpandIndicesOptions); |
There was a problem hiding this comment.
we put indices in a set I think, then ordering is not deterministic
javanna
approved these changes
Feb 13, 2019
|
sounds good @albertzaharovits thanks for pinging |
albertzaharovits
added a commit
that referenced
this pull request
Feb 14, 2019
This changes the output of the `_cat/indices` API with `Security` enabled. It is possible to only display the index name (and possibly the index health, depending on the request options) but not its stats (doc count, merges, size, etc). This is the case for closed indices which have index metadata in the cluster state but no associated shards, hence no shard stats. However, when `Security` is enabled, and the request contains wildcards, **open** indices without stats are a common occurrence. This is because the index names in the response table are picked up directly from the cluster state which is not filtered by `Security`'s _indexNameExpressionResolver_, unlike the stats data which is populated by the indices stats API which does go through the index name resolver. This is a bug, because it is circumventing `Security`'s function to hide unauthorized indices. This has been fixed by displaying the index names as they are resolved by the indices stats API. The outputs of these two APIs is now very similar: same index names, similar data but different format. Closes #37190
jasontedor
added a commit
to jasontedor/elasticsearch
that referenced
this pull request
Feb 14, 2019
* elastic/master: Remove immediate operation retry after mapping update (elastic#38873) Remove mentioning of types from bulk API docs (elastic#38896) SQL: change JDBC setup URL in the documentation (elastic#38564) Skip BWC tests in checkPart1 and checkPart2 (elastic#38730) Enable silent FollowersCheckerTest (elastic#38851) Update TESTING.asciidoc with platform specific instructions (elastic#38802) Use consistent view of realms for authentication (elastic#38815) Stabilize RareClusterState (elastic#38671) Increase Timeout in UnicastZenPingTests (elastic#38893) Do not recommend installing vagrant-winrm elastic#38887 _cat/indices with Security, hide names when wildcard (elastic#38824) SQL: fall back to using the field name for column label (elastic#38842) Fix LocalIndexFollowingIT#testRemoveRemoteConnection() test (elastic#38709) Remove joda time mentions in documentation (elastic#38720) Add enabled status for token and api key service (elastic#38687)
tlrx
added a commit
that referenced
this pull request
Jun 25, 2019
After two recent changes (#38824 and #33888), the _cat/indices API no longer report information for active recovering indices and non-replicated closed indices. It also misreport replicated closed indices that are potentially not authorized for the user. This commit changes how the cat action works by first using the Get Settings API in order to resolve authorized indices. It then uses the Cluster State, Cluster Health and Indices Stats APIs to retrieve information about the indices. Closes #39933
tlrx
added a commit
to tlrx/elasticsearch
that referenced
this pull request
Jun 25, 2019
After two recent changes (elastic#38824 and elastic#33888), the _cat/indices API no longer report information for active recovering indices and non-replicated closed indices. It also misreport replicated closed indices that are potentially not authorized for the user. This commit changes how the cat action works by first using the Get Settings API in order to resolve authorized indices. It then uses the Cluster State, Cluster Health and Indices Stats APIs to retrieve information about the indices. Closes elastic#39933
tlrx
added a commit
that referenced
this pull request
Jun 25, 2019
After two recent changes (#38824 and #33888), the _cat/indices API no longer report information for active recovering indices and non-replicated closed indices. It also misreport replicated closed indices that are potentially not authorized for the user. This commit changes how the cat action works by first using the Get Settings API in order to resolve authorized indices. It then uses the Cluster State, Cluster Health and Indices Stats APIs to retrieve information about the indices. Closes #39933
tlrx
added a commit
to tlrx/elasticsearch
that referenced
this pull request
Jun 26, 2019
After two recent changes (elastic#38824 and elastic#33888), the _cat/indices API no longer report information for active recovering indices and non-replicated closed indices. It also misreport replicated closed indices that are potentially not authorized for the user. This commit changes how the cat action works by first using the Get Settings API in order to resolve authorized indices. It then uses the Cluster State, Cluster Health and Indices Stats APIs to retrieve information about the indices. Closes elastic#39933
tlrx
added a commit
that referenced
this pull request
Jun 26, 2019
After two recent changes (#38824 and #33888), the _cat/indices API no longer report information for active recovering indices and non-replicated closed indices. It also misreport replicated closed indices that are potentially not authorized for the user. This commit changes how the cat action works by first using the Get Settings API in order to resolve authorized indices. It then uses the Cluster State, Cluster Health and Indices Stats APIs to retrieve information about the indices. Closes #39933
37 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This changes the output of the
_cat/indicesAPI withSecurityenabled.Previously, it wasIt is possible to only display the index name (and possibly the index health, depending on the options) but not its stats (doc count, merges, size, etc).However, under usual circumstances, it was not possible to actually observe this, maybe just under a transient state (the index name should exist, but with no shards assigned).This is the case for closed indices which have index metadata in the cluster state but no associated shards, hence no shard stats. However, whenSecurityis enabled, and the request contains wildcards, open indices without stats are a common occurrence. This is because the index names in the response table are picked up directly from the cluster state which is not filtered bySecurity's indexNameExpressionResolver, unlike thestats data which is populated by the indices stats API which does go through the index name resolver.
This is a bug, because it is circumventing
Security's function to hide unauthorized indices.This has been fixed by displaying the index names as they are resolved by the indices stats API. The outputs of these two APIs is now very similar: same index names , similar data but different format.
Closes #37190
Even though the main code changes went into the server code, they impact the
Securityplugin, so I have labeled it in consequence.cc @elastic/es-core-features