Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DLS optimization for rewrite to match all #80904

Closed
henningandersen opened this issue Nov 22, 2021 · 1 comment · Fixed by #81030
Closed

DLS optimization for rewrite to match all #80904

henningandersen opened this issue Nov 22, 2021 · 1 comment · Fixed by #81030
Assignees
@ywangd
Labels
>enhancement :Security/Authorization Roles, Privileges, DLS/FLS, RBAC/ABAC Team:Security Meta label for security team

Comments

@henningandersen
Copy link
Contributor

A possible way to limit users from searching cold or frozen tiers is to setup DLS with a terms query for the _tier field. If an index does not match the requested _tier, rewriting the query to the shard results in a MatchNoDocsQuery, which is handled well by DLS, caching only that the index is unavailable. However, if the role does not contain any other query parts than for the _tier field, the rewrite will result in a MatchAllDocsQuery. There is an optimization for this case too, but it does not kick in until after the query ran populating a bitset of all the documents in the shard. Instead we can optimize this by checking if the rewritten role query becomes MatchAllDocsQuery and use the special match_all bitset directly in this case. This would hopefully nearly eliminate the shard level overhead of having a role query with only a _tier predicate. There is room for further improvements, but we can consider those outside this issue.
@henningandersen henningandersen added >enhancement :Security/Authorization Roles, Privileges, DLS/FLS, RBAC/ABAC needs:triage Requires assignment of a team area label labels Nov 22, 2021
@elasticmachine elasticmachine added the Team:Security Meta label for security team label Nov 22, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-security (Team:Security)

@not-napoleon not-napoleon removed the needs:triage Requires assignment of a team area label label Nov 22, 2021
ywangd added a commit to ywangd/elasticsearch that referenced this issue Nov 25, 2021
@ywangd
Optimize DLS bitset building for matchAll query
c5230b7 
The PR avoids creating Weight and Scorer and stepping through
docIterator when building DLS bitSet for an effective matchAll query.
Instead it returns a MatchAllRoleBitSet directly after query rewritten
for this scenario.

Resolves: elastic#80904
@ywangd ywangd mentioned this issue Nov 25, 2021
Merged
ywangd added a commit that referenced this issue Nov 29, 2021
@ywangd
Optimize DLS bitset building for matchAll query (#81030)
806abee 
The PR avoids creating Weight and Scorer and stepping through
docIterator when building DLS bitSet for an effective matchAll query.
Instead it returns a MatchAllRoleBitSet directly after query rewritten
for this scenario.

Resolves: #80904
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ywangd ywangd

Labels
>enhancement :Security/Authorization Roles, Privileges, DLS/FLS, RBAC/ABAC Team:Security Meta label for security team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Optimize DLS bitset building for matchAll query ywangd/elasticsearch
4 participants
@not-napoleon @ywangd @elasticmachine @henningandersen