Adjust /_cat/templates not to request all metadata #78812
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Today `GET /_cat/templates` retrieves the whole cluster metadata from the master, which includes all sorts of unnecessary junk and consumes significant resources. This commit reimplements these endpoints using `GetIndexTemplatesAction` and `GetComposableIndexTemplateAction` which are much more efficient. The docs for this API indicate that it accepts a comma-separated list of template names/patterns of the form `GET /_cat/templates/name1,name2` but in fact today it only accepts a single name or pattern. This commit also adds support for multiple names/patterns as the docs claim.
DaveCTurner
added
>bug
:Data Management/CAT APIs
|
Pinging @elastic/es-data-management (Team:Data Management) |
| final GetIndexTemplatesRequest getIndexTemplatesRequest = new GetIndexTemplatesRequest(templateNames); | ||
| getIndexTemplatesRequest.local(request.paramAsBoolean("local", getIndexTemplatesRequest.local())); | ||
| getIndexTemplatesRequest.masterNodeTimeout(request.paramAsTime("master_timeout", getIndexTemplatesRequest.masterNodeTimeout())); | ||
| final StepListener<GetIndexTemplatesResponse> getIndexTemplatesStep = new StepListener<>(); |
There was a problem hiding this comment.
Maybe move this into the channel -> {} lambda, it's kind of weird having it outside of its scope when it's only used in the lambda? (at least it took me a little to reason out the failure paths here.
There was a problem hiding this comment.
Sure, will do once we've resolved the other point you raised too.
| return table; | ||
| } | ||
|
|
||
| private Predicate<String> getNamePredicate(String[] requestedNames) { |
There was a problem hiding this comment.
Why do we need this, shouldn't we rely on the template request to only returns stuff matching our initial cat request since we pass on the index name? (I'm sure we do, but I can't really figure out why I must admit :))
There was a problem hiding this comment.
TransportGetComponentTemplateAction only accepts a single name or pattern and throws an exception if it's not found - it might be not found because it's a legacy template name or because it's really just not there but in any case this API doesn't return a 404 in that case, just an empty response. We could optimise for the one-argument case if we think the number of component templates will be ovewhelming otherwise, but I judge that it's ok to just get them all.
original-brownbear
approved these changes
Oct 7, 2021
DaveCTurner
added a commit
to DaveCTurner/elasticsearch
that referenced
this pull request
Oct 7, 2021
Today `GET /_cat/templates` retrieves the whole cluster metadata from the master, which includes all sorts of unnecessary junk and consumes significant resources. This commit reimplements these endpoints using `GetIndexTemplatesAction` and `GetComposableIndexTemplateAction` which are much more efficient. The docs for this API indicate that it accepts a comma-separated list of template names/patterns of the form `GET /_cat/templates/name1,name2` but in fact today it only accepts a single name or pattern. This commit also adds support for multiple names/patterns as the docs claim. Backport of elastic#78812
elasticsearchmachine
pushed a commit
that referenced
this pull request
Oct 7, 2021
Today `GET /_cat/templates` retrieves the whole cluster metadata from the master, which includes all sorts of unnecessary junk and consumes significant resources. This commit reimplements these endpoints using `GetIndexTemplatesAction` and `GetComposableIndexTemplateAction` which are much more efficient. The docs for this API indicate that it accepts a comma-separated list of template names/patterns of the form `GET /_cat/templates/name1,name2` but in fact today it only accepts a single name or pattern. This commit also adds support for multiple names/patterns as the docs claim. Backport of #78812
DaveCTurner
added a commit
that referenced
this pull request
Oct 7, 2021
This reverts commit b07dbe5.
wjp719
added a commit
to wjp719/elasticsearch
that referenced
this pull request
Oct 7, 2021
…' into feature/data_stream_support_routing * wjp/feature/data_stream_support_routing: (44 commits) Revert "Adjust /_cat/templates not to request all metadata (elastic#78812)" Allow indices lookup to be built lazily (elastic#78745) [DOCS] Document default security in alpha2 (elastic#78227) Add cluster applier stats (elastic#77552) Fix failing URLDecodeProcessorTests::testProcessor test (elastic#78690) Upgrade to lucene snapshot ba75dc5e6bf (elastic#78817) Adjust /_cat/templates not to request all metadata (elastic#78812) Simplify build plugin license handling (elastic#77009) Fix SearchableSnapshotsBlobStoreCacheIntegTests.testBlobStoreCache (elastic#78616) Improve Docker image caching and testing (elastic#78552) Load knn vectors format with mmapfs (elastic#78724) Fix date math zone test to use negative minutes (elastic#78796) Changing name of shards field in node/stats api to shard_stats (elastic#78531) [DOCS] Fix system index refs in restore tutorial (elastic#78582) Add previously removed settings back for 8.0 (elastic#78784) TSDB: Fix template name in test Add a system property to forcibly format everything (elastic#78768) Revert "Adding config so that some tests will break if over-the-wire encryption fails (elastic#78409)" (elastic#78787) Must date math test failure Adding config so that some tests will break if over-the-wire encryption fails (elastic#78409) ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Today
GET /_cat/templatesretrieves the whole cluster metadata fromthe master, which includes all sorts of unnecessary junk and consumes
significant resources. This commit reimplements these endpoints using
GetIndexTemplatesActionandGetComposableIndexTemplateActionwhichare much more efficient.
The docs for this API indicate that it accepts a comma-separated list of
template names/patterns of the form
GET /_cat/templates/name1,name2but in fact today it only accepts a single name or pattern. This commit
also adds support for multiple names/patterns as the docs claim.