Only request wildcard expansion for hidden indices if supported #20938
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ycombinator
added
bug
module
Metricbeat
|
Pinging @elastic/integrations-services (Team:Services) |
|
Pinging @elastic/stack-monitoring (Stack monitoring) |
botelastic
bot
added
needs_team
ycombinator
force-pushed
the
indices-stats-hidden-bc
branch
from
51d0743
to
b69c255
Compare
ycombinator
added
v7.9.2
[zube]: In Review
needs_backport
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
|
jsoriano
approved these changes
Sep 2, 2020
| var BulkStatsAvailableVersion = common.MustNewVersion("8.0.0") | ||
| //ExpandWildcardsHiddenAvailableVersion is the version since when the "expand_wildcards" query parameter to | ||
| // the Indices Stats API can accept "hidden" as a value. | ||
| ExpandWildcardsHiddenAvailableVersion = common.MustNewVersion("7.9.0") |
There was a problem hiding this comment.
In #18703 an integration test was added and it was being executed against Elasticsearch 7.7.0, why did this test didn't fail?
Should we add some other test for the expand wildcards case?
There was a problem hiding this comment.
There was a problem hiding this comment.
Fixed constant in 1d6ceae.
There was a problem hiding this comment.
Added unit test cases in 08d22a5.
Comment on lines
133
to
134
| ew := u.Query().Get(expandWildcardsParam) | ||
| if !strings.HasSuffix(ew, hiddenSuffix) { |
There was a problem hiding this comment.
The path comes from a constant, do we need to do these checks to see if some suffix is already present?
There was a problem hiding this comment.
Yeah, I guess you're right. I was thinking of a different problem (#16044) but that's not applicable here. I will remove the checks.
There was a problem hiding this comment.
Removed in 854671e.
|
@jsoriano Thanks for the excellent review. I have addressed all your feedback. Please re-review when you get a chance. Thanks! |
jsoriano
approved these changes
Sep 3, 2020
|
Failing test seems related. |
ycombinator
removed
the
needs_backport
v1v
added a commit
to v1v/beats
that referenced
this pull request
Sep 3, 2020
…ne-2.0 * upstream/master: [Metricbeat][test] Disable ec2 flaky test (elastic#20959) Check if tracer is active before starting a transaction (elastic#20852) [Elastic Agent] Add support for variable replacement from providers (elastic#20839) Only request wildcard expansion for hidden indices if supported (elastic#20938) [Ingest Manager] New agent structure (symlinks) (elastic#20400) [Ingest Manager] Print a message confirming shutdown (elastic#20948) Skip flaky test on unix input (elastic#20942) [Ingest Manager] Align introspect-inspect naming in code (elastic#20952) [Filebeat][zeek] Map new x509 fields for ssl module (elastic#20927) [CI] fix regression with variable name (elastic#20930) [Autodiscover] Handle input-not-finished errors in config reload (elastic#20915) [Ingest Manager] Remove Success from fleet contract (elastic#20449)
v1v
added a commit
to v1v/beats
that referenced
this pull request
Sep 3, 2020
…-faster * upstream/master: [Metricbeat][test] Disable ec2 flaky test (elastic#20959) Check if tracer is active before starting a transaction (elastic#20852) [Elastic Agent] Add support for variable replacement from providers (elastic#20839) Only request wildcard expansion for hidden indices if supported (elastic#20938) [Ingest Manager] New agent structure (symlinks) (elastic#20400) [Ingest Manager] Print a message confirming shutdown (elastic#20948) Skip flaky test on unix input (elastic#20942) [Ingest Manager] Align introspect-inspect naming in code (elastic#20952) [Filebeat][zeek] Map new x509 fields for ssl module (elastic#20927)
ycombinator
added a commit
that referenced
this pull request
Sep 8, 2020
…s if supported (#20938) (#20962) * Only request wildcard expansion for hidden indices if supported (#20938) * Refactoring: inverting logic to make room for another case * Expand hidden indices wildcards if monitored ES supports that option * Adding CHANGELOG entry * Fixing formatting * Avoid unnecessary setting * Removing unnecessary suffix existence checks * Fixing feature version * Add test cases to unit test * Updating test # Conflicts: # metricbeat/module/elasticsearch/index/index_test.go * Fixing CHANGELOG
ycombinator
added a commit
that referenced
this pull request
Sep 8, 2020
…) (#20963) * Refactoring: inverting logic to make room for another case * Expand hidden indices wildcards if monitored ES supports that option * Adding CHANGELOG entry * Fixing formatting * Avoid unnecessary setting * Removing unnecessary suffix existence checks * Fixing feature version * Add test cases to unit test * Updating test # Conflicts: # metricbeat/module/elasticsearch/index/index_test.go
melchiormoulin
pushed a commit
to melchiormoulin/beats
that referenced
this pull request
Oct 14, 2020
…tic#20938) * Refactoring: inverting logic to make room for another case * Expand hidden indices wildcards if monitored ES supports that option * Adding CHANGELOG entry * Fixing formatting * Avoid unnecessary setting * Removing unnecessary suffix existence checks * Fixing feature version * Add test cases to unit test * Updating test
leweafan
pushed a commit
to leweafan/beats
that referenced
this pull request
Apr 28, 2023
…s if supported (elastic#20938) (elastic#20962) * Only request wildcard expansion for hidden indices if supported (elastic#20938) * Refactoring: inverting logic to make room for another case * Expand hidden indices wildcards if monitored ES supports that option * Adding CHANGELOG entry * Fixing formatting * Avoid unnecessary setting * Removing unnecessary suffix existence checks * Fixing feature version * Add test cases to unit test * Updating test # Conflicts: # metricbeat/module/elasticsearch/index/index_test.go * Fixing CHANGELOG
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR teaches the
elasticsearch/indexmetricset to check the version of the Elasticsearch cluster it's monitoring and, depending on that, ask the cluster to expand wildcards for hidden indices or not.Why is it important?
In #18703 we enhanced the
elasticsearch/indexmetricset to ask the monitored Elasticsearch cluster to expand wildcards for hidden indices. This enhancement was released in Metricbeat 7.9.0.Unfortunately this enhancement a backwards-incompatible change, as wildcard expansion for hidden indices is only supported starting Elasticsearch 7.9.0. So if a Metricbeat >= 7.90 tries to monitor an Elasticsearch cluster < 7.9.0, the
elasticsearch/indexmetricset is not able to collect metrics and throws the following error:Checklist
I have made corresponding changes to the documentationI have made corresponding change to the default configuration filesCHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.