Expose retention leases in shard stats #37991
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit exposes retention leases via shard-level stats.
jasontedor
added
>enhancement
v7.0.0
:Distributed Indexing/Distributed
jasontedor
requested review from
martijnvg,
bleskes,
gwbrown,
ywelsch and
dnhatn
|
Pinging @elastic/es-core-features |
|
Pinging @elastic/es-distributed |
24 tasks
|
I will add some tests for this later, but I want to get this up for early feedback and so that ILM can provide feedback and get unblocked. |
dnhatn
approved these changes
Jan 29, 2019
|
@elasticmachine run elasticsearch-ci/1 |
|
Thank you, @jasontedor! I'd like @martijnvg to also confirm as he's more familiar with CCR, but it looks like this (in combination with the SeqNoStats) does get us the information we need in order to determine if we can safely perform operations on a leader index. |
3 tasks
|
@jasontedor So in ILM we would do something like this to figure out whether an index is a ccr leader index? boolean waitWithNextStep = retentionLeaseStatsForIndex.getLeases()
.stream().
.filter(lease -> "ccr".equals(lease.source()))
.findAny() |
|
@martijnvg Yes, so we are missing a getter on the stats object. |
|
@jasontedor Cool. This approach looks good to me. |
* master: (29 commits) Fix limit on retaining sequence number (elastic#37992) Docs test fix, wait for shards active. Revert "Revert "Documented default values for index follow request parameters. (elastic#37917)"" Revert "Documented default values for index follow request parameters. (elastic#37917)" Ensure date parsing BWC compatibility (elastic#37929) SQL: Skip the nested and object field types in case of an ODBC request (elastic#37948) Use mappings to format doc-value fields by default. (elastic#30831) Give precedence to index creation when mixing typed templates with typeless index creation and vice-versa. (elastic#37871) Add classifier to tar.gz in docker compose (elastic#38011) Documented default values for index follow request parameters. (elastic#37917) Fix fetch source option in expand search phase (elastic#37908) Restore a noop _all metadata field for 6x indices (elastic#37808) Added ccr to xpack usage infrastructure (elastic#37256) Fix exit code for Security CLI tools (elastic#37956) Streamline S3 Repository- and Client-Settings (elastic#37393) Add version 6.6.1 (elastic#37975) Ensure task metadata not null in follow test (elastic#37993) Docs fix - missing callout Types removal - deprecate include_type_name with index templates (elastic#37484) Handle completion suggestion without contexts ...
ywelsch
suggested changes
Jan 30, 2019
server/src/main/java/org/elasticsearch/index/seqno/RetentionLeaseStats.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/action/admin/indices/stats/ShardStats.java
Show resolved
Hide resolved
|
@ywelsch Thanks for the feedback; I have responded to it. |
martijnvg
approved these changes
Jan 30, 2019
server/src/test/java/org/elasticsearch/index/seqno/RetentionLeaseStatsTests.java
Outdated
Show resolved
Hide resolved
server/src/test/java/org/elasticsearch/index/seqno/RetentionLeaseIT.java
Outdated
Show resolved
Hide resolved
ywelsch
approved these changes
Jan 30, 2019
server/src/main/java/org/elasticsearch/action/admin/indices/stats/ShardStats.java
Show resolved
Hide resolved
jasontedor
added a commit
to dnhatn/elasticsearch
that referenced
this pull request
Jan 30, 2019
* master: Expose retention leases in shard stats (elastic#37991) Make primary terms fields private in index shard (elastic#38036) ML: Add reason field in JobTaskState (elastic#38029) Log flush_stats and commit_stats in testMaybeFlush HLRC: Fix strict setting exception handling (elastic#37247) Test: Enable strict deprecation on all tests (elastic#36558) Removes typed calls from YAML REST tests (elastic#37611) Switch default time format for ingest from Joda to Java for v7 (elastic#37934) Remove deprecated Plugin#onModule extension points (elastic#37866) Geo: Fix Empty Geometry Collection Handling (elastic#37978)
jasontedor
added a commit
to jasontedor/elasticsearch
that referenced
this pull request
Jan 30, 2019
* master: Remove types from watcher docs (elastic#38002) Add test coverage for Painless general casting of boolean and Boolean (elastic#37780) Fixed test bug, lastFollowTime is null if there are no follower indices. Add ECS schema for user-agent ingest processor (elastic#37727) (elastic#37984) Extract TransportRequestDeduplication from ShardStateAction (elastic#37870) Expose retention leases in shard stats (elastic#37991)
jasontedor
added a commit
that referenced
this pull request
Jan 30, 2019
This commit exposes retention leases via shard-level stats.
danielmitterdorfer
removed
the
:Data Management/ILM+SLM
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit exposes retention leases via shard-level stats.
Relates #37165