Adding shard count to node stats api #75395
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
masseyke
added
>enhancement
:Data Management/Stats
elasticmachine
added
the
Team:Data Management
|
Pinging @elastic/es-core-features (Team:Core/Features) |
|
💚 CLA has been signed |
|
Initial PR comments are on an old PR that I closed out -- #75395 |
dakrone
requested changes
Jul 15, 2021
server/src/main/java/org/elasticsearch/index/shard/SimpleShardStats.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/index/shard/SimpleShardStats.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/index/shard/SimpleShardStats.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/index/shard/SimpleShardStats.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/action/admin/indices/stats/CommonStats.java
Show resolved
Hide resolved
|
Pinging @elastic/clients-team (Team:Clients) |
sethmlarson
approved these changes
Jul 16, 2021
Added a shards section to each node returned by the node stats api. Currently it only contains a count of shards
…stic#75232) SimpleFeatureFactory has no external dependencies now.
…elastic#75272) Lucene does not match points that are encoded to Integer.MAX_VALUE for intersects.
…lastic#75179) Some tests are fixed after typed api is available with compatible api. Also cat api returning text fixed some tests relates elastic#51816
This commit adds logging to the in memory LDAP server in LdapTestCase that indicates when the server starts and stops and the addresses on which it is listening. Relates: elastic#75176
…elastic#74984) Use point in time API for every checkpoint in transform. Using point in time reduces pressure on the source indexes, e.g. less refreshes. In case, pit isn't supported (e.g. when searching remote clusters) it falls back to ordinary search requests as before. closes elastic#73481
Adds a field usage API that reports shard-level statistics about which Lucene fields have been accessed, and which parts of the Lucene data structures have been accessed. Field usage statistics are automatically captured when queries are runnning on a cluster. A shard-level search request that accesses a given field, even if multiple times during that request, is counted as a single use.
Adapts BWC conditions after backporting elastic#74227 to 7.x branch (targeting 7.15).
Changes the way we fetch values from source in geo fields so we can format all data in one go.
Since the index may have more than one shard, we can't always predict how many documents will fall in each slice. This PR simply removes the checks, since we already test the slicing logic extensively in an integration test. The REST test is now just a sanity check for the API. Fixes elastic#75212.
…et-new indices (elastic#74803) This PR implements a system index access level that includes all system indices, except net-new system indices. This new access level is leveraged to fix the error in the Get Alias API described in elastic#74687.
This change adds additional assertion in GeoIpDownloaderIT.testInvalidTimestamp which makes sure that validity checks work both ways (so going out of validity and back) and it should fix race in cleanUp method leading to occasional failures. Closes elastic#75221 Closes elastic#74358
…tic#75308) Elasticsearch's server sometimes has to do things differently in order to handle searchable snapshots shards. In such cases it relies on index settings and hard coded values to know if a shard is a searchable snapshot one. This commit adds a new SearchableSnapshotsSettings class in server that provides methods to check if an index is a searchable snapshot index. This class also contains the names of some index settings related to searchable snapshots that are required by the server.
…c#75187) Warning related transformations missed the possibility to apply per single test only. Also a warning changed in elastic#67158 for indices.close so this PR also applies the transformation for 7.x test relates elastic#51816
This commit upgrades the existing SSPL licensed "ssl-config" library to include additional features that are supported by the X-Pack SSL library. This commit does not make any changes to X-Pack to use these new features - it introduces them in preparation for their future use by X-Pack. The reindex module is updated to reflect API changes in ssl-config
…snapshots (elastic#74977) Today there is no relationship between the lifecycle of a snapshot mounted as an index and the lifecycle of index itself. This lack of relationship makes it possible to delete a snapshot in a repository while the mounted index still exists, causing the shards to fail in the future. On the other hand creating a snapshot that contains a single index to later mount it as a searchable snapshot index becomes more and more natural for users and ILM; but it comes with the risk to forget to delete the snapshot when the searchable snapshot index is deleted from the cluster, "leaking" snapshots in repositories. We'd like to improve the situation and provide a way to automatically delete the snapshot when the mounted index is deleted. To opt in for this behaviour, a user has to enable a specific index setting when mounting the snapshot (the proposed name for the setting is index.store.snapshot.delete_searchable_snapshot). Elasticsearch then verifies that the snapshot to mount contains only 1 snapshotted index and that the snapshot is not used by another mounted index with a different value for the opt in setting, and mounts the snapshot with the new private index setting. This is the part implemented in this commit. In follow-up pull requests this index setting will be used when the last mounted index is deleted and removed from the cluster state in order to add the searchable snapshot id to a list of snapshots to delete in the repository metadata. Snapshots that are marked as "to delete" will not be able to be restored or cloned, and Elasticsearch will take care of deleting the snapshot as soon as possible. Then ILM will be changed to use this setting when mounting a snapshot as a cold or frozen index and delete_searchable_snapshot option in ILM will be removed. Finally, deleting a snapshot that is still used by mounted indices will be prevented.
…5370) Tested class is `NodeLoadDetector` thus the test class name should be `NodeLoadDetectorTests`.
This change introduces a CLI tool that can be used to create enrollment tokens. It doesn't require credentials, but simply write access to the local filesystem of a node. It uses an auto-generated user in the file-realm with superuser role. For this purpose, this change also introduces a base class for a CLI tool that can be used by any CLI tool needs to perform actions against an ES node as a superuser without requiring credentials from the user. It is worth noting that this doesn't change our existing thread model, because already an actor with write access to the fs of an ES node, can become superuser (again, by adding a superuser to the file realm, albeit manually). Co-authored-by: Adam Locke <[email protected]>
* [DOCS] Clarify usage of the enroll Kibana API I had a discussion with @bytebilly about whether users will call the enroll Kibana API on their own, or if it's only intended for internal use. It seems that only Kibana uses this API to verify the enrollment token from Elasticsearch, so I'm adding a note to indicate this usage, along with some explanatory information about when Kibana calls this API. * Incorporate changes from review feedback
…#75381) * Remove aliases from disk usage test * Remove aliases from field usage test
This adds a null check, as `getShardStats(index)` may return null. Co-authored-by: Elastic Machine <[email protected]>
Fixes a broken link in the `documentation.url` for the field usage stats API and enroll Kibana API. These broken links caused the build for the JS client docs to fail.
Increase the client timeout for CCS tests that are occasionally timing out.
Acting on a request from the Iron Bank folks.
masseyke
force-pushed
the
feature/shard-count-in-node-stats
branch
from
774cd52 to
41b3e18
Compare
|
I closed this because I realized I had not configured my email address properly in my git config. About to open a new one that I'll link. |
|
The new PR that replaces this one is #75433 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added a shards section to each node returned by the _node/stats api. Currently it only contains a count of shards