Dangling indices documentation #58751
Conversation
Further docs fixes. Update gateways docs to cover the new `gateway.gateway.auto_import_dangling_indices` setting, and rework text since the default behaviour is now not to import automatically.
pugnascotia
added
>enhancement
>docs
|
Pinging @elastic/es-docs (>docs) |
|
Pinging @elastic/es-distributed (:Distributed/Distributed) |
|
The test failures look related, holding off on a proper review for now. |
|
@elasticmachine run elasticsearch-ci/docs |
|
@DaveCTurner CI is now green. |
|
We discussed this in today's distrib team sync and agreed to default auto-importing to A common (unsafe) source of dangling indices is using |
|
Yes, on reflection I should have raised a separate PR - curse my laziness 😅 |
pugnascotia
changed the title
|
I split the server changes into #58898. |
Co-authored-by: David Turner <[email protected]>
There was a problem hiding this comment.
LGTM; I believe @lockewritesdocs is also taking a look too.
There was a problem hiding this comment.
Overall, LGTM. I left some editorial comments that you can choose to incorporate.
One suggestion: consider adding an Examples section to the Import dangling index API page. This addition provides users with a more comprehensive example and shows the response structure.
| When a node joins the cluster, if it finds any shards stored in its local data | ||
| directory that do not already exist in the cluster, it will consider those | ||
| shards to be "dangling". You can automatically import dangling indices | ||
| into the cluster using the following setting, however this is not safe, and |
There was a problem hiding this comment.
I'm a bit confused by the "this is not safe". Without further qualification, this would leave a reader puzzled as well.
| to `false`. | ||
|
|
||
| WARNING: The auto-import functionality was intended as a best effort to help users | ||
| who lose all master nodes. For example, if a new master node were to be |
There was a problem hiding this comment.
I'm wondering if we should add that that scenario (losing all master nodes) should be avoided at all cost, as it puts the cluster's metadata and data at risk. Also worth pointing out above (when talking about "this is not safe") that importing dangling indices does not provide any guarantees as to whether the data that's imported truly represents the latest state of the data as it was when the index was still part of the cluster.
There was a problem hiding this comment.
@ywelsch I've reworked the text to and address your points.
Co-authored-by: Adam Locke <[email protected]>
| its use is strongly discouraged in favour of the | ||
| <<dangling-indices-api,dedicated API>. | ||
|
|
||
| WARNING: Losing all master nodes is a situation that should be avoided at |
There was a problem hiding this comment.
@lockewritesdocs Is it overkill to have two WARNING: sections? I felt that they were making separate points, and all the content here ought to be scary.
Part of #48366. Add documentation for the dangling indices API added in #58176. Co-authored-by: David Turner <[email protected]> Co-authored-by: Adam Locke <[email protected]>
|
Backported to |
Part of #48366. This PR adds documentation for the dangling indices API added in #58176.