Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

The way to backup the searchable snapshot #93972

Closed
kunisen opened this issue Feb 21, 2023 · 2 comments
Closed

The way to backup the searchable snapshot #93972

kunisen opened this issue Feb 21, 2023 · 2 comments
Labels
:Distributed Indexing/Distributed A catch all label for anything in the Distributed Area. Please avoid if you can. :Distributed Indexing/Store Issues around managing unopened Lucene indices. If it touches Store.java, this is a likely label. >enhancement Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination.

Comments

@kunisen
Copy link
Contributor

kunisen commented Feb 21, 2023 •
edited
Loading

Description

On ESS for the cold/frozen tier, searchable snapshot data is stored in i.e. s3 bucket, and the reliability is guaranteed by CSP. Almost no data loss risk of natural disasters, however, if there’s a human error, data can be removed easily, i.e. wrongly configured ILM, or users deleting the data by accident. They may think we have a backup like normal indices, but actually, there’s not. (Searchable snapshot is also not configurable and manageable by SLM, so if it's accidentally deleted, there's no way to find the data back.)

Logically thinking, it's a SPOF.

It would be great to have a logic to take the backup of the searchable snapshots on the current ESS, i.e. thru mirroring the buckets.

cc @romain-chanu @psanz-estc as we discussed this
@kunisen kunisen added >enhancement :Distributed Indexing/Store Issues around managing unopened Lucene indices. If it touches Store.java, this is a likely label. :Distributed Indexing/Distributed A catch all label for anything in the Distributed Area. Please avoid if you can. Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. needs:triage Requires assignment of a team area label labels Feb 21, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

@elasticsearchmachine elasticsearchmachine removed the needs:triage Requires assignment of a team area label label Feb 21, 2023
@DaveCTurner
Copy link
Contributor

The manual describes how to back up a snapshot repository which includes searchable snapshots. We recognise that there is room for improvement in this area, and #54944 tracks that work. I am closing this as a duplicate of that issue.

@kunisen kunisen mentioned this issue Feb 27, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
:Distributed Indexing/Distributed A catch all label for anything in the Distributed Area. Please avoid if you can. :Distributed Indexing/Store Issues around managing unopened Lucene indices. If it touches Store.java, this is a likely label. >enhancement Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@DaveCTurner @kunisen @elasticsearchmachine