From 71763ca37f3e5b29569b21f04b4a3e7efb13830b Mon Sep 17 00:00:00 2001 From: David Turner Date: Tue, 9 Feb 2021 14:57:06 +0000 Subject: [PATCH] Add docs on repository reliability and backups (#68740) This commit spells out how important repository reliability is to searchable snapshots, and also documents a procedure for taking a backup of a snapshot repository. Relates #54944 --- .../searchable-snapshots/index.asciidoc | 16 +++++++++++++ .../register-repository.asciidoc | 24 +++++++++++++++++++ 2 files changed, 40 insertions(+) diff --git a/docs/reference/searchable-snapshots/index.asciidoc b/docs/reference/searchable-snapshots/index.asciidoc index 2d5b5bdbc8452..e638d92136101 100644 --- a/docs/reference/searchable-snapshots/index.asciidoc +++ b/docs/reference/searchable-snapshots/index.asciidoc @@ -119,6 +119,22 @@ which identifies its original index snapshot. It does not contain any data from the original index. The restore of a backup will fail to restore any {search-snap} indices whose original index snapshot is unavailable. +[discrete] +[[searchable-snapshots-reliability]] +=== Reliability of {search-snaps} + +The sole copy of the data in a {search-snap} index is the underlying snapshot, +stored in the repository. If the repository fails or corrupts the contents of +the snapshot then the data is lost. Although {es} may have made copies of the +data onto local storage, these copies may be incomplete and cannot be used to +recover any data after a repository failure. You must make sure that your +repository is reliable and protects against corruption of your data while it is +at rest in the repository. + +The blob storage offered by all major public cloud providers typically offers +very good protection against data loss or corruption. If you manage your own +repository storage then you are responsible for its reliability. + [discrete] [[searchable-snapshots-shared-cache]] === Shared snapshot cache diff --git a/docs/reference/snapshot-restore/register-repository.asciidoc b/docs/reference/snapshot-restore/register-repository.asciidoc index d3924a0940d34..9a242ef997958 100644 --- a/docs/reference/snapshot-restore/register-repository.asciidoc +++ b/docs/reference/snapshot-restore/register-repository.asciidoc @@ -294,3 +294,27 @@ subsequently cleaned up. Please note that most of the cleanup operations executed by this endpoint are automatically executed when deleting any snapshot from a repository. If you regularly delete snapshots, you will in most cases not get any or only minor space savings from using this functionality and should lower your frequency of invoking it accordingly. + +[discrete] +[[snapshots-repository-backup]] +=== Repository backup + +You may wish to make an independent backup of your repository, for instance so +that you have an archive copy of its contents that you can use to recreate the +repository in its current state at a later date. + +You must ensure that {es} does not write to the repository while you are taking +the backup of its contents. You can do this by unregistering it, or registering +it with `readonly: true`, on all your clusters. If {es} writes any data to the +repository during the backup then the contents of the backup may not be +consistent and it may not be possible to recover any data from it in future. + +Alternatively, if your repository supports it, you may take an atomic snapshot +of the underlying filesystem and then take a backup of this filesystem +snapshot. It is very important that the filesystem snapshot is taken +atomically. + +WARNING: You cannot use filesystem snapshots of individual nodes as a backup +mechanism. You must use the {es} snapshot and restore feature to copy the +cluster contents to a separate repository. Then, if desired, you can take a +filesystem snapshot of this repository.