Skip to content

Commit

Permalink
Add docs on repository reliability and backups (elastic#68740)
Browse files Browse the repository at this point in the history
This commit spells out how important repository reliability is to
searchable snapshots, and also documents a procedure for taking a backup
of a snapshot repository.

Relates elastic#54944
  • Loading branch information
DaveCTurner authored Feb 9, 2021
1 parent 0349266 commit 71763ca
Show file tree
Hide file tree
Showing 2 changed files with 40 additions and 0 deletions.
16 changes: 16 additions & 0 deletions docs/reference/searchable-snapshots/index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,22 @@ which identifies its original index snapshot. It does not contain any data from
the original index. The restore of a backup will fail to restore any
{search-snap} indices whose original index snapshot is unavailable.

[discrete]
[[searchable-snapshots-reliability]]
=== Reliability of {search-snaps}

The sole copy of the data in a {search-snap} index is the underlying snapshot,
stored in the repository. If the repository fails or corrupts the contents of
the snapshot then the data is lost. Although {es} may have made copies of the
data onto local storage, these copies may be incomplete and cannot be used to
recover any data after a repository failure. You must make sure that your
repository is reliable and protects against corruption of your data while it is
at rest in the repository.

The blob storage offered by all major public cloud providers typically offers
very good protection against data loss or corruption. If you manage your own
repository storage then you are responsible for its reliability.

[discrete]
[[searchable-snapshots-shared-cache]]
=== Shared snapshot cache
Expand Down
24 changes: 24 additions & 0 deletions docs/reference/snapshot-restore/register-repository.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -294,3 +294,27 @@ subsequently cleaned up.
Please note that most of the cleanup operations executed by this endpoint are automatically executed when deleting any snapshot from a
repository. If you regularly delete snapshots, you will in most cases not get any or only minor space savings from using this functionality
and should lower your frequency of invoking it accordingly.

[discrete]
[[snapshots-repository-backup]]
=== Repository backup

You may wish to make an independent backup of your repository, for instance so
that you have an archive copy of its contents that you can use to recreate the
repository in its current state at a later date.

You must ensure that {es} does not write to the repository while you are taking
the backup of its contents. You can do this by unregistering it, or registering
it with `readonly: true`, on all your clusters. If {es} writes any data to the
repository during the backup then the contents of the backup may not be
consistent and it may not be possible to recover any data from it in future.

Alternatively, if your repository supports it, you may take an atomic snapshot
of the underlying filesystem and then take a backup of this filesystem
snapshot. It is very important that the filesystem snapshot is taken
atomically.

WARNING: You cannot use filesystem snapshots of individual nodes as a backup
mechanism. You must use the {es} snapshot and restore feature to copy the
cluster contents to a separate repository. Then, if desired, you can take a
filesystem snapshot of this repository.

0 comments on commit 71763ca

Please sign in to comment.