Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOCS] Reformat clone index API docs #46762

Merged
merged 6 commits into from
Sep 25, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
132 changes: 86 additions & 46 deletions docs/reference/indices/clone-index.asciidoc
Original file line number Diff line number Diff line change
@@ -1,68 +1,82 @@
[[indices-clone-index]]
=== Clone Index
=== Clone index API
++++
<titleabbrev>Clone index</titleabbrev>
++++

The clone index API allows you to clone an existing index into a new index,
where each original primary shard is cloned into a new primary shard in
the new index.
Clones an existing index.

[float]
==== How does cloning work?

Cloning works as follows:
[source,console]
--------------------------------------------------
POST /twitter/_clone/cloned-twitter-index
--------------------------------------------------
// TEST[s/^/PUT twitter\n{"settings":{"index.number_of_shards" : 5,"blocks.write":true}}\n/]

* First, it creates a new target index with the same definition as the source
index.

* Then it hard-links segments from the source index into the target index. (If
the file system doesn't support hard-linking, then all segments are copied
into the new index, which is a much more time consuming process.)
[[clone-index-api-request]]
==== {api-request-title}

* Finally, it recovers the target index as though it were a closed index which
had just been re-opened.
`POST /<index>/_clone/<target-index>`

[float]
==== Preparing an index for cloning
`PUT /<index>/_clone/<target-index>`

Create a new index:

[source,console]
--------------------------------------------------
PUT my_source_index
{
"settings": {
"index.number_of_shards" : 5
}
}
--------------------------------------------------
[[clone-index-api-prereqs]]
==== {api-prereq-title}

In order to clone an index, the index must be marked as read-only,
and have <<cluster-health,health>> `green`.
To clone an index,
the index must be marked as read-only
and have a <<cluster-health,cluster health>> status of `green`.

This can be achieved with the following request:
For example,
the following request prevents write operations on `my_source_index`
so it can be cloned.
Metadata changes like deleting the index are still allowed.

[source,console]
--------------------------------------------------
PUT /my_source_index/_settings
{
"settings": {
"index.blocks.write": true <1>
"index.blocks.write": true
}
}
--------------------------------------------------
// TEST[continued]
// TEST[s/^/PUT my_source_index\n/]


<1> Prevents write operations to this index while still allowing metadata
changes like deleting the index.
[[clone-index-api-desc]]
==== {api-description-title}

[float]
==== Cloning an index
Use the clone index API
to clone an existing index into a new index,
where each original primary shard is cloned
into a new primary shard in the new index.

[[cloning-works]]
===== How cloning works

Cloning works as follows:

jrodewig marked this conversation as resolved.
Show resolved Hide resolved
* First, it creates a new target index with the same definition as the source
index.

* Then it hard-links segments from the source index into the target index. (If
the file system doesn't support hard-linking, then all segments are copied
into the new index, which is a much more time consuming process.)

* Finally, it recovers the target index as though it were a closed index which
had just been re-opened.

[[clone-index]]
===== Clone an index

To clone `my_source_index` into a new index called `my_target_index`, issue
the following request:

[source,console]
--------------------------------------------------
POST my_source_index/_clone/my_target_index
POST /my_source_index/_clone/my_target_index
--------------------------------------------------
// TEST[continued]

Expand All @@ -72,9 +86,9 @@ the cluster state -- it doesn't wait for the clone operation to start.
[IMPORTANT]
=====================================

Indices can only be cloned if they satisfy the following requirements:
Indices can only be cloned if they meet the following requirements:

* the target index must not exist
* The target index must not exist.

* The source index must have the same number of primary shards as the target index.

Expand All @@ -88,7 +102,7 @@ and accepts `settings` and `aliases` parameters for the target index:

[source,console]
--------------------------------------------------
POST my_source_index/_clone/my_target_index
POST /my_source_index/_clone/my_target_index
{
"settings": {
"index.number_of_shards": 5 <1>
Expand All @@ -107,10 +121,10 @@ POST my_source_index/_clone/my_target_index
NOTE: Mappings may not be specified in the `_clone` request. The mappings of
the source index will be used for the target index.

[float]
==== Monitoring the clone process
[[monitor-cloning]]
===== Monitor the cloning process

The clone process can be monitored with the <<cat-recovery,`_cat recovery`
The cloning process can be monitored with the <<cat-recovery,`_cat recovery`
API>>, or the <<cluster-health, `cluster health` API>> can be used to wait
until all primary shards have been allocated by setting the `wait_for_status`
parameter to `yellow`.
Expand All @@ -123,12 +137,38 @@ can be allocated on that node.

Once the primary shard is allocated, it moves to state `initializing`, and the
clone process begins. When the clone operation completes, the shard will
become `active`. At that point, Elasticsearch will try to allocate any
become `active`. At that point, {es} will try to allocate any
replicas and may decide to relocate the primary shard to another node.

[float]
==== Wait For Active Shards
[[clone-wait-active-shards]]
===== Wait for active shards

Because the clone operation creates a new index to clone the shards to,
the <<create-index-wait-for-active-shards,wait for active shards>> setting
on index creation applies to the clone index action as well.


[[clone-index-api-path-params]]
==== {api-path-parms-title}

`<index>`::
(Required, string)
Name of the source index to clone.

include::{docdir}/rest-api/common-parms.asciidoc[tag=target-index]


[[clone-index-api-query-params]]
==== {api-query-parms-title}

include::{docdir}/rest-api/common-parms.asciidoc[tag=wait_for_active_shards]

include::{docdir}/rest-api/common-parms.asciidoc[tag=timeoutparms]


[[clone-index-api-request-body]]
==== {api-request-body-title}

include::{docdir}/rest-api/common-parms.asciidoc[tag=target-index-aliases]

include::{docdir}/rest-api/common-parms.asciidoc[tag=target-index-settings]
2 changes: 2 additions & 0 deletions docs/reference/indices/create-index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ creating an index, you can specify the following:
--
(Optional, string) Name of the index you wish to create.

// tag::index-name-reqs[]
Index names must meet the following criteria:

- Lowercase only
Expand All @@ -43,6 +44,7 @@ Index names must meet the following criteria:
- Cannot start with `-`, `_`, `+`
- Cannot be `.` or `..`
- Cannot be longer than 255 bytes (note it is bytes, so multi-byte characters will count towards the 255 limit faster)
// end::index-name-reqs[]
--


Expand Down
31 changes: 28 additions & 3 deletions docs/reference/rest-api/common-parms.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,13 @@ tag::aliases[]
index. See <<indices-aliases>>.
end::aliases[]

tag::target-index-aliases[]
`aliases`::
(Optional, <<indices-aliases,alias object>>)
Index aliases which include the target index.
See <<indices-aliases>>.
end::target-index-aliases[]

tag::allow-no-indices[]
`allow_no_indices`::
(Optional, boolean) If `true`,
Expand Down Expand Up @@ -461,15 +468,22 @@ the segment has most likely been written to disk
but needs a <<indices-refresh,refresh>> to be searchable.
end::segment-search[]

tag::segment-size[]
Disk space used by the segment, such as `50kb`.
end::segment-size[]

tag::settings[]
`settings`::
(Optional, <<index-modules-settings,index setting object>>) Configuration
options for the index. See <<index-modules-settings>>.
end::settings[]

tag::segment-size[]
Disk space used by the segment, such as `50kb`.
end::segment-size[]
tag::target-index-settings[]
`settings`::
(Optional, <<index-modules-settings,index setting object>>)
Configuration options for the target index.
See <<index-modules-settings>>.
end::target-index-settings[]

tag::slices[]
`slices`::
Expand Down Expand Up @@ -506,6 +520,17 @@ tag::stats[]
purposes.
end::stats[]

tag::target-index[]
`<target-index>`::
+
--
(Required, string)
Name of the target index to create.

include::{docdir}/indices/create-index.asciidoc[tag=index-name-reqs]
--
end::target-index[]

tag::terminate_after[]
`terminate_after`::
(Optional, integer) The maximum number of documents to collect for each shard,
Expand Down