Skip to content

Commit

Permalink
[DOCS] Identify reloadable Azure repository plugin settings (#46358)
Browse files Browse the repository at this point in the history
  • Loading branch information
@lcawl
lcawl committed Sep 5, 2019
1 parent 4e3ecf0 commit bb01934
Show file tree
Hide file tree
Showing 2 changed files with 96 additions and 50 deletions.
145 changes: 95 additions & 50 deletions docs/plugins/repository-azure.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,6 @@ bin/elasticsearch-keystore add azure.client.default.account
bin/elasticsearch-keystore add azure.client.default.key
----------------------------------------------------------------

Where `account` is the azure account name and `key` the azure secret key. Instead of an azure secret key under `key`, you can alternatively
define a shared access signatures (SAS) token under `sas_token` to use for authentication instead. When using an SAS token instead of an
account key, the SAS token must have read (r), write (w), list (l), and delete (d) permissions for the repository base path and
all its contents. These permissions need to be granted for the blob service (b) and apply to resource types service (s), container (c), and
object (o).
These settings are used by the repository's internal azure client.

Note that you can also define more than one account:

[source,sh]
Expand All @@ -36,42 +29,8 @@ bin/elasticsearch-keystore add azure.client.secondary.account
bin/elasticsearch-keystore add azure.client.secondary.sas_token
----------------------------------------------------------------

`default` is the default account name which will be used by a repository,
unless you set an explicit one in the
<<repository-azure-repository-settings, repository settings>>.

The `account`, `key`, and `sas_token` storage settings are
{ref}/secure-settings.html#reloadable-secure-settings[reloadable]. After you
reload the settings, the internal azure clients, which are used to transfer the
snapshot, will utilize the latest settings from the keystore.

NOTE: In progress snapshot/restore jobs will not be preempted by a *reload*
of the storage secure settings. They will complete using the client as it was built
when the operation started.

You can set the client side timeout to use when making any single request. It can be defined globally, per account or both.
It's not set by default which means that Elasticsearch is using the
http://azure.github.io/azure-storage-java/com/microsoft/azure/storage/RequestOptions.html#setTimeoutIntervalInMs(java.lang.Integer)[default value]
set by the azure client (known as 5 minutes).

`max_retries` can help to control the exponential backoff policy. It will fix the number of retries
in case of failures before considering the snapshot is failing. Defaults to `3` retries.
The initial backoff period is defined by Azure SDK as `30s`. Which means `30s` of wait time
before retrying after a first timeout or failure. The maximum backoff period is defined by Azure SDK as
`90s`.

`endpoint_suffix` can be used to specify Azure endpoint suffix explicitly. Defaults to `core.windows.net`.

[source,yaml]
----
azure.client.default.timeout: 10s
azure.client.default.max_retries: 7
azure.client.default.endpoint_suffix: core.chinacloudapi.cn
azure.client.secondary.timeout: 30s
----

In this example, timeout will be `10s` per try for `default` with `7` retries before failing
and endpoint suffix will be `core.chinacloudapi.cn` and `30s` per try for `secondary` with `3` retries.
For more information about these settings, see
<<repository-azure-client-settings>>.

[IMPORTANT]
.Supported Azure Storage Account types
Expand All @@ -86,18 +45,104 @@ The Azure Repository plugin works with all Standard storage accounts
https://azure.microsoft.com/en-gb/documentation/articles/storage-premium-storage[Premium Locally Redundant Storage] (`Premium_LRS`) is **not supported** as it is only usable as VM disk storage, not as general storage.
===============================================

You can register a proxy per client using the following settings:
[[repository-azure-client-settings]]
==== Client settings

[source,yaml]
The client that you use to connect to Azure has a number of settings available.
The settings have the form `azure.client.CLIENT_NAME.SETTING_NAME`. By default,
`azure` repositories use a client named `default`, but this can be modified using
the <<repository-azure-repository-settings,repository setting>> `client`.
For example:

[source,js]
----
azure.client.default.proxy.host: proxy.host
azure.client.default.proxy.port: 8888
azure.client.default.proxy.type: http
PUT _snapshot/my_backup
{
"type": "azure",
"settings": {
"client": "secondary"
}
}
----
// CONSOLE
// TEST[skip:we don't have azure setup while testing this]

Supported values for `proxy.type` are `direct` (default), `http` or `socks`.
When `proxy.type` is set to `http` or `socks`, `proxy.host` and `proxy.port` must be provided.
Most client settings can be added to the `elasticsearch.yml` configuration file.
For example:

[source,yaml]
----
azure.client.default.timeout: 10s
azure.client.default.max_retries: 7
azure.client.default.endpoint_suffix: core.chinacloudapi.cn
azure.client.secondary.timeout: 30s
----

In this example, the client side timeout is `10s` per try for the `default`
account with `7` retries before failing. The endpoint suffix is
`core.chinacloudapi.cn` and `30s` per try for the `secondary` account with `3`
retries.

The `account`, `key`, and `sas_token` storage settings are reloadable secure
settings, which you add to the {es} keystore. For more information about
creating and updating the {es} keystore, see
{ref}/secure-settings.html[Secure settings]. After you reload the settings, the
internal Azure clients, which are used to transfer the snapshot, utilize the
latest settings from the keystore.

NOTE: In progress snapshot or restore jobs will not be preempted by a *reload*
of the storage secure settings. They will complete using the client as it was
built when the operation started.

The following list contains the available client settings. Those that must be
stored in the keystore are marked as "secure"; the other settings belong in the
`elasticsearch.yml` file.

`account` ({ref}/secure-settings.html[Secure], {ref}/secure-settings.html#reloadable-secure-settings[reloadable])::
The Azure account name, which is used by the repository's internal Azure client.

`endpoint_suffix`::
The Azure endpoint suffix to connect to. The default value is
`core.windows.net`.

`key` ({ref}/secure-settings.html[Secure], {ref}/secure-settings.html#reloadable-secure-settings[reloadable])::
The Azure secret key, which is used by the repository's internal Azure client. Alternatively, use `sas_token`.

`max_retries`::
The number of retries to use when an Azure request fails. This setting helps
control the exponential backoff policy. It specifies the number of retries
that must occur before the snapshot fails. The default value is `3`. The
initial backoff period is defined by Azure SDK as `30s`. Thus there is `30s`
of wait time before retrying after a first timeout or failure. The maximum
backoff period is defined by Azure SDK as `90s`.

`proxy.host`::
The host name of a proxy to connect to Azure through. For example: `azure.client.default.proxy.host: proxy.host`.

`proxy.port`::
The port of a proxy to connect to Azure through. For example, `azure.client.default.proxy.port: 8888`.

`proxy.type`::
Register a proxy type for the client. Supported values are `direct`, `http`,
and `socks`. For example: `azure.client.default.proxy.type: http`. When
`proxy.type` is set to `http` or `socks`, `proxy.host` and `proxy.port` must
also be provided. The default value is `direct`.

`sas_token` ({ref}/secure-settings.html[Secure], {ref}/secure-settings.html#reloadable-secure-settings[reloadable])::
A shared access signatures (SAS) token, which the repository's internal Azure
client uses for authentication. The SAS token must have read (r), write (w),
list (l), and delete (d) permissions for the repository base path and all its
contents. These permissions must be granted for the blob service (b) and apply
to resource types service (s), container (c), and object (o). Alternatively,
use `key`.

`timeout`::
The client side timeout for any single request to Azure. The value should
specify the time unit. For example, a value of `5s` specifies a 5 second
timeout. There is no default value, which means that {es} uses the
http://azure.github.io/azure-storage-java/com/microsoft/azure/storage/RequestOptions.html#setTimeoutIntervalInMs(java.lang.Integer)[default value]
set by the Azure client (known as 5 minutes). This setting can be defined
globally, per account, or both.

[[repository-azure-repository-settings]]
==== Repository settings
Expand Down
1 change: 1 addition & 0 deletions docs/reference/setup/secure-settings.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,7 @@ of reloading after each modification.

There are reloadable secure settings for:

* {plugins}/repository-azure-client-settings.html[The Azure repository plugin]
* {plugins}/discovery-ec2-usage.html#_configuring_ec2_discovery[The EC2 discovery plugin]
* {plugins}/repository-gcs-client.html[The GCS repository plugin]
* {plugins}/repository-s3-client.html[The S3 repository plugin]

0 comments on commit bb01934

Please sign in to comment.