Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document transport.compress trade-offs more clearly #108458

Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 29 additions & 24 deletions docs/reference/modules/cluster/remote-clusters-settings.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -59,35 +59,40 @@ you configure the remotes.

`cluster.remote.<cluster_alias>.transport.compress`::

Per cluster setting that enables you to configure compression for requests
to a specific remote cluster. This setting impacts only requests
sent to the remote cluster. If the inbound request is compressed,
Elasticsearch compresses the response. The setting options are `true`,
`indexing_data`, and `false`. If unset, the global `transport.compress` is
used as the fallback setting.
Per-cluster setting that enables you to configure compression for requests to
a specific remote cluster. The handling cluster will automatically compress
responses to compressed requests. The setting options are `true`,
`indexing_data`, and `false`. If unset, defaults to the behaviour specified
by the node-wide `transport.compress` setting. See the
<<transport-settings-compress,documentation for the `transport.compress`
setting>> for further information.

`cluster.remote.<cluster_alias>.transport.compression_scheme`::

Per cluster setting that enables you to configure compression scheme for
requests to a specific remote cluster. This setting impacts only requests
sent to the remote cluster. If an inbound request is compressed, {es}
compresses the response using the same compression scheme. The setting options
are `deflate` and `lz4`. If unset, the global `transport.compression_scheme`
is used as the fallback setting.
Per-cluster setting that enables you to configure the compression scheme for
requests to a specific cluster if those requests are selected to be
compressed by to the `cluster.remote.<cluster_alias>.transport.compress`
setting. The handling cluster will automatically use the same compression
scheme for responses as for the corresponding requests. The setting options
are `deflate` and `lz4`. If unset, defaults to the behaviour specified by the
node-wide `transport.compression_scheme` setting. See the
<<transport-settings-compression-scheme,documentation for the
`transport.compression_scheme` setting>> for further information.


`cluster.remote.<cluster_alias>.credentials` (<<secure-settings,Secure>>, <<reloadable-secure-settings,Reloadable>>)::
[[remote-cluster-credentials-setting]]

Per cluster setting for configuring <<remote-clusters-api-key,remote clusters with the API Key based model>>.
This setting takes the encoded value of a
<<security-api-create-cross-cluster-api-key,cross-cluster API key>> and must be set
in the <<secure-settings,{es} keystore>> on each node in the cluster.
The presence (or not) of this setting determines which model a remote cluster uses.
If present, the remote cluster uses the API key based model.
Otherwise, it uses the certificate based model.
If the setting is added, removed, or updated in the <<secure-settings,{es} keystore>> and reloaded via the
<<cluster-nodes-reload-secure-settings>> API, the cluster will automatically rebuild its connection to the remote.
`cluster.remote.<cluster_alias>.credentials`::

(<<secure-settings,Secure>>, <<reloadable-secure-settings,Reloadable>>)
Per-cluster setting for configuring <<remote-clusters-api-key,remote clusters
with the API Key based model>>. This setting takes the encoded value of a
<<security-api-create-cross-cluster-api-key,cross-cluster API key>> and must
be set in the <<secure-settings,{es} keystore>> on each node in the cluster.
The presence (or not) of this setting determines which model a remote cluster
uses. If present, the remote cluster uses the API key based model. Otherwise,
it uses the certificate based model. If the setting is added, removed, or
updated in the <<secure-settings,{es} keystore>> and reloaded via the
<<cluster-nodes-reload-secure-settings>> API, the cluster will automatically
rebuild its connection to the remote.

[[remote-cluster-sniff-settings]]
==== Sniff mode remote cluster settings
Expand Down
40 changes: 33 additions & 7 deletions docs/reference/modules/transport.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -47,18 +47,44 @@ different from `transport.port`. Defaults to the port assigned via
The connect timeout for initiating a new connection (in
time setting format). Defaults to `30s`.

[[transport-settings-compress]]
`transport.compress`::
(<<static-cluster-setting,Static>>, string)
Set to `true`, `indexing_data`, or `false` to configure transport compression
between nodes. The option `true` will compress all data. The option
`indexing_data` will compress only the raw index data sent between nodes during
ingest, ccr following (excluding bootstrap), and operations based shard recovery
(excluding transferring lucene files). Defaults to `indexing_data`.
Determines which transport requests are compressed before sending them to
another node. {es} will compress transport responses if and only if the
corresponding request was compressed. See also `transport.compression_scheme`,
which specifies the compression scheme which is used. Accepts the following
values:
+
--
`false`::

No transport requests are compressed. This option uses the most network
bandwidth, but avoids the CPU overhead of compression and decompression.

`indexing_data`::

Compresses only the raw indexing data sent between nodes during ingest, CCR
following (excluding bootstrapping) and operations-based shard recovery
(excluding file-based recovery which copies the raw Lucene data). This option
is a good trade-off between network bandwidth savings and the extra CPU
required for compression and decompression. This option is the default.

`true`::

All transport requests are compressed. This option may perform better than
`indexing_data` in terms of network bandwidth, but will require the most CPU
for compression and decompression work.
--

[[transport-settings-compression-scheme]]
`transport.compression_scheme`::
(<<static-cluster-setting,Static>>, string)
Configures the compression scheme for `transport.compress`. The options are
`deflate` or `lz4`. If `lz4` is configured and the remote node has not been
Configures the compression scheme for requests which are selected for
compression by to the `transport.compress` setting. Accepts either `deflate` or
`lz4`, which offer different trade-offs between compression ratio and CPU
usage. {es} will use the same compression scheme for responses as for the
corresponding requests. If `lz4` is configured and the remote node has not been
upgraded to a version supporting `lz4`, the traffic will be sent uncompressed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LZ4 was added in V_7_14_0 which cannot communicate on non-handshake requests with current version nodes. So I don't think we need to document this fallback behavior anymore?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right good point, I did wonder whether that was obsolete.

Defaults to `lz4`.

Expand Down