Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow users to select the cloud metadata providers #13812

Merged
merged 4 commits into from
Sep 26, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Fixed a crash under Windows when fetching processes information. {pull}12833[12833]
- Update to Golang 1.12.7. {pull}12931[12931]
- Remove `in_cluster` configuration parameter for Kuberentes, now in-cluster configuration is used only if no other kubeconfig is specified {pull}13051[13051]
- Disable Alibaba Cloud and Tencent Cloud metadata providers by default. {pull}13812[12812]

*Auditbeat*

Expand Down Expand Up @@ -257,6 +258,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Add autodetection mode for add_docker_metadata and enable it by default in included configuration files{pull}13374[13374]
- Added `monitoring.cluster_uuid` setting to associate Beat data with specified ES cluster in Stack Monitoring UI. {pull}13182[13182]
- Add autodetection mode for add_kubernetes_metadata and enable it by default in included configuration files. {pull}13473[13473]
- Add `providers` setting to `add_cloud_metadata` processor. {pull}13812[13812]

*Auditbeat*

Expand Down
25 changes: 21 additions & 4 deletions libbeat/docs/processors-using.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -505,8 +505,8 @@ not:
=== Add cloud metadata

The `add_cloud_metadata` processor enriches each event with instance metadata
from the machine's hosting provider. At startup it will detect the hosting
provider and cache the instance metadata.
from the machine's hosting provider. At startup it will query a list of hosting
providers and cache the instance metadata.

The following cloud providers are supported:

Expand All @@ -518,6 +518,10 @@ The following cloud providers are supported:
- Azure Virtual Machine
- Openstack Nova

The Alibaba Cloud and Tencent cloud providers are disabled by default, because
they require to access a remote host. The `providers` setting allows users to
select a list of default providers to query.

The simple configuration below enables the processor.

[source,yaml]
Expand All @@ -526,7 +530,7 @@ processors:
- add_cloud_metadata: ~
-------------------------------------------------------------------------------

The `add_cloud_metadata` processor has two optional configuration settings.
The `add_cloud_metadata` processor has three optional configuration settings.
The first one is `timeout` which specifies the maximum amount of time to wait
for a successful response when detecting the hosting provider. The default
timeout value is `3s`.
Expand All @@ -535,7 +539,20 @@ If a timeout occurs then no instance metadata will be added to the events. This
makes it possible to enable this processor for all your deployments (in the
cloud or on-premise).

The second optional configuration setting is `overwrite`. When `overwrite` is
The second optional setting is `providers`. The `providers` settings accepts a
list of cloud provider names to be used. If `providers` is not configured, then
all providers that do not access a remote endpoint are enabled by default.

List of names the `providers` setting supports:
- "alibaba", or "ecs" for the Alibaba Cloud provider (disabled by default).
- "azure" for Azure Virtual Machine (enabled by default).
- "digitalocean" for Digital Ocean (enabled by default).
- "aws", or "ec2" for Amazon Web Services (enabled by default).
- "gcp" for Google Copmute Enging (enabled by default).
- "openstack", or "nova" for Openstack Nova (enabled by default).
- "tencent", or "qcloud" for Tencent Cloud (disabled by default).

The third optional configuration setting is `overwrite`. When `overwrite` is
`true`, `add_cloud_metadata` overwrites existing `cloud.*` fields (`false` by
default).

Expand Down
Loading