Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update kubernetes provider documentation and 8.6 release notes #2552

Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,10 @@ To automatically identify a Redis Pod and monitor it with the Redis integration,
The condition `${kubernetes.labels.app} == 'redis'` will make the {agent} look for a Pod with the label `app:redis` within the scope defined in its manifest.

For a list of provider fields that you can use in conditions, refer to <<kubernetes-provider>>.
Some examples of conditions usage are:
1. For a pod with label `app.kubernetes.io/name=ingress-nginx` the condition should be `condition: ${kubernetes.labels.app.kubernetes.io/name} == "ingress-nginx"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

2. For a pod with annotation `prometheus.io/scrape: "true"` the condition should be `${kubernetes.annotations.prometheus.io/scrape} == "true"`
3. For a pod with name `kube-scheduler-kind-control-plane` the condition should be `${kubernetes.pod.name == "kube-scheduler-kind-control-plane"`


The `redis` input defined in the {agent} manifest only specifies the`info` metricset. To learn about other available metricsets and their configuration settings, refer to the {metricbeat-ref}/metricbeat-module-redis.html[Redis module page].
Expand Down Expand Up @@ -67,7 +71,9 @@ You should now be able to see Redis data flowing in on index `metrics-redis.info

NOTE: All assets (dashboards, ingest pipelines, and so on) related to the Redis integration are not installed. You need to explicitly <<install-uninstall-integration-assets,install them through {kib}>>.

To set the target host dynamically for a targeted Pod based on its labels, use a variable in the {agent} policy to return path information from the provider:
Conditions can also be used in the `Kubernetes` package in order to set the target host dynamically for a targeted Pod based on its labels.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

by Kubernetes package are you referring to the https://docs.elastic.co/en/integrations/kubernetes ?

for standalone we usually use in documentation input instead of package, for example - here or here

can we keep it consistent?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The input can have whatever name the user wants so kubernetes input would not be accurate wording. Also in each input we have the package information inside the meta. And although not used anywhere, the name of the package is the same used in integrations.
Also I initially had written input. But I changed it after @ChrsMark comment (#2552 (comment)).
Chris are your thoughts the same as mine in this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you for clarifications, I don't have a strong opinion on it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No strong opinion either :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok as long you don't have a strong opinion I decided to use input instead of package to keep consistency because I am not sure if the user at this point is 100% familiar with packages.

This is useful for datasets that target specific pods like `kube-scheduler` or `kube-controller-manager`.
The following configuration will enable `kubernetes.scheduler` dataset only for pods which have the label `component=kube-scheduler` defined.

[source,yaml]
----
Expand All @@ -84,6 +90,13 @@ To set the target host dynamically for a targeted Pod based on its labels, use a
condition: ${kubernetes.labels.component} == 'kube-scheduler'
----

NOTE: Pods' labels and annotations can be used in autodiscover conditions. In the case of labels or annotations that include dots(`.`), they can be used in conditions exactly as
they are defined in the pods. For example `condition: ${kubernetes.labels.app.kubernetes.io/name} == 'ingress-nginx'`. This should not be confused with the dedoted (by default) labels and annotations
stored into Elasticsearch(<<kubernetes-provider>>).

WARNING: Before 8.6 release, labels used in autodiscover conditions were dedoted in case `labels.dedot` parameter was set to `true` in Kubernetes Provider
configuration (by default `true`). The same did not apply for annotations. This was fixed in 8.6 release. Refer to <<release-notes-8.6.0>>.

WARNING: In some "As a Service" Kubernetes implementations, like GKE, the control plane nodes or even the Pods running on them won’t be visible. In these cases, it won’t be possible to use scheduler metricsets, necessary for this example. Refer https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-kubernetes.html#_scheduler_and_controllermanager[scheduler and controller manager] to find more information.

Following the Redis example, if you deploy another Redis Pod with a different port, it should be detected. To check this, go, for example, to the field `service.address` under `metrics-redis.info-default`. It should be displaying two different services.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,10 @@ The available keys are:


These are the fields available within config templating. The `kubernetes.*` fields will be available on each emitted event.

NOTE: `kubernetes.labels.*` and `kubernetes.annotations.*` used in config templating are not dedoted and should not be confused with
labels and annotations added in the final Elasticsearch document and which are dedoted by default. For examples refer to <<conditions-based-autodiscover>>.

Note that not all of these fields are available by default and special configuration options
are needed in order to include them.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,21 @@ provided. For more information, refer to {agent-pull}1140[#1140].
Remove the `--pprof` argument from any scripts or commands you use.
====

[discrete]
[[breaking-1398]]
.Not dedoted Kubernetes pod labels to be used for autodiscover conditions
[%collapsible]
====
*Details* +
Kubernetes pod labels used in autodiscover conditions are not dedoted anymore. This means that
`.` are not replaced with `_` in labels like `app.kubernetes.io/component=controller`.
This follows the same approach as kubernetes annotations. For more information refer to <<kubernetes-provider>>.

*Impact* +
Any template used for standalone elastic agent or installed integration that makes use
of dedoted kubernetes labels inside conditions has to be updated.
====

[discrete]
[[known-issues-8.6.0]]
=== Known issues
Expand Down