[CONTP-375] Expose k8s resource labels as tags to configure tagger

[gabedos]

What this PR does / why we need it:

CONTP-375 Exposing kubernetesResourcesAnnotationsAsTags and kubernetesResourcesLabelsAsTags options in a generic format. We recently added support for these fields in PR, however, it's not exposed in the public helm chart.

Special notes for your reviewer:

QA'd locally via helm install datadog-agent -f ~/.../values/linux.yaml ~/Desktop/Datadog/helm-charts/charts/datadogfollowing steps for parent issue

datadog:
  logLevel: DEBUG
  kubelet:
    tlsVerify: false

  logs:
    enabled: true
    containerCollectAll: true

  clusterTagger:
    # datadog.clusterTagger.collectKubernetesTags -- Enables Kubernetes resources tags collection.
    collectKubernetesTags: true

  podLabelsAsTags:
    stale-label: stale-label
  podAnnotationsAsTags:
    stale-annotation: stale-annotation

  namespaceLabelsAsTags:
    stale-label: stale-label

  namespaceAnnotationsAsTags:
    stale-annotation: stale-annotation-should-be-overridden

  nodeLabelsAsTags:
    kubernetes.io/arch: should-not-be-used-should-be-overridden

  # NEW
  kubernetesResourcesLabelsAsTags:
    deployments.apps:
      x-team: team-from-label
    pods:
      x-ref: reference
    namespaces:
      kubernetes.io/metadata.name: name-as-tag
    nodes:
      kubernetes.io/os: os-as-tag
      kubernetes.io/arch: arch-as-tag

  # NEW
  kubernetesResourcesAnnotationsAsTags:
    deployments.apps:
      x-team: team-from-annotation
    pods:
      x-ann: annotation-reference
    namespaces:
      stale-annotation: annotation-as-tag

agents:
  enabled: true
  replicas: 1
  image:
    tag: 7.58.0-rc.1
    doNotCheckTag: true

clusterAgent:
  enabled: true
  replicas: 1
  image:
    tag: 7.58.0-rc.1
    doNotCheckTag: true

Checklist

[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

• Chart Version bumped
• Documentation has been updated with helm-docs (run: .github/helm-docs.sh)
• CHANGELOG.md has been updated
• Variables are documented in the README.md

[gabedos]

/merge

[dd-devflow]

❌ MergeQueue

You are not allowed to use the merge queue towards main.

If you need support, contact us on Slack #devflow with those details!

[adel121]

Hey @gabedos

Thanks for the PR.

I think we still need to add the necessary rbacs for the feature to work.
A similar thing is done in the DD operator here.

For more details:

This feature flags requests tagging metrics based on metadata (labels and annotations) of kubernetes resources. Each resource is identified by the api group name and the resource type. The agent needs to know the labels and annotations of the requested resources in order to apply the tagging accordingly. For this to work, the cluster agent needs to have the permission to watch these resources. These permissions are controlled/granted thanks to cluster roles and service accounts. If the DCA doesn't have the correct rbacs, the feature will fail.

If you tested your changes and it worked, it might be because you tested it with resources that the agent has permission to watch by default with the default helm installation.