Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable Deployment and Replicaset enrichment by default #62

Merged
merged 2 commits into from
Oct 11, 2023
Merged

Disable Deployment and Replicaset enrichment by default #62

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
295cd10
Disable Deployment and Replicaset enrichment by default
gizas Oct 4, 2023
3b2630a
Fixing tests in order to pass validation with deployment and cronjobs…
gizas Oct 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions kubernetes/metadata/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ func GetDefaultResourceMetadataConfig() *AddResourceMetadataConfig {
return &AddResourceMetadataConfig{
Node: metaCfg,
Namespace: metaCfg,
Deployment: true,
CronJob: true,
Deployment: false,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks sane to me. This would need a changelog entry in the proper section as a breaking change.

Since this is a breaking change (with valid reasoning) please wait for validation from @tommyers-elastic and @mlunadia before merging.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 from me

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please make it at least configurable? We use kubernetes.deployment do distinguish between pod types while configuring elastic-agent. Hint based discovery is still limited so we rely on condition based.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bartoszcisek you can still enable it in your manifest

You need to enable the kubernetes provider

data:
  agent.yml: |-
    ...
    providers.kubernetes:
      add_resource_metadata:
        deployment: true
        cronjob: true

I am testing with 8.11.0-SNAPSHOT
I exec in my elastic agent pod:

elastic-agent inspect -v --variables --variables-wait 2s
....
- data_stream:
    namespace: default
  id: filestream-container-logs-d105fc5a-36b3-4d5a-b97a-288d141781de-kubernetes-7107f2fe-3ae6-49fb-9aa6-5de2a5e23259.local-path-provisioner
  meta:
    package:
      name: kubernetes
      version: 1.46.0
  name: kubernetes-1
  original_id: filestream-container-logs-d105fc5a-36b3-4d5a-b97a-288d141781de
  - add_fields:
      fields:
        container:
          name: local-path-provisioner
        deployment:   < -------This is present
          name: local-path-provisioner

And if you need also your metrics to be enriched, you can change the state_pod and state_container datasets with the add_resource_metadata block

streams:
          - id: >-
              kubernetes/metrics-kubernetes.state_container-d105fc5a-36b3-4d5a-b97a-288d141781de
            data_stream:
              type: metrics
              dataset: kubernetes.state_container
           ...
            add_resource_metadata:
              deployment: true
              cronjob: true

         - id: >-
              kubernetes/metrics-kubernetes.state_pod-d105fc5a-36b3-4d5a-b97a-288d141781de
            data_stream:
              type: metrics
              dataset: kubernetes.state_pod
           ...
            add_metadata: true
            add_resource_metadata:
              deployment: true
              cronjob: true

There is also the option to use ingest pipelines as described in this new doc https://www.elastic.co/guide/en/fleet/master/ingest-pipeline-kubernetes.html

CronJob: false,
}
}
14 changes: 8 additions & 6 deletions kubernetes/metadata/pod_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -345,9 +345,10 @@ func TestPod_Generate(t *testing.T) {
"ip": "127.0.0.5",
},
"namespace": defaultNs,
"deployment": mapstr.M{
"name": "nginx-deployment",
},
// We comment below block because add_resource_metadata.deployment: false by default
// "deployment": mapstr.M{
// "name": "nginx-deployment",
// },
"replicaset": mapstr.M{
"name": "nginx-rs",
},
Expand Down Expand Up @@ -403,9 +404,10 @@ func TestPod_Generate(t *testing.T) {
"ip": "127.0.0.5",
},
"namespace": defaultNs,
"deployment": mapstr.M{
"name": "nginx-deployment",
},
// We comment below block because add_resource_metadata.deployment: false by default
// "deployment": mapstr.M{
// "name": "nginx-deployment",
// },
"replicaset": mapstr.M{
"name": "nginx-rs",
},
Expand Down
Loading