Add support for named ports in autodiscover (hints & templates) #19398

ChrsMark · 2020-06-25T15:37:59Z

What does this PR do?

Adds support for named ports in hints based autodiscover.
When for instance we have a target container like:

spec:
  containers:
    - name: natsqueue
      image: nats
      command: ["/nats-server"]
      args: ["-m", "8222"]
      ports:
        - name: web
          containerPort: 8222
          protocol: TCP

then we can reference the name of the port with co.elastic.metrics/hosts: '${data.host}:${data.ports.web}' in the annotations.

Why is it important?

To cover cases like #15796.

Manual Testing

Hints based Autodiscover

Deploy Metricbeat on k8s with hints autodiscover enabled:

metricbeat.autodiscover:
  providers:
    - type: kubernetes
      node: ${NODE_NAME}
      hints.enabled: true

Deploy a target Pod using the following spec:

apiVersion: v1
kind: Service
metadata:
  name: nats
  labels:
    app: nats
spec:
  clusterIP: None
  ports:
  - name: web
    port: 8222
    protocol: TCP
  selector:
    app: nats
  type: ClusterIP
---
apiVersion: v1
kind: Pod
metadata:
  name: nats
  labels:
    app: name
  annotations:
    co.elastic.metrics/module: nats
    co.elastic.metrics/metricsets: stats
    co.elastic.metrics/hosts: '${data.host}:${data.ports.web}'
spec:
  containers:
    - name: natsqueue
      image: nats
      command: ["/nats-server"]
      args: ["-m", "8222"]
      ports:
        - name: web
          containerPort: 8222
          protocol: TCP

Make sure that Nats metrics are being collected
Verify that old behaviour is not broken by deploying target with the following 2 different annotations:
co.elastic.metrics/hosts: '${data.host}:8222' and co.elastic.metrics/hosts: '${data.host}:${data.port}

Template based Autodiscover

Deploy Metricbeat on k8s with hints autodiscover enabled:

metricbeat.autodiscover:
  providers:
    - type: kubernetes
      host: "${NODE_NAME}"
      templates:
        - condition:
            contains:
              kubernetes.annotations.nats.io/scrape: "true"
          config:
            - module: nats
              period: 10s
              hosts: ["${data.host}:${data.ports.web}"]

Deploy a target Pod using the following spec:

apiVersion: v1
kind: Service
metadata:
  name: nats
  labels:
    app: nats
spec:
  clusterIP: None
  ports:
  - name: web
    port: 8222
    protocol: TCP
  selector:
    app: nats
  type: ClusterIP
---
apiVersion: v1
kind: Pod
metadata:
  name: nats
  labels:
    app: name
  annotations:
    nats.io/scrape: "true"
spec:
  containers:
    - name: natsqueue
      image: nats
      command: ["/nats-server"]
      args: ["-m", "8222"]
      ports:
        - name: web
          containerPort: 8222
          protocol: TCP

Make sure that Nats metrics are being collected
Verify that old behaviour is not broken by deploying target with the following 2 different annotations:
co.elastic.metrics/hosts: '${data.host}:8222' and co.elastic.metrics/hosts: '${data.host}:${data.port}

Closes #15796

Signed-off-by: chrismark <[email protected]>

elasticmachine · 2020-06-25T15:51:01Z

💔 Tests Failed

Expand to view the summary

Build stats

Build Cause: [Pull request #19398 updated]
Start Time: 2020-07-07T07:25:15.544+0000
Duration: 79 min 18 sec

Test stats 🧪

Test	Results
Failed	2
Passed	9514
Skipped	1577
Total	11093

Test errors

Expand to view the tests failures

Name: Build and Test / Metricbeat OSS Unit tests / test_process – test_system.Test
- Age: 1
- Duration: 1.592
- Error Details: False is not true : fd not found in any process events
Name: Build and Test / Metricbeat x-pack / Metricbeat x-pack / TestFetch – stats
- Age: 1
- Duration: 0
- Error Details: Failed

Steps errors

Expand to view the steps failures

Name: Mage build unitTest
- Description: mage build unitTest
- Duration: 9 min 10 sec
- Start Time: 2020-07-07T07:49:39.703+0000
- log
Name: Report to Codecov
- Description: curl -sSLo codecov https://codecov.io/bash for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat do FILE="${i}/build/coverage/full.cov" if [ -f "${FILE}" ]; then bash codecov -f "${FILE}" fi done
- Duration: 2 min 22 sec
- Start Time: 2020-07-07T07:58:52.630+0000
- log
Name: Mage build test
- Description: mage build test
- Duration: 37 min 9 sec
- Start Time: 2020-07-07T07:51:50.239+0000
- log
Name: Report to Codecov
- Description: curl -sSLo codecov https://codecov.io/bash for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat do FILE="${i}/build/coverage/full.cov" if [ -f "${FILE}" ]; then bash codecov -f "${FILE}" fi done
- Duration: 2 min 22 sec
- Start Time: 2020-07-07T08:29:03.703+0000
- log
Name: Make -C packetbeat testsuite
- Description: make -C packetbeat testsuite
- Duration: 1 min 41 sec
- Start Time: 2020-07-07T07:49:33.792+0000
- log
Name: Report to Codecov
- Description: curl -sSLo codecov https://codecov.io/bash for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat do FILE="${i}/build/coverage/full.cov" if [ -f "${FILE}" ]; then bash codecov -f "${FILE}" fi done
- Duration: 2 min 22 sec
- Start Time: 2020-07-07T07:50:16.327+0000
- log
Name: Report to Codecov
- Description: curl -sSLo codecov https://codecov.io/bash for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat do FILE="${i}/build/coverage/full.cov" if [ -f "${FILE}" ]; then bash codecov -f "${FILE}" fi done
- Duration: 2 min 22 sec
- Start Time: 2020-07-07T08:12:36.527+0000
- log

Log output

Expand to view the last 100 lines of log output

[2020-07-07T08:43:50.323Z]  OS/Arch:           darwin/amd64
[2020-07-07T08:43:50.323Z]  Experimental:      false
[2020-07-07T08:43:50.323Z] Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
[2020-07-07T08:43:50.323Z] It requires Docker daemon to be installed and running
[2020-07-07T08:44:04.933Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats
[2020-07-07T08:44:05.238Z] + find . -type f -name TEST*.xml -path */build/* -delete
[2020-07-07T08:44:05.252Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Lint
[2020-07-07T08:44:05.349Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Packetbeat-oss
[2020-07-07T08:44:05.428Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Winlogbeat-oss
[2020-07-07T08:44:05.539Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Elastic-Agent-x-pack
[2020-07-07T08:44:05.636Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Auditbeat-crosscompile
[2020-07-07T08:44:05.734Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Dockerlogbeat
[2020-07-07T08:44:05.827Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Journalbeat-oss
[2020-07-07T08:44:05.906Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Generators-Metricbeat-Linux
[2020-07-07T08:44:05.982Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Functionbeat-x-pack
[2020-07-07T08:44:06.087Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Elastic-Agent-x-pack-Windows
[2020-07-07T08:44:06.181Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Elastic-Agent-Mac-OS-X
[2020-07-07T08:44:06.268Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Metricbeat-OSS-Unit-tests
[2020-07-07T08:44:06.356Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Auditbeat-oss-Windows
[2020-07-07T08:44:06.479Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Auditbeat-x-pack
[2020-07-07T08:44:06.559Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Metricbeat-crosscompile
[2020-07-07T08:44:06.655Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Heartbeat-oss
[2020-07-07T08:44:06.731Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Winlogbeat-Windows-x-pack
[2020-07-07T08:44:06.805Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Auditbeat-x-pack-Windows
[2020-07-07T08:44:06.880Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Auditbeat-oss-Mac-OS-X
[2020-07-07T08:44:06.954Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Libbeat-x-pack
[2020-07-07T08:44:07.030Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Filebeat-Windows
[2020-07-07T08:44:07.103Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Filebeat-x-pack-Mac-OS-X
[2020-07-07T08:44:07.175Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Filebeat-Mac-OS-X
[2020-07-07T08:44:07.249Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Filebeat-x-pack-Windows
[2020-07-07T08:44:07.324Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Auditbeat-oss-Linux
[2020-07-07T08:44:07.397Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Auditbeat-x-pack-Mac-OS-X
[2020-07-07T08:44:07.471Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Winlogbeat-Windows
[2020-07-07T08:44:07.543Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Metricbeat-x-pack-Windows
[2020-07-07T08:44:07.628Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Filebeat-x-pack
[2020-07-07T08:44:07.710Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Metricbeat-Windows
[2020-07-07T08:44:07.795Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Metricbeat-Mac-OS-X
[2020-07-07T08:44:07.867Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Generators-Beat-Linux
[2020-07-07T08:44:07.940Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Metricbeat-x-pack-Mac-OS-X
[2020-07-07T08:44:08.012Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Filebeat-oss
[2020-07-07T08:44:08.095Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Functionbeat-Mac-OS-X-x-pack
[2020-07-07T08:44:08.168Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Metricbeat-Python-integration-tests
[2020-07-07T08:44:08.242Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Metricbeat-x-pack
[2020-07-07T08:44:08.328Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Heartbeat-Mac-OS-X
[2020-07-07T08:44:08.406Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests
[2020-07-07T08:44:08.493Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Libbeat-oss
[2020-07-07T08:44:08.567Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Generators-Metricbeat-Mac-OS-X
[2020-07-07T08:44:08.639Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Libbeat-crosscompile
[2020-07-07T08:44:08.710Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Libbeat-stress-tests
[2020-07-07T08:44:08.782Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Functionbeat-Windows
[2020-07-07T08:44:08.859Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Heartbeat-Windows
[2020-07-07T08:44:08.929Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Generators-Beat-Mac-OS-X
[2020-07-07T08:44:09.290Z] + cat
[2020-07-07T08:44:09.290Z] + /usr/local/bin/runbld ./runbld-script
[2020-07-07T08:44:09.290Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-07-07T08:44:15.911Z] runbld>>> runbld started
[2020-07-07T08:44:15.911Z] runbld>>> 1.6.12/f45d832f2ba0aa2722ab4ec1fda8ad140f027f8b
[2020-07-07T08:44:18.473Z] runbld>>> The following profiles matched the job 'Beats/beats-beats-mbp/PR-19398' in order of occurrence in the config (last value wins).
[2020-07-07T08:44:19.417Z] runbld>>> Debug logging enabled.
[2020-07-07T08:44:19.417Z] runbld>>> Storing result
[2020-07-07T08:44:19.679Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-07-07T08:44:19.679Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200707084419-CF000742
[2020-07-07T08:44:19.679Z] runbld>>> Adding system facts.
[2020-07-07T08:44:20.627Z] runbld>>> Adding vcs info for the latest commit:  72f96235a377c5197481b1a1266beed875c6f22d
[2020-07-07T08:44:20.893Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-07-07T08:44:20.893Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-07-07T08:44:20.893Z] Processing JUnit reports with runbld...
[2020-07-07T08:44:20.893Z] + echo 'Processing JUnit reports with runbld...'
[2020-07-07T08:44:21.156Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-07-07T08:44:21.156Z] runbld>>> DURATION: 23ms
[2020-07-07T08:44:21.156Z] runbld>>> STDOUT: 40 bytes
[2020-07-07T08:44:21.156Z] runbld>>> STDERR: 49 bytes
[2020-07-07T08:44:21.156Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-07-07T08:44:21.156Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats
[2020-07-07T08:44:22.546Z] runbld>>> Storing build metadata: 
[2020-07-07T08:44:22.546Z] runbld>>> Adding test report.
[2020-07-07T08:44:22.546Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats
[2020-07-07T08:44:23.119Z] runbld>>> Found 113 test output files
[2020-07-07T08:44:23.694Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-istio.xml
[2020-07-07T08:44:23.694Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-tomcat.xml
[2020-07-07T08:44:23.694Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-iis.xml
[2020-07-07T08:44:23.694Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-openmetrics.xml
[2020-07-07T08:44:23.694Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-activemq.xml
[2020-07-07T08:44:25.089Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-graphite.xml
[2020-07-07T08:44:25.089Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-windows.xml
[2020-07-07T08:44:25.352Z] runbld>>> Test output logs contained: Errors: 0 Failures: 2 Tests: 10947 Skipped: 1339
[2020-07-07T08:44:25.615Z] runbld>>> Storing result
[2020-07-07T08:44:25.615Z] runbld>>> FAILURES: 2
[2020-07-07T08:44:26.190Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-07-07T08:44:26.190Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200707084419-CF000742
[2020-07-07T08:44:26.452Z] runbld>>> Email notification disabled by environment variable.
[2020-07-07T08:44:26.452Z] runbld>>> Slack notification disabled by environment variable.
[2020-07-07T08:44:32.539Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-19398
[2020-07-07T08:44:32.764Z] [INFO] getVaultSecret: Getting secrets
[2020-07-07T08:44:32.827Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-07-07T08:44:33.598Z] + chmod 755 generate-build-data.sh
[2020-07-07T08:44:33.598Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-19398/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-19398/runs/27 FAILURE 4757787
[2020-07-07T08:44:33.598Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-19398/runs/27/steps/?limit=10000 -o steps-info.json
[2020-07-07T08:44:35.460Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-19398/runs/27/tests/?status=FAILED -o tests-errors.json
[2020-07-07T08:44:35.460Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-19398/runs/27/log/ -o pipeline-log.txt

Signed-off-by: chrismark <[email protected]>

elasticmachine · 2020-06-26T12:56:54Z

Pinging @elastic/integrations-platforms (Team:Platforms)

Signed-off-by: chrismark <[email protected]>

jsoriano

Nice feature! But there are some small things that I think need to be addressed.

libbeat/common/kubernetes/k8skeystore/kubernetes_keystore.go

metricbeat/autodiscover/builder/hints/metrics.go

metricbeat/autodiscover/builder/hints/metrics_test.go

Signed-off-by: chrismark <[email protected]>

ChrsMark · 2020-06-29T08:28:06Z

@jsoriano thanks for the review. Comments have been addressed, let me know what you think!

jsoriano

Looks good, added a question about the keystore fix and some nitpicking.

metricbeat/autodiscover/builder/hints/metrics.go

libbeat/common/kubernetes/k8skeystore/kubernetes_keystore.go

metricbeat/autodiscover/builder/hints/metrics_test.go

metricbeat/autodiscover/builder/hints/metrics.go

jsoriano

LGTM once CI goes green. Thanks!

Signed-off-by: chrismark <[email protected]>

…d_ports

ChrsMark · 2020-07-06T14:02:48Z

This PR was updated in order to not only cover hints based Autodiscover but also static Autodiscover templates.

@exekias @jsoriano could you please have another look?

Signed-off-by: chrismark <[email protected]>

kaiyan-sheng

looks good to me!

jsoriano

LGTM, spotted an small typo in docs, and added a suggestion for tests, but it can be done later.

metricbeat/docs/autodiscover-kubernetes-config.asciidoc

Signed-off-by: chrismark <[email protected]>

ChrsMark · 2020-07-07T07:25:52Z

Thanks for the reviews @kaiyan-sheng and @jsoriano!

I've added a couple of tests for config templates too.

jsoriano

Thanks for adding the extra tests!

jsoriano · 2020-07-07T08:05:00Z

libbeat/autodiscover/template/config_test.go

+				},
+			},
+			expected: []*common.Config{configPorts},
+		},


…tic#19398) (cherry picked from commit 1b22373)

…) (#19694) (cherry picked from commit 1b22373)

…tic#19398)

Add support for named ports in hints autodiscover

86446ae

Signed-off-by: chrismark <[email protected]>

ChrsMark self-assigned this Jun 25, 2020

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jun 25, 2020

ChrsMark mentioned this pull request Jun 25, 2020

Kubernetes autodiscover - multiple ports #15796

Closed

andresrc added the Team:Platforms Label for the Integrations - Platforms team label Jun 26, 2020

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jun 26, 2020

ChrsMark added 2 commits June 26, 2020 15:44

Don't evaluate keystore if var does not match initial pattern

77156e6

Signed-off-by: chrismark <[email protected]>

fmt code

855ca41

Signed-off-by: chrismark <[email protected]>

ChrsMark added autodiscovery enhancement v7.9.0 needs_backport PR is waiting to be backported to other branches. test-plan Add this PR to be manual test plan labels Jun 26, 2020

ChrsMark added 2 commits June 26, 2020 15:51

Add changelog

8a82130

Signed-off-by: chrismark <[email protected]>

Add unit test

9417248

Signed-off-by: chrismark <[email protected]>

ChrsMark marked this pull request as ready for review June 26, 2020 12:56

fmt

0df13fd

Signed-off-by: chrismark <[email protected]>

jsoriano requested changes Jun 26, 2020

View reviewed changes

andresrc added [zube]: Inbox [zube]: In Review and removed [zube]: Inbox labels Jun 28, 2020

Review changes and docs

bcd44f2

Signed-off-by: chrismark <[email protected]>

ChrsMark force-pushed the autodiscover_named_ports branch from 81d1d0d to bcd44f2 Compare June 29, 2020 08:25

jsoriano self-requested a review June 29, 2020 08:43

jsoriano reviewed Jun 29, 2020

View reviewed changes

jsoriano approved these changes Jun 29, 2020

View reviewed changes

Minor changes

c5d5228

Signed-off-by: chrismark <[email protected]>

ChrsMark added 5 commits July 6, 2020 16:30

Add named ports support in templates

02e9eaf

Signed-off-by: chrismark <[email protected]>

Fix changelog

d187a9d

Signed-off-by: chrismark <[email protected]>

Remove regexp

561d14f

Signed-off-by: chrismark <[email protected]>

Add tests

b2ce6d6

Signed-off-by: chrismark <[email protected]>

Add template respective docs

0e60318

Signed-off-by: chrismark <[email protected]>

ChrsMark changed the title ~~Add support for named ports in hints autodiscover~~ Add support for named ports in autodiscover (hints & templates) Jul 6, 2020

Merge remote-tracking branch 'upstream/master' into autodiscover_name…

595d4fc

…d_ports

ChrsMark added 2 commits July 6, 2020 17:15

fmt

3631254

Signed-off-by: chrismark <[email protected]>

Fix tests

09fff4c

Signed-off-by: chrismark <[email protected]>

kaiyan-sheng approved these changes Jul 6, 2020

View reviewed changes

jsoriano self-requested a review July 6, 2020 16:06

jsoriano reviewed Jul 6, 2020

View reviewed changes

metricbeat/docs/autodiscover-kubernetes-config.asciidoc Show resolved Hide resolved

metricbeat/docs/autodiscover-kubernetes-config.asciidoc Outdated Show resolved Hide resolved

fix docs and add test for config templates

ae46a3a

Signed-off-by: chrismark <[email protected]>

ChrsMark requested a review from jsoriano July 7, 2020 07:26

jsoriano approved these changes Jul 7, 2020

View reviewed changes

libbeat/autodiscover/template/config_test.go

},

},

expected: []*common.Config{configPorts},

},

Copy link

Member

jsoriano Jul 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

ChrsMark merged commit 1b22373 into elastic:master Jul 7, 2020

zube bot added [zube]: Done and removed [zube]: In Review labels Jul 7, 2020

ChrsMark mentioned this pull request Jul 7, 2020

Cherry-pick #19398 to 7.x: Add support for named ports in autodiscover (hints & templates) #19694

Merged

ChrsMark added a commit to ChrsMark/beats that referenced this pull request Jul 7, 2020

Add support for named ports in autodiscover (hints & templates) (elas…

095ad9f

…tic#19398) (cherry picked from commit 1b22373)

ChrsMark removed the needs_backport PR is waiting to be backported to other branches. label Jul 7, 2020

ChrsMark added a commit that referenced this pull request Jul 7, 2020

Add support for named ports in autodiscover (hints & templates) (#19398…

5db114f

…) (#19694) (cherry picked from commit 1b22373)

ChrsMark mentioned this pull request Jul 13, 2020

Metricbeat: Hint based autodiscover for docker does not work when internal and exposed port differ #19825

Closed

andresrc added test-plan-added This PR has been added to the test plan and removed [zube]: Done labels Jul 14, 2020

melchiormoulin pushed a commit to melchiormoulin/beats that referenced this pull request Oct 14, 2020

Add support for named ports in autodiscover (hints & templates) (elas…

63528db

…tic#19398)

ChrsMark mentioned this pull request Apr 11, 2022

Add support for port mapping in docker hints #31243

Merged

tetianakravchenko mentioned this pull request Jan 15, 2024

Align the kubernetes provider variables documentation #37601

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for named ports in autodiscover (hints & templates) #19398

Add support for named ports in autodiscover (hints & templates) #19398

ChrsMark commented Jun 25, 2020 •

edited by andresrc

Loading

elasticmachine commented Jun 25, 2020 •

edited

Loading

Build stats

Test stats 🧪

elasticmachine commented Jun 26, 2020

jsoriano left a comment

ChrsMark commented Jun 29, 2020

jsoriano left a comment

jsoriano left a comment

ChrsMark commented Jul 6, 2020 •

edited

Loading

kaiyan-sheng left a comment

jsoriano left a comment

ChrsMark commented Jul 7, 2020

jsoriano left a comment

jsoriano Jul 7, 2020

Add support for named ports in autodiscover (hints & templates) #19398

Add support for named ports in autodiscover (hints & templates) #19398

Conversation

ChrsMark commented Jun 25, 2020 • edited by andresrc Loading

What does this PR do?

Why is it important?

Manual Testing

Hints based Autodiscover

Template based Autodiscover

elasticmachine commented Jun 25, 2020 • edited Loading

💔 Tests Failed

Build stats

Test stats 🧪

Test errors

Steps errors

Log output

elasticmachine commented Jun 26, 2020

jsoriano left a comment

Choose a reason for hiding this comment

ChrsMark commented Jun 29, 2020

jsoriano left a comment

Choose a reason for hiding this comment

jsoriano left a comment

Choose a reason for hiding this comment

ChrsMark commented Jul 6, 2020 • edited Loading

kaiyan-sheng left a comment

Choose a reason for hiding this comment

jsoriano left a comment

Choose a reason for hiding this comment

ChrsMark commented Jul 7, 2020

jsoriano left a comment

Choose a reason for hiding this comment

jsoriano Jul 7, 2020

Choose a reason for hiding this comment

ChrsMark commented Jun 25, 2020 •

edited by andresrc

Loading

elasticmachine commented Jun 25, 2020 •

edited

Loading

ChrsMark commented Jul 6, 2020 •

edited

Loading