[stable/heartbeat] Add Heartbeat to check status of your services #5766

mumoshu · 2018-05-25T09:13:41Z

Manually verified to work by installing it like helm install ~/charts/stable/heartbeat --name hb --namespace hb and then reading logs:

$ stern --namespace hb heart
+ hb-heartbeat-zljn2 › heartbeat
hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:17.531ZINFOinstance/beat.go:468Home path: [/usr/share/heartbeat] Config path: [/usr/share/heartbeat] Data path: [/usr/share/heartbeat/data] Logs path: [/usr/share/heartbeat/logs]
hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:17.532ZINFOinstance/beat.go:475Beat UUID: cc04f1e7-c1a2-4e9c-beca-009539becd87
hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:17.532ZINFOinstance/beat.go:213Setup Beat: heartbeat; Version: 6.2.4
hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.534ZINFOadd_cloud_metadata/add_cloud_metadata.go:297add_cloud_metadata: hosting provider type not detected.
hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.534ZINFOfileout/file.go:76Initialized file output. path=/usr/share/heartbeat/data/heartbeat max_size_bytes=10240000 max_backups=5 permissions=-rw-------
hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.535ZINFOpipeline/module.go:76Beat name: hb-heartbeat-zljn2
hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.535ZWARNbeater/heartbeat.go:24Beta: Heartbeat is beta software
hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.535ZINFObeater/manager.go:110Select (active) monitor icmp
hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.537ZINFO[monitoring]log/log.go:97Starting metrics logging every 30s
hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.538ZINFOinstance/beat.go:301heartbeat start running.
hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.538ZINFObeater/heartbeat.go:56heartbeat is running! Hit CTRL-C to stop it.
hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:50.538ZINFO[monitoring]log/log.go:124Non-zero metrics in the last 30s{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":40,"time":40},"total":{"ticks":70,"time":74,"value":70},"user":{"ticks":30,"time":34}},"info":{"ephemeral_id":"23539714-a27c-40bb-a124-805b7e009958","uptime":{"ms":33026}},"memstats":{"gc_next":4194304,"memory_alloc":2423104,"memory_total":4047776,"rss":22454272}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":5,"batches":5,"total":5},"type":"file","write":{"bytes":2306}},"pipeline":{"clients":1,"events":{"active":1,"published":6,"total":6},"queue":{"acked":5}}},"system":{"cpu":{"cores":2},"load":{"1":0.75,"15":0.38,"5":0.55,"norm":{"1":0.375,"15":0.19,"5":0.275}}}}}}

Templates and values.yaml is mostly similar to stable/filebeat's.
The main difference is config: in values.yaml, which directly reflects heartbeat's own configuration file.

mumoshu · 2018-05-25T09:14:10Z

/assign @viglesiasce

mumoshu · 2018-05-25T09:14:29Z

@viglesiasce Would you mind reviewing this? Thanks!

mumoshu · 2018-06-01T05:53:17Z

Could anyone review this? Thanks.

exekias · 2018-06-12T10:29:25Z

stable/heartbeat/templates/daemonset.yaml

+          path: /var/log
+      - name: varlibdockercontainers
+        hostPath:
+          path: /var/lib/docker/containers


I don't think hearbeat needs to access this folder, Filebeat uses it to retrieve logs, but metadata is retrieved from the API

exekias · 2018-06-12T10:35:48Z

stable/heartbeat/templates/daemonset.yaml

+        - name: NODE_NAME
+          valueFrom:
+            fieldRef:
+              fieldPath: spec.nodeName


these 2 variables probably come from an older version, since 6.2 we no longer need them

exekias · 2018-06-12T10:35:59Z

stable/heartbeat/templates/daemonset.yaml

+      volumes:
+      - name: varlog
+        hostPath:
+          path: /var/log


This folder isn't needed either

exekias · 2018-06-12T10:36:31Z

stable/heartbeat/templates/daemonset.yaml

+      - name: data
+        hostPath:
+          path: /var/lib/heartbeat
+          type: DirectoryOrCreate


nice, this improves our current manifests, I'll copy it to upstream ❤️

exekias · 2018-06-12T10:39:04Z

stable/heartbeat/values.yaml

+  processors:
+  - add_cloud_metadata:
+
+  output.file:


How about leaving an elasticsearch output section commented out just after this one? That would help the user configuring the most common use case. Something like this:

# output.elasticsearch: # hosts: ["localhost:9200"] # protocol: "https" # username: "elastic" # password: "changeme"

Sounds good!

mumoshu · 2018-06-13T02:11:22Z

@exekias Thanks for the review! Every point made sense to me / was addressed in the follow-up commits 👍

unguiculus

Please add a config table to the readme.

unguiculus · 2018-06-17T19:39:37Z

stable/heartbeat/templates/daemonset.yaml

@@ -0,0 +1,77 @@
+apiVersion: apps/v1beta2


Since you require K8s 1.9+ in the readme, you could go for apps/v1.

unguiculus · 2018-06-17T19:40:04Z

/assign

Manually verified to work by installing it like `helm install ~/charts/stable/heartbeat --name hb --namespace hb` and then reading logs: ```console $ stern --namespace hb heart + hb-heartbeat-zljn2 › heartbeat hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:17.531ZINFOinstance/beat.go:468Home path: [/usr/share/heartbeat] Config path: [/usr/share/heartbeat] Data path: [/usr/share/heartbeat/data] Logs path: [/usr/share/heartbeat/logs] hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:17.532ZINFOinstance/beat.go:475Beat UUID: cc04f1e7-c1a2-4e9c-beca-009539becd87 hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:17.532ZINFOinstance/beat.go:213Setup Beat: heartbeat; Version: 6.2.4 hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.534ZINFOadd_cloud_metadata/add_cloud_metadata.go:297add_cloud_metadata: hosting provider type not detected. hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.534ZINFOfileout/file.go:76Initialized file output. path=/usr/share/heartbeat/data/heartbeat max_size_bytes=10240000 max_backups=5 permissions=-rw------- hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.535ZINFOpipeline/module.go:76Beat name: hb-heartbeat-zljn2 hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.535ZWARNbeater/heartbeat.go:24Beta: Heartbeat is beta software hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.535ZINFObeater/manager.go:110Select (active) monitor icmp hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.537ZINFO[monitoring]log/log.go:97Starting metrics logging every 30s hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.538ZINFOinstance/beat.go:301heartbeat start running. hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.538ZINFObeater/heartbeat.go:56heartbeat is running! Hit CTRL-C to stop it. hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:50.538ZINFO[monitoring]log/log.go:124Non-zero metrics in the last 30s{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":40,"time":40},"total":{"ticks":70,"time":74,"value":70},"user":{"ticks":30,"time":34}},"info":{"ephemeral_id":"23539714-a27c-40bb-a124-805b7e009958","uptime":{"ms":33026}},"memstats":{"gc_next":4194304,"memory_alloc":2423104,"memory_total":4047776,"rss":22454272}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":5,"batches":5,"total":5},"type":"file","write":{"bytes":2306}},"pipeline":{"clients":1,"events":{"active":1,"published":6,"total":6},"queue":{"acked":5}}},"system":{"cpu":{"cores":2},"load":{"1":0.75,"15":0.38,"5":0.55,"norm":{"1":0.375,"15":0.19,"5":0.275}}}}}} ``` Templates and values.yaml is mostly similar to stable/filebeat's. The main difference is `config:` in values.yaml, which directly reflects heartbeat's own configuration file. Other changelog: - Include the configuration table in README - Include the commented-out elasticsearch output to support common use-cases

mumoshu · 2018-06-18T07:30:51Z

@unguiculus Thanks for reviewing! I've addressed your comments in the last commit. Would you mind taking a look again?

unguiculus · 2018-06-18T18:54:46Z

/ok-to-test

unguiculus · 2018-06-18T18:58:17Z

/lgtm

k8s-ci-robot · 2018-06-18T18:58:24Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mumoshu, unguiculus

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [unguiculus]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Manually verified to work by installing it like `helm install ~/charts/stable/heartbeat --name hb --namespace hb` and then reading logs: ```console $ stern --namespace hb heart + hb-heartbeat-zljn2 › heartbeat hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:17.531ZINFOinstance/beat.go:468Home path: [/usr/share/heartbeat] Config path: [/usr/share/heartbeat] Data path: [/usr/share/heartbeat/data] Logs path: [/usr/share/heartbeat/logs] hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:17.532ZINFOinstance/beat.go:475Beat UUID: cc04f1e7-c1a2-4e9c-beca-009539becd87 hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:17.532ZINFOinstance/beat.go:213Setup Beat: heartbeat; Version: 6.2.4 hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.534ZINFOadd_cloud_metadata/add_cloud_metadata.go:297add_cloud_metadata: hosting provider type not detected. hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.534ZINFOfileout/file.go:76Initialized file output. path=/usr/share/heartbeat/data/heartbeat max_size_bytes=10240000 max_backups=5 permissions=-rw------- hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.535ZINFOpipeline/module.go:76Beat name: hb-heartbeat-zljn2 hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.535ZWARNbeater/heartbeat.go:24Beta: Heartbeat is beta software hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.535ZINFObeater/manager.go:110Select (active) monitor icmp hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.537ZINFO[monitoring]log/log.go:97Starting metrics logging every 30s hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.538ZINFOinstance/beat.go:301heartbeat start running. hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.538ZINFObeater/heartbeat.go:56heartbeat is running! Hit CTRL-C to stop it. hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:50.538ZINFO[monitoring]log/log.go:124Non-zero metrics in the last 30s{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":40,"time":40},"total":{"ticks":70,"time":74,"value":70},"user":{"ticks":30,"time":34}},"info":{"ephemeral_id":"23539714-a27c-40bb-a124-805b7e009958","uptime":{"ms":33026}},"memstats":{"gc_next":4194304,"memory_alloc":2423104,"memory_total":4047776,"rss":22454272}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":5,"batches":5,"total":5},"type":"file","write":{"bytes":2306}},"pipeline":{"clients":1,"events":{"active":1,"published":6,"total":6},"queue":{"acked":5}}},"system":{"cpu":{"cores":2},"load":{"1":0.75,"15":0.38,"5":0.55,"norm":{"1":0.375,"15":0.19,"5":0.275}}}}}} ``` Templates and values.yaml is mostly similar to stable/filebeat's. The main difference is `config:` in values.yaml, which directly reflects heartbeat's own configuration file. Other changelog: - Include the configuration table in README - Include the commented-out elasticsearch output to support common use-cases

Manually verified to work by installing it like `helm install ~/charts/stable/heartbeat --name hb --namespace hb` and then reading logs: ```console $ stern --namespace hb heart + hb-heartbeat-zljn2 › heartbeat hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:17.531ZINFOinstance/beat.go:468Home path: [/usr/share/heartbeat] Config path: [/usr/share/heartbeat] Data path: [/usr/share/heartbeat/data] Logs path: [/usr/share/heartbeat/logs] hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:17.532ZINFOinstance/beat.go:475Beat UUID: cc04f1e7-c1a2-4e9c-beca-009539becd87 hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:17.532ZINFOinstance/beat.go:213Setup Beat: heartbeat; Version: 6.2.4 hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.534ZINFOadd_cloud_metadata/add_cloud_metadata.go:297add_cloud_metadata: hosting provider type not detected. hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.534ZINFOfileout/file.go:76Initialized file output. path=/usr/share/heartbeat/data/heartbeat max_size_bytes=10240000 max_backups=5 permissions=-rw------- hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.535ZINFOpipeline/module.go:76Beat name: hb-heartbeat-zljn2 hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.535ZWARNbeater/heartbeat.go:24Beta: Heartbeat is beta software hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.535ZINFObeater/manager.go:110Select (active) monitor icmp hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.537ZINFO[monitoring]log/log.go:97Starting metrics logging every 30s hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.538ZINFOinstance/beat.go:301heartbeat start running. hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:20.538ZINFObeater/heartbeat.go:56heartbeat is running! Hit CTRL-C to stop it. hb-heartbeat-zljn2 heartbeat 2018-05-25T09:07:50.538ZINFO[monitoring]log/log.go:124Non-zero metrics in the last 30s{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":40,"time":40},"total":{"ticks":70,"time":74,"value":70},"user":{"ticks":30,"time":34}},"info":{"ephemeral_id":"23539714-a27c-40bb-a124-805b7e009958","uptime":{"ms":33026}},"memstats":{"gc_next":4194304,"memory_alloc":2423104,"memory_total":4047776,"rss":22454272}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"acked":5,"batches":5,"total":5},"type":"file","write":{"bytes":2306}},"pipeline":{"clients":1,"events":{"active":1,"published":6,"total":6},"queue":{"acked":5}}},"system":{"cpu":{"cores":2},"load":{"1":0.75,"15":0.38,"5":0.55,"norm":{"1":0.375,"15":0.19,"5":0.275}}}}}} ``` Templates and values.yaml is mostly similar to stable/filebeat's. The main difference is `config:` in values.yaml, which directly reflects heartbeat's own configuration file. Other changelog: - Include the configuration table in README - Include the commented-out elasticsearch output to support common use-cases Signed-off-by: voron <[email protected]>

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 25, 2018

k8s-ci-robot assigned viglesiasce May 25, 2018

mumoshu changed the title ~~feat: stable/heartbeat to check status of your services~~ [stable/heartbeat] Heartbeat to check status of your services May 25, 2018

mumoshu mentioned this pull request May 28, 2018

feat: Heartbeat + Awsbeats s12v/awsbeats#30

Closed

mumoshu changed the title ~~[stable/heartbeat] Heartbeat to check status of your services~~ [stable/heartbeat] Add Heartbeat to check status of your services May 30, 2018

mumoshu mentioned this pull request Jun 5, 2018

[stable/elastalert] Fix verbose/confusing index names on Elasticsearch 6.x #5912

Merged

exekias reviewed Jun 12, 2018

View reviewed changes

exekias mentioned this pull request Jun 12, 2018

[stable/apm-server] Elastic APM Server #6058

Merged

unguiculus suggested changes Jun 17, 2018

View reviewed changes

k8s-ci-robot assigned unguiculus Jun 17, 2018

mumoshu force-pushed the heartbeat branch from 789ea3e to 868f5a6 Compare June 18, 2018 01:30

k8s-ci-robot removed the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jun 18, 2018

unguiculus approved these changes Jun 18, 2018

View reviewed changes

k8s-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jun 18, 2018

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 18, 2018

k8s-ci-robot merged commit 15f492d into helm:master Jun 18, 2018

mumoshu deleted the heartbeat branch June 20, 2018 00:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[stable/heartbeat] Add Heartbeat to check status of your services #5766

[stable/heartbeat] Add Heartbeat to check status of your services #5766

mumoshu commented May 25, 2018

mumoshu commented May 25, 2018

mumoshu commented May 25, 2018

mumoshu commented Jun 1, 2018

exekias Jun 12, 2018

exekias Jun 12, 2018

exekias Jun 12, 2018

exekias Jun 12, 2018

exekias Jun 12, 2018 •

edited

Loading

mumoshu Jun 13, 2018

mumoshu commented Jun 13, 2018

unguiculus left a comment

unguiculus Jun 17, 2018

unguiculus commented Jun 17, 2018

mumoshu commented Jun 18, 2018

unguiculus commented Jun 18, 2018

unguiculus commented Jun 18, 2018

k8s-ci-robot commented Jun 18, 2018

[stable/heartbeat] Add Heartbeat to check status of your services #5766

[stable/heartbeat] Add Heartbeat to check status of your services #5766

Conversation

mumoshu commented May 25, 2018

mumoshu commented May 25, 2018

mumoshu commented May 25, 2018

mumoshu commented Jun 1, 2018

exekias Jun 12, 2018

Choose a reason for hiding this comment

exekias Jun 12, 2018

Choose a reason for hiding this comment

exekias Jun 12, 2018

Choose a reason for hiding this comment

exekias Jun 12, 2018

Choose a reason for hiding this comment

exekias Jun 12, 2018 • edited Loading

Choose a reason for hiding this comment

mumoshu Jun 13, 2018

Choose a reason for hiding this comment

mumoshu commented Jun 13, 2018

unguiculus left a comment

Choose a reason for hiding this comment

unguiculus Jun 17, 2018

Choose a reason for hiding this comment

unguiculus commented Jun 17, 2018

mumoshu commented Jun 18, 2018

unguiculus commented Jun 18, 2018

unguiculus commented Jun 18, 2018

k8s-ci-robot commented Jun 18, 2018

exekias Jun 12, 2018 •

edited

Loading