[Metricbeat] VSphere module dashboards

[philippkahr]

Hi,

I created some dashboards for the VSphere module.

#14101

[elasticmachine]

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

[elasticmachine]

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

[philippkahr]

The Travis build is failing with

Get the licenses available from ['./vendor', './metricbeat/vendor', './metricbeat/module/kvm/vendor', './metricbeat/module/postgresql/vendor', './metricbeat/module/vsphere/vendor', './metricbeat/module/mysql/vendor']

but I do not know how to fix this.

[ChrsMark]

Hi @philippkahr! Great work on this, thank you!

I was able to load the dashboards without a problem! My two points are:

1. Could you add an updated screenshot of the dashboards in this PR so as people can comment on the visualisations and propose improvements to make them more meaningful?
2. I wonder if it is ok to have 2 different dashboards for this module.

[philippkahr]

Hi @ChrsMark

thanks for reviewing my changes. Sorry for the "typos", maybe we should add a naming scheme and example to the contributions guide?

1. Could you add an updated screenshot of the dashboards in this PR so as people can comment on the visualizations and propose improvements to make them more meaningful?

Sure! I will do that later tonight.

2. I wonder if it is ok to have 2 different dashboards for this module.
   Hmm, that depends really. I think the first dashboard is really for the cluster and hosts. The other one is a drill down into the stats of the virtual machine. Combining those would not make much sense.

[ChrsMark]

Hi @ChrsMark

thanks for reviewing my changes. Sorry for the "typos", maybe we should add a naming scheme and example to the contributions guide?

Sure! We are currently in discussions about improving the Developer Guides in general. Feel free to open an issue mentioning that proposal.

[ChrsMark]

Thank you for taking the time to open the issue @philippkahr!

We can go on with this PR regardless, so feel free to add screenshot(s) of the dashboards in new comments here so as to verify them and maybe suggest improvements. 🙂

[Jarzamendia]

I'm very excited to see this in action! My team are very happy with the Infrastructure dashboard. If we can also see the VSphere metrics to, this stack will add so much value to our environment!

[philippkahr]

Hi,

sorry, it took so long.

I had to add a lot of white boxes and some example text since I am using real data. I hope that this is still ok.

Metricbeat VSphere Hosts Overview ECS

Metricbeat VSphere VMs overview ECS

The heatmap itself is used to show where a VM has been on an ESXi host in the timeframe. This might show you if a VM is likely to be moved around a lot. The other dashboards are self-explanatory, to me at least :).

[Jarzamendia]

A very good view would be a VM distributed by the hosts. Similar to what you see on the Infrastructure tab. This would allow us to quickly identify hosts that could soon become overloaded if we know which VMs will soon have access spikes.

[philippkahr]

Hi @ChrsMark

can you please double-check if I did the rebase correctly? I am doing this for the first time.

inside the vspheredashboard branch from my forked repo
$ git remote -v

origin  [email protected]:philippkahr/beats.git (fetch)
origin  [email protected]:philippkahr/beats.git (push)
upstream        https://github.com/elastic/beats.git (fetch)
upstream        https://github.com/elastic/beats.git (push) 

$ git rebase upstream/master

First, rewinding head to replay your work on top of it...
Applying: https://github.com/elastic/beats/issues/14101 renamed everything according to xyz [metricbeat] ECS, imported and exported the dashboards, created the module.yml
Applying: https://github.com/elastic/beats/pull/14135 added dashboard
Applying: https://github.com/elastic/beats/pull/14135 trying to fix the error from travis ci
Applying: https://github.com/elastic/beats/pull/14135 fixing naming pattern missmatch
Applying: https://github.com/elastic/beats/pull/14135: fixed: empty description, fixed: expected title with format 'Some title [Metricbeat Module]'
Applying: https://github.com/elastic/beats/pull/14135: travis told me to run `Error: some files are not up-to-date. Run 'mage fmt update' then review and commit the changes. Modified: [metricbeat/docs/modules_list.asciidoc]`
Applying: https://github.com/elastic/beats/pull/14135 Remove submodule beats from vendor
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-vm.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-vm.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-host.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-host.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-vm.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-vm.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-vm.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-host.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-host.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-host.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-vm.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-vm.json
Applying: https://github.com/elastic/beats/pull/14135 added the vsphere dashboards to the docs
Applying: make update

$ git push -f origin vspheredashboards 

Enumerating objects: 211, done.
Counting objects: 100% (211/211), done.
Delta compression using up to 8 threads
Compressing objects: 100% (161/161), done.
Writing objects: 100% (197/197), 1.78 MiB | 2.33 MiB/s, done.
Total 197 (delta 138), reused 10 (delta 3)
remote: Resolving deltas: 100% (138/138), completed with 13 local objects.
To github.com:philippkahr/beats.git
 + 13de30c44...41c36d5a3 vspheredashboards -> vspheredashboards (forced update)

[ChrsMark]

Hi @ChrsMark

can you please double-check if I did the rebase correctly? I am doing this for the first time.

inside the vspheredashboard branch from my forked repo
$ git remote -v

origin  [email protected]:philippkahr/beats.git (fetch)
origin  [email protected]:philippkahr/beats.git (push)
upstream        https://github.com/elastic/beats.git (fetch)
upstream        https://github.com/elastic/beats.git (push) 

$ git rebase upstream/master

First, rewinding head to replay your work on top of it...
Applying: https://github.com/elastic/beats/issues/14101 renamed everything according to xyz [metricbeat] ECS, imported and exported the dashboards, created the module.yml
Applying: https://github.com/elastic/beats/pull/14135 added dashboard
Applying: https://github.com/elastic/beats/pull/14135 trying to fix the error from travis ci
Applying: https://github.com/elastic/beats/pull/14135 fixing naming pattern missmatch
Applying: https://github.com/elastic/beats/pull/14135: fixed: empty description, fixed: expected title with format 'Some title [Metricbeat Module]'
Applying: https://github.com/elastic/beats/pull/14135: travis told me to run `Error: some files are not up-to-date. Run 'mage fmt update' then review and commit the changes. Modified: [metricbeat/docs/modules_list.asciidoc]`
Applying: https://github.com/elastic/beats/pull/14135 Remove submodule beats from vendor
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-vm.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-vm.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-host.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-host.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-vm.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-vm.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-vm.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-host.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-host.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-host.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-vm.json
Applying: Update metricbeat/module/vsphere/_meta/kibana/7/dashboard/metricbeat-vsphere-vm.json
Applying: https://github.com/elastic/beats/pull/14135 added the vsphere dashboards to the docs
Applying: make update

$ git push -f origin vspheredashboards 

Enumerating objects: 211, done.
Counting objects: 100% (211/211), done.
Delta compression using up to 8 threads
Compressing objects: 100% (161/161), done.
Writing objects: 100% (197/197), 1.78 MiB | 2.33 MiB/s, done.
Total 197 (delta 138), reused 10 (delta 3)
remote: Resolving deltas: 100% (138/138), completed with 13 local objects.
To github.com:philippkahr/beats.git
 + 13de30c44...41c36d5a3 vspheredashboards -> vspheredashboards (forced update)

Yeah it looks ok, this is the way to go with rebasing. Thank you!

[ChrsMark]

ok to test

[ChrsMark]

Loaded the dashboards successfully. Found one left-over on one of the tittles.

After that I think we are ok to merge.

[ChrsMark]

ok to test

[ChrsMark]

Failing tests, seem to be irrelevant. Merging.

Thank you so much for contributing this @philippkahr 🙂 !