Architecture
The final goal of the entire project was to create performance dashboards for vSphere with Open Source software only. Real time, short and long term data had to be displayed to allow performance troubleshooting as well as long term pattern recognition and trend analysis. Or course, VMware vRealize Operations Manager would have provided that, too, but at high purchase cost and possibly high integration cost into the existing monitoring and dashboard infrastructure that already exists.
Collectd seemd to be the obvious choice to hook into: I provides the mechanisms that allow efficient grabbing of performance data making use of intensive caching which proved to be extremely important: The vSphere API [1] is not exactly made for efficient grabbing of performance data without a good mechanism to cache object information. Moreover it already provides a variety of "writer" plugins to dispatch data to different backend such as Graphite. We decided to use Python to develop the plugin and Pysphere [2] to access the vSphere API.
So, the target architecture of my development looks like that:
Collectsphere integrates into Collectd as a Python plugin and feeds values into Collectd. Collectd dispatches those values to a Graphite writer plugin which sends the values to the Graphite system - usually via UDP. Graphite stores the data and provides access for Grafana. Finally, Grafana accesses data from Graphite and displays it in nice and clean dashboards.
[1] https://www.vmware.com/support/developer/vc-sdk/visdk41pubs/ApiReference