-
Notifications
You must be signed in to change notification settings - Fork 2.4k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[receiver/vcenter] Construct MetricsReceiver Component (#9224)
* add vcenter vSAN collection * checkpoint on getting property collection working * checkpoint before integration test * dual receivers under root receiver pointer * checkpoint before updated mdatagen * use syslog receiver rather than tcplogreceiver * getting more performance counter refinements * remove unneccessary component addition * try to fix go.mod resolution issues * try to fix go.mod resolution issues pt 2 * addlicense * fix go.mod by fixing require directive * add readme for metrics * update readme * fix go.mod referring nonexistent version * add performance manager tests * more tests * add more attributes to virtual machines and host systems * add more attributes to virtual machines and host systems * spike changelog entry * fix go.mod in both places * fix go.mod in configschema * add // import github.com/open-telemetry/opentelemetry-collector-contrib/receiver/vmwarevcenterreceiver to imports * add quotations * add to receiver lifecycle * remove extra go generate direction * fix typo of utilizaiton in metric description * small changes to interval id in performance queries to be more consistent * PR feedback including omitting company name prefix * PR feedback to not fail starting the component on potential network failures * minor grammar correction in vcenter readme * update expected metrics * update host_effective attribute value * remove PerformanceInterval customizability * add to codeowners * fix indentation on merge conflict * fix changelog entry place so its in the new components section * update to be on 0.49.0 of the collector' * add PR number to changelog * regenerate with newer version of mdatagen * move error log if unable to connect on start to receiver.Start() rather than scraper.Start() * fix test cases from last commit * minor update to config with tests * fix metric description * use utc for host vsan collection as well * update comments of public facing methods * return errors on getting clusters to the scraper errors * PR feedback #1 * instantiate new client if client is nil * update all descriptions to have punctuation * three more descs * move ensureReceiver up to once we validated as a config * some more PR feedback * looking into race conditions * run go tidy * fix import order and remove unneccessary mutex * remove mutex from struct * refactor client to responsible for knowing if the vsan endpoints are reachable * fix integration test referencing old var * change metrics.metrics => metrics.settings, update client pr feedback * remove vSAN collection temporarily * remove extra metric attributes for vSAN * remove vsan specific variables * clean up host PerfCounter disk latency metrics and fix some descriptions to better reflect interval * add 20s interval to extended documentation as needed * mdatagen fixes * add integration test metric scrape * fix import order * go up to 0.49.1 * gotidy * add replace directive for semconv * gotidy fixes * fix component not being on 0.50.0 * update to v0.50.1-0.20220429151328-041f39835df7 * use newer mdatagen * remove any logging functionality change && update documentation * fix integration test from flattening of config * fix scraper start not erroring if connection cannot be established * make scrapertest less flaky * format test json * Apply suggestions from code review Co-authored-by: Daniel Jaglowski <[email protected]> * adjust metric definition for vcenter.host.disk.throughput * remove comment and move pm level 2 metrics to appropriate section * try to be respective of datacenters * fix only vCenter server functionality * try building out a mock server for test coverage * make goporto * fix build issues * use latest mdatagen * add newlines to ends of xml recordings * fix integration test * moved around scrapererrors because now the receiver is datacenter dependent * try and do an audit of performance metrics and requests/responses * update testdata with correct units * make tidy * make tidy * update collector version * fix local testing code including modules * remove deprecated use of commonponenterror * pr feedback; add method of collection recording, return poweredOn/poweredOff VMs * remove content.json * fix description change in scraper_test.go * update collector version * bump replaced module; rebuild load tests * fix alibaba version auto localizing Co-authored-by: Daniel Jaglowski <[email protected]>
- Loading branch information
1 parent
41e4235
commit 6bdc5e5
Showing
63 changed files
with
55,337 additions
and
0 deletions.
There are no files selected for viewing
Validating CODEOWNERS rules …
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| include ../../Makefile.Common |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| # vCenter Receiver | ||
|
|
||
| This receiver fetches metrics from a vCenter or ESXi host running VMware vSphere APIs. | ||
|
|
||
| Supported pipeline types: `metrics` | ||
|
|
||
| > :construction: This receiver is in **Alpha**. Configuration fields and metric data model are subject to change. | ||
| ## Prerequisites | ||
|
|
||
| This receiver has been built to support ESXi and vCenter versions: | ||
|
|
||
| - 7.5 | ||
| - 7.0 | ||
| - 6.7 | ||
|
|
||
| A “Read Only” user assigned to a vSphere with permissions to the vCenter server, cluster and all subsequent resources being monitored must be specified in order for the receiver to retrieve information about them. | ||
|
|
||
| ## Configuration | ||
|
|
||
|
|
||
| | Parameter | Default | Type | Notes | | ||
| | --- | --- | --- | --- | | ||
| | endpoint | | String | Endpoint to the vCenter Server or ESXi host that has the sdk path enabled. Required. The expected format is `<protocol>://<hostname>` <br><br> i.e: `https://vcsa.hostname.localnet` | | ||
| | username | | String | Required | | ||
| | password | | String | Required | | ||
| | tls | | TLSClientSetting | Not Required. Will use defaults for [configtls.TLSClientSetting](https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configtls/README.md). By default insecure settings are rejected and certificate verification is on. | | ||
| | collection_interval | 2m | Duration | This receiver collects metrics on an interval. If the vCenter is fairly large, this value may need to be increased. Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h` | | ||
|
|
||
| ### Example Configuration | ||
|
|
||
| ```yaml | ||
| receivers: | ||
| vcenter: | ||
| endpoint: http://localhost:15672 | ||
| username: otelu | ||
| password: $VCENTER_PASSWORD | ||
| collection_interval: 5m | ||
| metrics: [] | ||
| ``` | ||
| The full list of settings exposed for this receiver are documented [here](./config.go) with detailed sample configurations [here](./testdata/config.yaml). TLS config is documented further under the [opentelemetry collector's configtls package](https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configtls/README.md). | ||
| ## Metrics | ||
| Details about the metrics produced by this receiver can be found in [metadata.yaml](./metadata.yaml) with further documentation in [documentation.md](./documentation.md) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,154 @@ | ||
| // Copyright The OpenTelemetry Authors | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| package vcenterreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/vcenterreceiver" | ||
|
|
||
| import ( | ||
| "context" | ||
| "fmt" | ||
| "net/url" | ||
|
|
||
| "github.com/vmware/govmomi" | ||
| "github.com/vmware/govmomi/find" | ||
| "github.com/vmware/govmomi/object" | ||
| "github.com/vmware/govmomi/performance" | ||
| "github.com/vmware/govmomi/property" | ||
| "github.com/vmware/govmomi/vim25" | ||
| vt "github.com/vmware/govmomi/vim25/types" | ||
| ) | ||
|
|
||
| // vcenterClient is a client that | ||
| type vcenterClient struct { | ||
| moClient *govmomi.Client | ||
| vimDriver *vim25.Client | ||
| finder *find.Finder | ||
| pc *property.Collector | ||
| pm *performance.Manager | ||
| cfg *Config | ||
| } | ||
|
|
||
| func newVcenterClient(c *Config) *vcenterClient { | ||
| return &vcenterClient{ | ||
| cfg: c, | ||
| } | ||
| } | ||
|
|
||
| // EnsureConnection will establish a connection to the vSphere SDK if not already established | ||
| func (vc *vcenterClient) EnsureConnection(ctx context.Context) error { | ||
| if vc.moClient != nil { | ||
| return nil | ||
| } | ||
|
|
||
| sdkURL, err := vc.cfg.SDKUrl() | ||
| if err != nil { | ||
| return err | ||
| } | ||
| client, err := govmomi.NewClient(ctx, sdkURL, vc.cfg.Insecure) | ||
| if err != nil { | ||
| return fmt.Errorf("unable to connect to vSphere SDK on listed endpoint: %w", err) | ||
| } | ||
| tlsCfg, err := vc.cfg.LoadTLSConfig() | ||
| if err != nil { | ||
| return err | ||
| } | ||
| client.DefaultTransport().TLSClientConfig = tlsCfg | ||
| user := url.UserPassword(vc.cfg.Username, vc.cfg.Password) | ||
| err = client.Login(ctx, user) | ||
| if err != nil { | ||
| return fmt.Errorf("unable to login to vcenter sdk: %w", err) | ||
| } | ||
| vc.moClient = client | ||
| vc.vimDriver = client.Client | ||
| vc.pc = property.DefaultCollector(vc.vimDriver) | ||
| vc.finder = find.NewFinder(vc.vimDriver) | ||
| vc.pm = performance.NewManager(vc.vimDriver) | ||
| return nil | ||
| } | ||
|
|
||
| // Disconnect will logout of the autenticated session | ||
| func (vc *vcenterClient) Disconnect(ctx context.Context) error { | ||
| if vc.moClient != nil { | ||
| return vc.moClient.Logout(ctx) | ||
| } | ||
| return nil | ||
| } | ||
|
|
||
| // Clusters returns the clusterComputeResources of the vSphere SDK | ||
| func (vc *vcenterClient) Datacenters(ctx context.Context) ([]*object.Datacenter, error) { | ||
| datacenters, err := vc.finder.DatacenterList(ctx, "*") | ||
| if err != nil { | ||
| return []*object.Datacenter{}, fmt.Errorf("unable to get datacenter lists: %w", err) | ||
| } | ||
| return datacenters, nil | ||
| } | ||
|
|
||
| // Clusters returns the clusterComputeResources of the vSphere SDK | ||
| func (vc *vcenterClient) Clusters(ctx context.Context, datacenter *object.Datacenter) ([]*object.ClusterComputeResource, error) { | ||
| vc.finder = vc.finder.SetDatacenter(datacenter) | ||
| clusters, err := vc.finder.ClusterComputeResourceList(ctx, "*") | ||
| if err != nil { | ||
| return []*object.ClusterComputeResource{}, err | ||
| } | ||
| return clusters, nil | ||
| } | ||
|
|
||
| // ResourcePools returns the resourcePools in the vSphere SDK | ||
| func (vc *vcenterClient) ResourcePools(ctx context.Context) ([]*object.ResourcePool, error) { | ||
| rps, err := vc.finder.ResourcePoolList(ctx, "*") | ||
| if err != nil { | ||
| return nil, fmt.Errorf("unable to retrieve resource pools: %w", err) | ||
| } | ||
| return rps, err | ||
| } | ||
|
|
||
| func (vc *vcenterClient) VMs(ctx context.Context) ([]*object.VirtualMachine, error) { | ||
| vms, err := vc.finder.VirtualMachineList(ctx, "*") | ||
| if err != nil { | ||
| return nil, fmt.Errorf("unable to retrieve resource pools: %w", err) | ||
| } | ||
| return vms, err | ||
| } | ||
|
|
||
| type perfSampleResult struct { | ||
| counters map[string]*vt.PerfCounterInfo | ||
| results []performance.EntityMetric | ||
| } | ||
|
|
||
| func (vc *vcenterClient) performanceQuery( | ||
| ctx context.Context, | ||
| spec vt.PerfQuerySpec, | ||
| names []string, | ||
| objs []vt.ManagedObjectReference, | ||
| ) (*perfSampleResult, error) { | ||
| if vc.pm == nil { | ||
| return &perfSampleResult{}, nil | ||
| } | ||
| vc.pm.Sort = true | ||
| sample, err := vc.pm.SampleByName(ctx, spec, names, objs) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| result, err := vc.pm.ToMetricSeries(ctx, sample) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| counterInfoByName, err := vc.pm.CounterInfoByName(ctx) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| return &perfSampleResult{ | ||
| counters: counterInfoByName, | ||
| results: result, | ||
| }, nil | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| // Copyright The OpenTelemetry Authors | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| package vcenterreceiver // import github.com/open-telemetry/opentelemetry-collector-contrib/receiver/vcenterreceiver | ||
|
|
||
| import ( | ||
| "context" | ||
| "testing" | ||
|
|
||
| "github.com/stretchr/testify/require" | ||
| "github.com/vmware/govmomi/find" | ||
| "github.com/vmware/govmomi/simulator" | ||
| "github.com/vmware/govmomi/vim25" | ||
| ) | ||
|
|
||
| func TestGetClusters(t *testing.T) { | ||
| simulator.Test(func(ctx context.Context, c *vim25.Client) { | ||
| finder := find.NewFinder(c) | ||
| client := vcenterClient{ | ||
| vimDriver: c, | ||
| finder: finder, | ||
| } | ||
| dc, err := finder.DefaultDatacenter(ctx) | ||
| require.NoError(t, err) | ||
| clusters, err := client.Clusters(ctx, dc) | ||
| require.NoError(t, err) | ||
| require.NotEmpty(t, clusters, 0) | ||
| }) | ||
| } | ||
|
|
||
| func TestGetResourcePools(t *testing.T) { | ||
| simulator.Test(func(ctx context.Context, c *vim25.Client) { | ||
| finder := find.NewFinder(c) | ||
| client := vcenterClient{ | ||
| vimDriver: c, | ||
| finder: finder, | ||
| } | ||
| resourcePools, err := client.ResourcePools(ctx) | ||
| require.NoError(t, err) | ||
| require.NotEmpty(t, resourcePools) | ||
| }) | ||
| } | ||
|
|
||
| func TestGetVMs(t *testing.T) { | ||
| simulator.Test(func(ctx context.Context, c *vim25.Client) { | ||
| finder := find.NewFinder(c) | ||
| client := vcenterClient{ | ||
| vimDriver: c, | ||
| finder: finder, | ||
| } | ||
| vms, err := client.VMs(ctx) | ||
| require.NoError(t, err) | ||
| require.NotEmpty(t, vms) | ||
| }) | ||
| } |
Oops, something went wrong.