Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Oracledb discovery #3633

Merged
Merged
Changes from 21 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
7328559
update bundled test and configs for oraclediscovery
greatestusername Sep 12, 2023
5cd9a57
Merge branch 'signalfx:main' into oracledb-discovery
greatestusername-splunk Sep 12, 2023
169dbfa
update bundled test and configs for oraclediscovery
greatestusername Sep 12, 2023
22740a7
Merge branch 'oracledb-discovery' of https://github.com/greatestusern…
greatestusername Sep 12, 2023
d3eb285
running bundled_test now evaluate metrics matching
greatestusername Sep 13, 2023
e1fdd6e
add working bundled_test
greatestusername Sep 13, 2023
b140890
update fail and partial messages for receiver config variables
greatestusername Sep 14, 2023
d6abba9
order of bundled configs matters
greatestusername Sep 14, 2023
9b6365c
make fmt and add more time
greatestusername Sep 14, 2023
6dd4758
lower durations, add in args for endpoint and service
greatestusername Sep 14, 2023
b8b6e00
reduce time on image pull and add to windows bundle
greatestusername Sep 19, 2023
60b62ba
add to bundle_windows_test
greatestusername Sep 19, 2023
56e903a
Merge branch 'signalfx:main' into oracledb-discovery
greatestusername Sep 19, 2023
35f5ab0
lower times on tests
greatestusername Sep 20, 2023
d36f0c2
address feedback on yaml template and test using env var
greatestusername Sep 20, 2023
224689d
gotta make generate-metrics
greatestusername Sep 20, 2023
3594b53
re-arrange bundled test
greatestusername Sep 21, 2023
f3aa6d1
add oracledb to the discovery readme
greatestusername Sep 20, 2023
b843405
make generate-metrics fmt
greatestusername Sep 21, 2023
09e68ed
Merge branch 'signalfx:main' into oracledb-discovery
greatestusername Sep 22, 2023
a4594da
address feedback
greatestusername Sep 22, 2023
dfff930
Update internal/confmapprovider/discovery/README.md
greatestusername Sep 22, 2023
dddcf85
fix to re-check and other attribute fields
greatestusername Sep 22, 2023
5f0eb86
add new test data for bundled test
greatestusername Sep 22, 2023
3f64271
add generated yaml
greatestusername Sep 22, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
#####################################################################################
# This file is generated by the Splunk Distribution of the OpenTelemetry Collector. #
# #
# It reflects the default configuration bundled in the Collector executable for use #
# in discovery mode (--discovery) and is provided for reference or customization. #
# Please note that any changes made to this file will need to be reconciled during #
# upgrades of the Collector. #
#####################################################################################
# oracledb:
# enabled: true
# rule:
# docker_observer: type == "container" and any([name, image, command], {# matches "(?i)oracle"}) and not (command matches "splunk.discovery")
# host_observer: type == "hostport" and command matches "(?i)oracle" and not (command matches "splunk.discovery")
# k8s_observer: type == "port" and pod.name matches "(?i)oracle"
# config:
# default:
# endpoint: splunk.discovery.default
# username: splunk.discovery.default
# password: splunk.discovery.default
# service: splunk.discovery.default
# status:
# metrics:
# successful:
# - strict: oracledb.cpu_time
# first_only: true
# log_record:
# severity_text: info
# body: oracledb receiver is working!
# statements:
# failed:
# - regexp: "connection refused"
# first_only: true
# log_record:
# severity_text: info
# append_pattern: true
# body: The container is not serving http connections.
# - regexp: "received goaway and there are no active streams"
# first_only: true
# log_record:
# severity_text: info
# append_pattern: true
# body: Unable to connect and scrape metrics.
# - regexp: "dial tcp: lookup"
# first_only: true
# log_record:
# severity_text: info
# append_pattern: true
# body: Unable to resolve oracledb tcp endpoint
# - regexp: 'error executing select .*: EOF'
# first_only: true
# log_record:
# severity_text: info
# append_pattern: true
# body: Unable to execute select from oracledb. Verify endpoint and user permissions.
# partial:
# - regexp: "listener does not currently know of service requested"
# first_only: true
# log_record:
# severity_text: info
# append_pattern: true
# body: >-
# Make sure your oracledb service is correctly specified using the
# `--set splunk.discovery.receivers.oracledb.config.service="<service>"` command or the
# `SPLUNK_DISCOVERY_RECEIVERS_oracledb_CONFIG_service="<service>"` environment variable.
# - regexp: 'invalid username/password'
# first_only: true
# log_record:
# severity_text: info
# append_pattern: true
# body: >-
# Make sure your user credentials are correctly specified using the
# `--set splunk.discovery.receivers.oracledb.config.username="<username>"` and
# `--set splunk.discovery.receivers.oracledb.config.password="<password>"` command or the
# `SPLUNK_DISCOVERY_RECEIVERS_oracledb_CONFIG_username="<username>"` and
# `SPLUNK_DISCOVERY_RECEIVERS_oracledb_CONFIG_password="<password>"` environment variables.
3 changes: 2 additions & 1 deletion internal/confmapprovider/discovery/README.md
Original file line number Diff line number Diff line change
@@ -135,9 +135,10 @@ By default, the discovery mode is provided with pre-made discovery config compon

The following components have bundled discovery configurations in the last Splunk OpenTelemetry Collector release:

I. Smart Agent receiver
I. Receivers
* `collectd/mysql` monitor type ([Linux](./bundle/bundle.d/receivers/smartagent-collectd-mysql.discovery.yaml))
* `collectd/nginx` monitor type ([Linux](./bundle/bundle.d/receivers/smartagent-collectd-nginx.discovery.yaml))
* `oracledb` monitor type ([Linux](./bundle/bundle.d/receivers/oracledb.discovery.yaml))
* `postgresql` monitor type ([Linux and Windows](./bundle/bundle.d/receivers/smartagent-postgresql.discovery.yaml))
greatestusername marked this conversation as resolved.
Show resolved Hide resolved

II. Extensions
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
#####################################################################################
# Do not edit manually! #
# All changes must be made to associated .tmpl file before running 'make bundle.d'. #
#####################################################################################
oracledb:
enabled: true
rule:
docker_observer: type == "container" and any([name, image, command], {# matches "(?i)oracle"}) and not (command matches "splunk.discovery")
host_observer: type == "hostport" and command matches "(?i)oracle" and not (command matches "splunk.discovery")
k8s_observer: type == "port" and pod.name matches "(?i)oracle"
config:
default:
endpoint: splunk.discovery.default
username: splunk.discovery.default
password: splunk.discovery.default
service: splunk.discovery.default
status:
metrics:
successful:
- strict: oracledb.cpu_time
first_only: true
log_record:
severity_text: info
body: oracledb receiver is working!
statements:
failed:
- regexp: "connection refused"
first_only: true
log_record:
severity_text: info
append_pattern: true
body: The container is not serving http connections.
- regexp: "received goaway and there are no active streams"
first_only: true
log_record:
severity_text: info
append_pattern: true
body: Unable to connect and scrape metrics.
- regexp: "dial tcp: lookup"
first_only: true
log_record:
severity_text: info
append_pattern: true
body: Unable to resolve oracledb tcp endpoint
- regexp: 'error executing select .*: EOF'
first_only: true
log_record:
severity_text: info
append_pattern: true
body: Unable to execute select from oracledb. Verify endpoint and user permissions.
partial:
- regexp: "listener does not currently know of service requested"
first_only: true
log_record:
severity_text: info
append_pattern: true
body: >-
Make sure your oracledb service is correctly specified using the
`--set splunk.discovery.receivers.oracledb.config.service="<service>"` command or the
`SPLUNK_DISCOVERY_RECEIVERS_oracledb_CONFIG_service="<service>"` environment variable.
- regexp: 'invalid username/password'
first_only: true
log_record:
severity_text: info
append_pattern: true
body: >-
Make sure your user credentials are correctly specified using the
`--set splunk.discovery.receivers.oracledb.config.username="<username>"` and
`--set splunk.discovery.receivers.oracledb.config.password="<password>"` command or the
`SPLUNK_DISCOVERY_RECEIVERS_oracledb_CONFIG_username="<username>"` and
`SPLUNK_DISCOVERY_RECEIVERS_oracledb_CONFIG_password="<password>"` environment variables.
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
{{ receiver "oracledb" }}:
enabled: true
rule:
docker_observer: type == "container" and any([name, image, command], {# matches "(?i)oracle"}) and not (command matches "splunk.discovery")
host_observer: type == "hostport" and command matches "(?i)oracle" and not (command matches "splunk.discovery")
k8s_observer: type == "port" and pod.name matches "(?i)oracle"
config:
default:
endpoint: {{ defaultValue }}
username: {{ defaultValue }}
password: {{ defaultValue }}
service: {{ defaultValue }}
status:
metrics:
successful:
- strict: oracledb.cpu_time
first_only: true
log_record:
severity_text: info
body: oracledb receiver is working!
statements:
failed:
rmfitzpatrick marked this conversation as resolved.
Show resolved Hide resolved
- regexp: "connection refused"
first_only: true
log_record:
severity_text: info
append_pattern: true
body: The container is not serving http connections.
- regexp: "received goaway and there are no active streams"
first_only: true
log_record:
severity_text: info
append_pattern: true
body: Unable to connect and scrape metrics.
- regexp: "dial tcp: lookup"
first_only: true
log_record:
severity_text: info
append_pattern: true
body: Unable to resolve oracledb tcp endpoint
- regexp: 'error executing select .*: EOF'
first_only: true
log_record:
severity_text: info
append_pattern: true
body: Unable to execute select from oracledb. Verify endpoint and user permissions.
partial:
- regexp: "listener does not currently know of service requested"
first_only: true
log_record:
severity_text: info
append_pattern: true
body: >-
Make sure your oracledb service is correctly specified using the
`--set {{ configProperty "service" "<service>" }}` command or the
`{{ configPropertyEnvVar "service" "<service>" }}` environment variable.
- regexp: 'invalid username/password'
first_only: true
log_record:
severity_text: info
append_pattern: true
body: >-
Make sure your user credentials are correctly specified using the
`--set {{ configProperty "username" "<username>" }}` and
`--set {{ configProperty "password" "<password>" }}` command or the
`{{ configPropertyEnvVar "username" "<username>" }}` and
`{{ configPropertyEnvVar "password" "<password>" }}` environment variables.

2 changes: 2 additions & 0 deletions internal/confmapprovider/discovery/bundle/bundle_gen.go
Original file line number Diff line number Diff line change
@@ -25,6 +25,8 @@
//go:generate discoverybundler -r -c -d ../../../../cmd/otelcol/config/collector/config.d.linux/receivers -t bundle.d/receivers/smartagent-collectd-mysql.discovery.yaml.tmpl
//go:generate discoverybundler -r -t bundle.d/receivers/smartagent-collectd-nginx.discovery.yaml.tmpl
//go:generate discoverybundler -r -c -d ../../../../cmd/otelcol/config/collector/config.d.linux/receivers -t bundle.d/receivers/smartagent-collectd-nginx.discovery.yaml.tmpl
//go:generate discoverybundler -r -t bundle.d/receivers/oracledb.discovery.yaml.tmpl
//go:generate discoverybundler -r -c -d ../../../../cmd/otelcol/config/collector/config.d.linux/receivers -t bundle.d/receivers/oracledb.discovery.yaml.tmpl

//go:generate discoverybundler -r -t bundle.d/extensions/docker-observer.discovery.yaml.tmpl
//go:generate discoverybundler -r -c -d ../../../../cmd/otelcol/config/collector/config.d.linux/extensions -t bundle.d/extensions/docker-observer.discovery.yaml.tmpl
Original file line number Diff line number Diff line change
@@ -27,6 +27,7 @@ func TestBundleDir(t *testing.T) {
receivers, err := fs.Glob(BundledFS, "bundle.d/receivers/*.discovery.yaml")
require.NoError(t, err)
require.Equal(t, []string{
"bundle.d/receivers/oracledb.discovery.yaml",
"bundle.d/receivers/smartagent-collectd-mysql.discovery.yaml",
"bundle.d/receivers/smartagent-collectd-nginx.discovery.yaml",
"bundle.d/receivers/smartagent-postgresql.discovery.yaml",
Original file line number Diff line number Diff line change
@@ -27,5 +27,6 @@ import (
// build errors.
//
//go:embed bundle.d/extensions/*.discovery.yaml
//go:embed bundle.d/receivers/oracledb.discovery.yaml
//go:embed bundle.d/receivers/smartagent-postgresql.discovery.yaml
var BundledFS embed.FS
Original file line number Diff line number Diff line change
@@ -27,6 +27,7 @@ func TestBundleDir(t *testing.T) {
receivers, err := fs.Glob(BundledFS, "bundle.d/receivers/*.discovery.yaml")
require.NoError(t, err)
require.Equal(t, []string{
"bundle.d/receivers/oracledb.discovery.yaml",
"bundle.d/receivers/smartagent-postgresql.discovery.yaml",
}, receivers)

61 changes: 61 additions & 0 deletions tests/receivers/oracledb/bundled_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
// Copyright Splunk, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

//go:build integration

package tests

import (
"fmt"
"runtime"
"testing"

"github.com/signalfx/splunk-otel-collector/tests/testutils"
)

func TestOracledbDockerObserver(t *testing.T) {
testutils.SkipIfNotContainerTest(t)
if runtime.GOOS == "darwin" {
t.Skip("unable to share sockets between mac and d4m vm: https://github.com/docker/for-mac/issues/483#issuecomment-758836836")
}

testutils.AssertAllMetricsReceived(t, "all.yaml", "otlp_exporter.yaml",
oracledb, []testutils.CollectorBuilder{
func(c testutils.Collector) testutils.Collector {
cc := c.(*testutils.CollectorContainer)
cc.Container = cc.Container.WithBinds("/var/run/docker.sock:/var/run/docker.sock:ro")
cc.Container = cc.Container.WillWaitForLogs("Discovering for next")
cc.Container = cc.Container.WithUser(fmt.Sprintf("999:%d", testutils.GetDockerGID(t)))
return cc
},
func(collector testutils.Collector) testutils.Collector {
return collector.WithEnv(map[string]string{
"ORACLEDB_URL": "oracle://otel:password@localhost:1521/xe",
"SPLUNK_DISCOVERY_DURATION": "10s",
// confirm that debug logging doesn't affect runtime
"SPLUNK_DISCOVERY_LOG_LEVEL": "debug",
"ORACLE_PASSWORD": "password",
}).WithArgs(
"--discovery",
"--set", "splunk.discovery.receivers.oracledb.config.username=otel",
"--set", "splunk.discovery.receivers.oracledb.config.password='${ORACLE_PASSWORD}'",
"--set", "splunk.discovery.receivers.oracledb.config.endpoint=localhost:1521",
"--set", "splunk.discovery.receivers.oracledb.config.service=xe",
"--set", `splunk.discovery.extensions.k8s_observer.enabled=false`,
"--set", `splunk.discovery.extensions.host_observer.enabled=false`,
)
},
},
)
}
4 changes: 2 additions & 2 deletions tests/receivers/oracledb/oracledb_test.go
Original file line number Diff line number Diff line change
@@ -29,15 +29,15 @@ import (
// account for startup time is to wait for the container to be healthy before continuing test.
var oracledb = []testutils.Container{testutils.NewContainer().WithContext(
path.Join(".", "testdata", "server"),
).WithName("oracledb").WithExposedPorts("1521:1521").WillWaitForHealth(15 * time.Minute)}
).WithName("oracledb").WithExposedPorts("1521:1521").WillWaitForHealth(5 * time.Minute)}

// This test ensures the collector can connect to an Oracle DB, and properly get metrics. It's not intended to
// test the receiver itself.
func TestOracleDBIntegration(t *testing.T) {
testutils.AssertAllMetricsReceived(t, "all.yaml", "all_metrics_config.yaml",
oracledb, []testutils.CollectorBuilder{
func(collector testutils.Collector) testutils.Collector {
return collector.WithEnv(map[string]string{"ORACLEDB_URL": "oracle://otel:password@localhost:1521/XE"})
return collector.WithEnv(map[string]string{"ORACLEDB_URL": "oracle://otel:password@localhost:1521/xe"})
},
},
)
13 changes: 13 additions & 0 deletions tests/receivers/oracledb/testdata/otlp_exporter.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
exporters:
otlp:
endpoint: "${OTLP_ENDPOINT}"
tls:
insecure: true

service:
telemetry:
logs:
level: debug
pipelines:
metrics:
exporters: [otlp]
5 changes: 2 additions & 3 deletions tests/receivers/oracledb/testdata/resource_metrics/all.yaml
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
resource_metrics:
- attributes:
oracledb.instance.name: localhost:1521/XE
rmfitzpatrick marked this conversation as resolved.
Show resolved Hide resolved
scope_metrics:
- scope_metrics:
- instrumentation_scope:
attributes: {}
name: otelcol/oracledbreceiver
version: <VERSION_FROM_BUILD>
metrics:

Unchanged files with check annotations Beta

// MetricsBuilder provides an interface for scrapers to report metrics while taking care of all the transformations
// required to produce metric representation defined in metadata and user config.
type MetricsBuilder struct {

Check failure on line 7675 in internal/receiver/databricksreceiver/internal/metadata/generated_metrics.go

GitHub Actions / generate-metrics

struct with 3704 pointer bytes could be 3400
metricsBuffer pmetric.Metrics
buildInfo component.BuildInfo
metricDatabricksJobsActiveTotal metricDatabricksJobsActiveTotal
// ResourceBuilder is a helper struct to build resources predefined in metadata.yaml.
// The ResourceBuilder is not thread-safe and must not to be used in multiple goroutines.
type ResourceBuilder struct {

Check failure on line 11 in internal/receiver/databricksreceiver/internal/metadata/generated_resource.go

GitHub Actions / generate-metrics

struct with 16 pointer bytes could be 8
res pcommon.Resource
config ResourceAttributesConfig
}