Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Oracledb discovery #3633

Merged
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
7328559
update bundled test and configs for oraclediscovery
greatestusername Sep 12, 2023
5cd9a57
Merge branch 'signalfx:main' into oracledb-discovery
greatestusername-splunk Sep 12, 2023
169dbfa
update bundled test and configs for oraclediscovery
greatestusername Sep 12, 2023
22740a7
Merge branch 'oracledb-discovery' of https://github.com/greatestusern…
greatestusername Sep 12, 2023
d3eb285
running bundled_test now evaluate metrics matching
greatestusername Sep 13, 2023
e1fdd6e
add working bundled_test
greatestusername Sep 13, 2023
b140890
update fail and partial messages for receiver config variables
greatestusername Sep 14, 2023
d6abba9
order of bundled configs matters
greatestusername Sep 14, 2023
9b6365c
make fmt and add more time
greatestusername Sep 14, 2023
6dd4758
lower durations, add in args for endpoint and service
greatestusername Sep 14, 2023
b8b6e00
reduce time on image pull and add to windows bundle
greatestusername Sep 19, 2023
60b62ba
add to bundle_windows_test
greatestusername Sep 19, 2023
56e903a
Merge branch 'signalfx:main' into oracledb-discovery
greatestusername Sep 19, 2023
35f5ab0
lower times on tests
greatestusername Sep 20, 2023
d36f0c2
address feedback on yaml template and test using env var
greatestusername Sep 20, 2023
224689d
gotta make generate-metrics
greatestusername Sep 20, 2023
3594b53
re-arrange bundled test
greatestusername Sep 21, 2023
f3aa6d1
add oracledb to the discovery readme
greatestusername Sep 20, 2023
b843405
make generate-metrics fmt
greatestusername Sep 21, 2023
09e68ed
Merge branch 'signalfx:main' into oracledb-discovery
greatestusername Sep 22, 2023
a4594da
address feedback
greatestusername Sep 22, 2023
dfff930
Update internal/confmapprovider/discovery/README.md
greatestusername Sep 22, 2023
dddcf85
fix to re-check and other attribute fields
greatestusername Sep 22, 2023
5f0eb86
add new test data for bundled test
greatestusername Sep 22, 2023
3f64271
add generated yaml
greatestusername Sep 22, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
#####################################################################################
# This file is generated by the Splunk Distribution of the OpenTelemetry Collector. #
# #
# It reflects the default configuration bundled in the Collector executable for use #
# in discovery mode (--discovery) and is provided for reference or customization. #
# Please note that any changes made to this file will need to be reconciled during #
# upgrades of the Collector. #
#####################################################################################
# #####################################################################################
# # Do not edit manually! #
# # All changes must be made to associated .tmpl file before running 'make bundle.d'. #
# #####################################################################################
# oracledb:
# enabled: true
# rule:
# docker_observer: type == "container" and any([name, image, command], {# matches "(?i)oracle"}) and not (command matches "splunk.discovery")
# host_observer: type == "hostport" and command matches "(?i)oracle" and not (command matches "splunk.discovery")
# k8s_observer: type == "port" and pod.name matches "(?i)oracle"
# config:
# default:
# endpoint: localhost:1521
# username: otel
# password: password
# service: xe
# status:
# metrics:
# successful:
# - strict: oracledb.cpu_time
# first_only: true
# log_record:
# severity_text: info
# body: oracledb receiver is working!
# statements:
# failed:
# - regexp: "connection refused"
# first_only: true
# log_record:
# severity_text: info
# append_pattern: true
# body: The container is not serving http connections.
# - regexp: "received goaway and there are no active streams"
# first_only: true
# log_record:
# severity_text: info
# append_pattern: true
# body: Unable to connect and scrape metrics.
# partial:
# - regexp: 'invalid username/password'
# first_only: true
# log_record:
# severity_text: info
# append_pattern: true
# body: >-
# Make sure your user credentials are correctly specified using the
# `--set splunk.discovery.receivers.oracledb.config.username="<username>"` and
# `--set splunk.discovery.receivers.oracledb.config.password="<password>"` command or the
# `SPLUNK_DISCOVERY_RECEIVERS_oracledb_CONFIG_username="<username>"` and
# `SPLUNK_DISCOVERY_RECEIVERS_oracledb_CONFIG_password="<password>"` environment variables.
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
#####################################################################################
# Do not edit manually! #
# All changes must be made to associated .tmpl file before running 'make bundle.d'. #
#####################################################################################
#####################################################################################
# Do not edit manually! #
# All changes must be made to associated .tmpl file before running 'make bundle.d'. #
#####################################################################################
oracledb:
enabled: true
rule:
docker_observer: type == "container" and any([name, image, command], {# matches "(?i)oracle"}) and not (command matches "splunk.discovery")
host_observer: type == "hostport" and command matches "(?i)oracle" and not (command matches "splunk.discovery")
k8s_observer: type == "port" and pod.name matches "(?i)oracle"
config:
default:
endpoint: localhost:1521
username: otel
password: password
service: xe
status:
metrics:
successful:
- strict: oracledb.cpu_time
first_only: true
log_record:
severity_text: info
body: oracledb receiver is working!
statements:
failed:
- regexp: "connection refused"
first_only: true
log_record:
severity_text: info
append_pattern: true
body: The container is not serving http connections.
- regexp: "received goaway and there are no active streams"
first_only: true
log_record:
severity_text: info
append_pattern: true
body: Unable to connect and scrape metrics.
partial:
- regexp: 'invalid username/password'
first_only: true
log_record:
severity_text: info
append_pattern: true
body: >-
Make sure your user credentials are correctly specified using the
`--set splunk.discovery.receivers.oracledb.config.username="<username>"` and
`--set splunk.discovery.receivers.oracledb.config.password="<password>"` command or the
`SPLUNK_DISCOVERY_RECEIVERS_oracledb_CONFIG_username="<username>"` and
`SPLUNK_DISCOVERY_RECEIVERS_oracledb_CONFIG_password="<password>"` environment variables.
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
#####################################################################################
# Do not edit manually! #
# All changes must be made to associated .tmpl file before running 'make bundle.d'. #
#####################################################################################
greatestusername marked this conversation as resolved.
Show resolved Hide resolved
oracledb:
greatestusername marked this conversation as resolved.
Show resolved Hide resolved
enabled: true
rule:
docker_observer: type == "container" and any([name, image, command], {# matches "(?i)oracle"}) and not (command matches "splunk.discovery")
host_observer: type == "hostport" and command matches "(?i)oracle" and not (command matches "splunk.discovery")
k8s_observer: type == "port" and pod.name matches "(?i)oracle"
config:
default:
endpoint: localhost:1521
username: otel
password: password
service: xe
status:
metrics:
successful:
- strict: oracledb.cpu_time
first_only: true
log_record:
severity_text: info
body: oracledb receiver is working!
statements:
failed:
rmfitzpatrick marked this conversation as resolved.
Show resolved Hide resolved
- regexp: "connection refused"
first_only: true
log_record:
severity_text: info
append_pattern: true
body: The container is not serving http connections.
- regexp: "received goaway and there are no active streams"
first_only: true
log_record:
severity_text: info
append_pattern: true
body: Unable to connect and scrape metrics.
partial:
- regexp: 'invalid username/password'
first_only: true
log_record:
severity_text: info
append_pattern: true
body: >-
Make sure your user credentials are correctly specified using the
`--set splunk.discovery.receivers.oracledb.config.username="<username>"` and
greatestusername marked this conversation as resolved.
Show resolved Hide resolved
`--set splunk.discovery.receivers.oracledb.config.password="<password>"` command or the
`SPLUNK_DISCOVERY_RECEIVERS_oracledb_CONFIG_username="<username>"` and
`SPLUNK_DISCOVERY_RECEIVERS_oracledb_CONFIG_password="<password>"` environment variables.
2 changes: 2 additions & 0 deletions internal/confmapprovider/discovery/bundle/bundle_gen.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@
//go:generate discoverybundler -r -c -d ../../../../cmd/otelcol/config/collector/config.d.linux/receivers -t bundle.d/receivers/smartagent-collectd-mysql.discovery.yaml.tmpl
//go:generate discoverybundler -r -t bundle.d/receivers/smartagent-collectd-nginx.discovery.yaml.tmpl
//go:generate discoverybundler -r -c -d ../../../../cmd/otelcol/config/collector/config.d.linux/receivers -t bundle.d/receivers/smartagent-collectd-nginx.discovery.yaml.tmpl
//go:generate discoverybundler -r -t bundle.d/receivers/oracledb.discovery.yaml.tmpl
//go:generate discoverybundler -r -c -d ../../../../cmd/otelcol/config/collector/config.d.linux/receivers -t bundle.d/receivers/oracledb.discovery.yaml.tmpl

//go:generate discoverybundler -r -t bundle.d/extensions/docker-observer.discovery.yaml.tmpl
//go:generate discoverybundler -r -c -d ../../../../cmd/otelcol/config/collector/config.d.linux/extensions -t bundle.d/extensions/docker-observer.discovery.yaml.tmpl
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ func TestBundleDir(t *testing.T) {
"bundle.d/receivers/smartagent-collectd-mysql.discovery.yaml",
"bundle.d/receivers/smartagent-collectd-nginx.discovery.yaml",
"bundle.d/receivers/smartagent-postgresql.discovery.yaml",
"bundle.d/receivers/oracledb.discovery.yaml",
}, receivers)

extensions, err := fs.Glob(BundledFS, "bundle.d/extensions/*.discovery.yaml")
Expand Down
80 changes: 80 additions & 0 deletions tests/receivers/oracledb/bundled_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
// Copyright Splunk, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

//go:build integration

package tests

import (
"fmt"
"path"
// "runtime"
"testing"
"time"

"github.com/stretchr/testify/require"

"github.com/signalfx/splunk-otel-collector/tests/testutils"
)

func TestOracledbDockerObserver(t *testing.T) {
// testutils.SkipIfNotContainerTest(t)
// if runtime.GOOS == "darwin" {
// t.Skip("unable to share sockets between mac and d4m vm: https://github.com/docker/for-mac/issues/483#issuecomment-758836836")
// }
tc := testutils.NewTestcase(t)
rmfitzpatrick marked this conversation as resolved.
Show resolved Hide resolved
defer tc.PrintLogsOnFailure()
defer tc.ShutdownOTLPReceiverSink()

_, stop := tc.Containers(
testutils.NewContainer().WithContext(
path.Join(".", "testdata", "server"),
).WithName("oracledb").WithExposedPorts("1521:1521").WillWaitForHealth(45 * time.Minute),
)

defer stop()
fmt.Println("Now starting Otel Collector")

_, shutdown := tc.SplunkOtelCollectorContainer(
"otlp_exporter.yaml",
func(c testutils.Collector) testutils.Collector {
cc := c.(*testutils.CollectorContainer)
cc.Container = cc.Container.WithBinds("/var/run/docker.sock:/var/run/docker.sock:ro")
cc.Container = cc.Container.WillWaitForLogs("Discovering for next")
cc.Container = cc.Container.WithUser(fmt.Sprintf("999:%d", testutils.GetDockerGID(t)))
return cc
},

func(c testutils.Collector) testutils.Collector {
return c.WithEnv(map[string]string{
// runner seems to be slow
"SPLUNK_DISCOVERY_DURATION": "90s",
// confirm that debug logging doesn't affect runtime
"SPLUNK_DISCOVERY_LOG_LEVEL": "debug",
}).WithArgs(
"--discovery",
"--set", "splunk.discovery.receivers.oracledb.config.username=otel",
"--set", "splunk.discovery.receivers.oracledb.config.password=password",
greatestusername marked this conversation as resolved.
Show resolved Hide resolved
"--set", `splunk.discovery.extensions.k8s_observer.enabled=false`,
"--set", `splunk.discovery.extensions.host_observer.enabled=false`,
)
},
)
fmt.Println("Collector started (or not)")

defer shutdown()

expectedResourceMetrics := tc.ResourceMetrics("all.yaml")
require.NoError(t, tc.OTLPReceiverSink.AssertAllMetricsReceived(t, *expectedResourceMetrics, 5*time.Minute))
}
4 changes: 2 additions & 2 deletions tests/receivers/oracledb/oracledb_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,15 +29,15 @@ import (
// account for startup time is to wait for the container to be healthy before continuing test.
var oracledb = []testutils.Container{testutils.NewContainer().WithContext(
path.Join(".", "testdata", "server"),
).WithName("oracledb").WithExposedPorts("1521:1521").WillWaitForHealth(15 * time.Minute)}
).WithName("oracledb").WithExposedPorts("1521:1521").WillWaitForHealth(45 * time.Minute)}
greatestusername marked this conversation as resolved.
Show resolved Hide resolved

// This test ensures the collector can connect to an Oracle DB, and properly get metrics. It's not intended to
// test the receiver itself.
func TestOracleDBIntegration(t *testing.T) {
testutils.AssertAllMetricsReceived(t, "all.yaml", "all_metrics_config.yaml",
oracledb, []testutils.CollectorBuilder{
func(collector testutils.Collector) testutils.Collector {
return collector.WithEnv(map[string]string{"ORACLEDB_URL": "oracle://otel:password@localhost:1521/XE"})
return collector.WithEnv(map[string]string{"ORACLEDB_URL": "oracle://otel:password@localhost:1521/xe"})
},
},
)
Expand Down
13 changes: 13 additions & 0 deletions tests/receivers/oracledb/testdata/otlp_exporter.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
exporters:
otlp:
endpoint: "${OTLP_ENDPOINT}"
tls:
insecure: true

service:
telemetry:
logs:
level: debug
pipelines:
metrics:
exporters: [otlp]
5 changes: 2 additions & 3 deletions tests/receivers/oracledb/testdata/resource_metrics/all.yaml
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
resource_metrics:
- attributes:
oracledb.instance.name: localhost:1521/XE
rmfitzpatrick marked this conversation as resolved.
Show resolved Hide resolved
scope_metrics:
- scope_metrics:
- instrumentation_scope:
attributes: {}
name: otelcol/oracledbreceiver
version: <VERSION_FROM_BUILD>
metrics:
Expand Down
Loading