Skip to content

Latest commit

 

History

History
41 lines (35 loc) · 3.39 KB

README.md

File metadata and controls

41 lines (35 loc) · 3.39 KB
Raw

OpenTelemetry Collector Testbed

Testbed is a controlled environment and tools for conducting end-to-end tests for the Otel Collector, including reproducible short-term benchmarks, correctness tests, long-running stability tests and maximum load stress tests.

Usage

For each type of tests that should have a summary report create a new directory and then a test suite function which utilizes *testing.M. This function should delegate all functionality to testbed.DoTestMain supplying a global instance of testbed.TestResultsSummary to it.

Each test case within the suite should create a testbed.TestCase and supply implementations of each of the various interfaces the NewTestCase function takes as parameters.

Pluggable Test Components

  • DataProvider - Generates test data to send to receiver under test.
    • PerfTestDataProvider - Implementation of the DataProvider for use in performance tests. Tracing IDs are based on the incremented batch and data items counters.
    • GoldenDataProvider - Implementation of DataProvider for use in correctness tests. Provides data from the "Golden" dataset generated using pairwise combinatorial testing techniques.
  • DataSender - Sends data to the collector instance under test.
    • JaegerGRPCDataSender - Implementation of DataSender which sends to jaeger receiver.
    • OCTraceDataSender - Implementation of DataSender which sends to opencensus receiver.
    • OCMetricsDataSender - Implementation of DataSender which sends to opencensus receiver.
    • OTLPTraceDataSender - Implementation of DataSender which sends to otlp receiver.
    • OTLPMetricsDataSender - Implementation of DataSender which sends to otlp receiver.
    • ZipkinDataSender - Implementation of DataSender which sends to zipkin receiver.
  • DataReceiver - Receives data from the collector instance under test and stores it for use in test assertions.
    • OCDataReceiver - Implementation of DataReceiver which receives data from opencensus exporter.
    • JaegerDataReceiver - Implementation of DataReceiver which receives data from jaeger exporter.
    • OTLPDataReceiver - Implementation of DataReceiver which receives data from otlp exporter.
    • ZipkinDataReceiver - Implementation of DataReceiver which receives data from zipkin exporter.
  • OtelcolRunner - Configures, starts and stops one or more instances of otelcol which will be the subject of testing being executed.
    • ChildProcess - Implementation of OtelcolRunner runs a single otelcol as a child process on the same machine as the test executor.
    • InProcessCollector - Implementation of OtelcolRunner runs a single otelcol as a go routine within the same process as the test executor.
  • TestCaseValidator - Validates and reports on test results.
    • PerfTestValidator - Implementation of TestCaseValidator for test suites using PerformanceResults for summarizing results.
    • CorrectnessTestValidator - Implementation of TestCaseValidator for test suites using CorrectnessResults for summarizing results.
  • TestResultsSummary - Records itemized test case results plus a summary of one category of testing.
    • PerformanceResults - Implementation of TestResultsSummary with fields suitable for reporting performance test results.
    • CorrectnessResults - Implementation of TestResultsSummary with fields suitable for reporting data translation correctness test results.