[geneva] Add support for exporting otlp metrics via user_events on Linux #2113

CodeBlanch · 2024-09-27T18:39:45Z

Changes

GenevaExporter can now export otlp metrics via user_events on Linux

Details

Users have to opt-into this via connection string:

   .AddGenevaMetricExporter(o =>
   {
      o.ConnectionString = "PrivatePreviewEnableOtlpProtobufEncoding=true";
   })

TODOs

Planning to do this stuff as follow-up PRs:

Need to add a CI leg to run these linux integration tests which require certain kernel versions + elevated execution.

Merge requirement checklist

CONTRIBUTING guidelines followed (license requirements, nullable enabled, static analysis, etc.)
Unit tests added/updated
Appropriate CHANGELOG.md files updated for non-trivial changes
Changes in public API reviewed (if applicable)

codecov · 2024-09-27T18:42:26Z

Codecov Report

Attention: Patch coverage is 34.04255% with 31 lines in your changes missing coverage. Please review.

Project coverage is 71.56%. Comparing base (71655ce) to head (ca92300).
Report is 534 commits behind head on main.

Files with missing lines	Patch %	Lines
...ics/Transport/MetricUnixUserEventsDataTransport.cs	0.00%	24 Missing ⚠️
...ry.Exporter.Geneva/Internal/ExporterEventSource.cs	22.22%	7 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2113      +/-   ##
==========================================
- Coverage   73.91%   71.56%   -2.36%     
==========================================
  Files         267      346      +79     
  Lines        9615    13511    +3896     
==========================================
+ Hits         7107     9669    +2562     
- Misses       2508     3842    +1334

Flag	Coverage Δ
unittests-Exporter.Geneva	`58.55% <34.04%> (?)`
unittests-Exporter.InfluxDB	`95.88% <ø> (?)`
unittests-Exporter.Instana	`71.24% <ø> (?)`
unittests-Exporter.OneCollector	`94.34% <ø> (?)`
unittests-Exporter.Stackdriver	`75.73% <ø> (?)`
unittests-Extensions	`88.63% <ø> (?)`
unittests-Extensions.AWS	`83.41% <ø> (?)`
unittests-Extensions.Enrichment	`100.00% <ø> (?)`
unittests-Instrumentation.AWS	`66.66% <ø> (?)`
unittests-Instrumentation.AWSLambda	`74.39% <ø> (?)`
unittests-Instrumentation.AspNet	`76.73% <ø> (?)`
unittests-Instrumentation.AspNetCore	`70.06% <ø> (?)`
unittests-Instrumentation.ConfluentKafka	`14.12% <ø> (?)`
unittests-Instrumentation.ElasticsearchClient	`79.87% <ø> (?)`
unittests-Instrumentation.EntityFrameworkCore	`55.49% <ø> (?)`
unittests-Instrumentation.EventCounters	`76.36% <ø> (?)`
unittests-Instrumentation.GrpcNetClient	`79.61% <ø> (?)`
unittests-Instrumentation.Hangfire	`93.58% <ø> (?)`
unittests-Instrumentation.Http	`73.57% <ø> (?)`
unittests-Instrumentation.Owin	`85.97% <ø> (?)`
unittests-Instrumentation.Process	`100.00% <ø> (?)`
unittests-Instrumentation.Quartz	`78.94% <ø> (?)`
unittests-Instrumentation.Runtime	`97.53% <ø> (?)`
unittests-Instrumentation.SqlClient	`90.55% <ø> (?)`
unittests-Instrumentation.StackExchangeRedis	`69.92% <ø> (?)`
unittests-Instrumentation.Wcf	`78.47% <ø> (?)`
unittests-PersistentStorage	`65.78% <ø> (?)`
unittests-Resources.AWS	`77.85% <ø> (?)`
unittests-Resources.Azure	`83.89% <ø> (?)`
unittests-Resources.Container	`72.41% <ø> (?)`
unittests-Resources.Gcp	`72.54% <ø> (?)`
unittests-Resources.Host	`73.94% <ø> (?)`
unittests-Resources.OperatingSystem	`77.20% <ø> (?)`
unittests-Resources.Process	`100.00% <ø> (?)`
unittests-Resources.ProcessRuntime	`77.08% <ø> (?)`
unittests-Sampler.AWS	`87.74% <ø> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...ternal/Transports/UnixDomainSocketDataTransport.cs	`96.29% <100.00%> (ø)`
...Metrics/OtlpProtobuf/OtlpProtobufMetricExporter.cs	`82.60% <100.00%> (ø)`
...ry.Exporter.Geneva/Internal/ExporterEventSource.cs	`13.51% <22.22%> (+8.75%)`	⬆️
...ics/Transport/MetricUnixUserEventsDataTransport.cs	`0.00% <0.00%> (ø)`

... and 368 files with indirect coverage changes

cijothomas · 2024-09-27T18:50:29Z

src/OpenTelemetry.Exporter.Geneva/CHANGELOG.md

+  [user_events](https://docs.kernel.org/trace/user_events.html) on Linux when
+  OTLP protobuf encoding is enabled via the
+  `PrivatePreviewEnableOtlpProtobufEncoding=true` connection string switch.
+  ([#2113](https://github.com/open-telemetry/opentelemetry-dotnet-contrib/pull/2113))


With this, PrivatePreviewEnableOtlpProtobufEncoding=true is now supported in both Widows and Linux. Windows uses ETW as transport, while Linux uses user-events as transport.

@CodeBlanch can you add this to changelog before merge

Updated CHANGELOG and REDME where this stuff was mentioned

src/OpenTelemetry.Exporter.Geneva/Metrics/OtlpProtobuf/OtlpProtobufMetricExporter.cs

src/OpenTelemetry.Exporter.Geneva/Metrics/Transport/MetricUnixUserEventsDataTransport.cs

reyang · 2024-09-27T19:10:29Z

It is a huge PR with lots of changes, some are related to the title or the PR, some are clean ups and renames, etc. Could you @CodeBlanch break it up to smaller PRs? Thanks!

test/OpenTelemetry.Exporter.Geneva.Tests/UnixUserEventsDataTransportTests.cs

…event-transport

CodeBlanch · 2024-10-10T22:47:50Z

@cijothomas This should be good for a final review. What I was doing was working to get something up and running on an actual distro to validate everything: https://github.com/CodeBlanchOrg/opentelemetry-dotnet-contrib/actions/runs/11249188144/job/31276142590?pr=53#step:7:158

Tested now using WSL2 & ubuntu-22.04

cijothomas · 2024-10-11T00:16:22Z

src/OpenTelemetry.Exporter.Geneva/Internal/Transports/UnixDomainSocketDataTransport.cs

@@ -73,7 +73,7 @@ private bool Connect()
        }
        catch (Exception ex)
        {
-            ExporterEventSource.Log.ExporterException("UDS Connect failed.", ex);
+            ExporterEventSource.Log.TransportException(nameof(UnixDomainSocketDataTransport), "Connection failure", ex);


nit: Include something like "will retry connecting in background", to indicate this may be recovered.

cijothomas · 2024-10-11T00:19:26Z

test/OpenTelemetry.Exporter.Geneva.Tests/UnixUserEventsDataTransportTests.cs

+public class UnixUserEventsDataTransportTests
+{
+    /*
+     * Instructions for running these tests:


Thanks for writing this steps!
My own linux kernel with user-events is broken, so not testing it myself, but will comeback to this as soon as kernel is updated in wsl!

cijothomas

LGTM.
THe main readme on genevaexporter can now say "exports to ETW, UnixSocket or user events"!

cijothomas · 2024-10-11T00:20:38Z

@reyang @rajkumar-rangaraj Would be good to get one more pair of eyes before merge.

reyang · 2024-10-11T03:16:12Z

@reyang @rajkumar-rangaraj Would be good to get one more pair of eyes before merge.

FYI - most likely I won't have bandwidth in the next 4-5 weeks.

rajkumar-rangaraj · 2024-10-11T17:24:57Z

src/OpenTelemetry.Exporter.Geneva/Metrics/Transport/MetricUnixUserEventsDataTransport.cs

+
+            ExporterEventSource.Log.TransportInformation(
+                nameof(MetricUnixUserEventsDataTransport),
+                $"Tracepoint registration for 'otlp_metrics' failed with recoverable result: '{this.metricsTracepoint.RegisterResult}'. Entering running state.");


This message is slightly confusing, we are saying failed with recoverable result and entering running state too.

rajkumar-rangaraj

LGTM, I like the way PerfTracepointListener is implemented. :)

CodeBlanch added 4 commits September 25, 2024 17:50

Support export of metrics via user_events on Linux.

7fe894e

Tweak.

470d0c9

Tweak.

395fe67

Tweaks and test implementation.

7a6f003

CodeBlanch requested a review from a team as a code owner September 27, 2024 18:39

CodeBlanch had a problem deploying to external September 27, 2024 18:39 — with GitHub Actions Failure

github-actions bot assigned cijothomas, CodeBlanch, rajkumar-rangaraj, reyang and utpilla Sep 27, 2024

github-actions bot requested review from cijothomas, rajkumar-rangaraj, reyang and utpilla September 27, 2024 18:40

github-actions bot added infra Infra work - CI/CD, code coverage, linters comp:exporter.geneva Things related to OpenTelemetry.Exporter.Geneva labels Sep 27, 2024

CHANGELOG patch.

df6b1f0

CodeBlanch had a problem deploying to external September 27, 2024 18:46 — with GitHub Actions Failure