[Integration]OTEL Metrics build-in Vega Visualization Support

[YANG-DB]

What is the integration resource ?

Add VEGA visualization support for any type of OTEL Metrics signals:

• Histogram
• Summary
• Gauge

What is the integration source protocol ?

• OTEL metrics signals

What is the integration target protocol ?

• Simple Schema For Observability Metrics

Which agents would you use to ship this data ?

Data Prepper has a metrics pipeline to ship metrics data into opensearch

otel-metrics-pipeline:
  workers: 8
  delay: 3000
  source:
    otel_metrics_source:
      health_check_service: true
      ssl: false
  buffer:
    bounded_blocking:
      buffer_size: 1024 # max number of records the buffer accepts
      batch_size: 1024 # max number of records the buffer drains after each read
  processor:
    - otel_metrics:
        calculate_histogram_buckets: true
        calculate_exponential_histogram_buckets: true
        exponential_histogram_max_allowed_scale: 10
        flatten_attributes: false
  sink:
    - opensearch:
        hosts: ["https://opensearch-node1:9200"]
        username: "admin"
        password: "***"
        insecure: true
        index_type: custom
        index: otel-metrics-%{yyyy.MM.dd}
        bulk_size: 4

See opentelemetry demo application data prepper's pipeline

Which Dashboards would you be using ?

Here are few potential rendering examples:

---

Do you have any additional context?

Screen.Recording.2024-03-07.at.10.44.22.AM.mov

---

• vega related issue
• Vega example for such binned histogram