[chore] [receiver/datadog] Add support for v1 series #33957

carrieedwards · 2024-07-09T00:23:41Z

Description:
This PR adds support for V1 series, as well as batches the metrics by resource, scope, and datapoint attributes. The batching code will also be required for future PRs which will add support for v2 series endpoints, service checks, and sketches.

Follow up of #33631 and #33922.

The full version of the code can be found in the cedwards/datadog-metrics-receiver-full branch, or in Grafana Alloy: https://github.com/grafana/alloy/tree/main/internal/etc/datadogreceiver

Link to tracking Issue:
#18278

Testing:
Unit tests, as well as an end-to-end test, have been added.

linux-foundation-easycla · 2024-07-09T00:23:45Z

The committers listed above are authorized under a signed CLA.

✅ login: fedetorres93 / name: Federico Torres (d5f7724)
✅ login: carrieedwards / name: Carrie Edwards (bb6240c, 9f9d651, 7344265, e42f62f, 4010573, 0fedbe2, 57c35ad, ee4e3e4, 83dd84e, c8f8665)

receiver/datadogreceiver/batcher.go

receiver/datadogreceiver/metrics_translator.go

sh0rez · 2024-07-09T10:13:23Z

receiver/datadogreceiver/metrics_translator.go

+			metric.Sum().SetAggregationTemporality(pmetric.AggregationTemporalityDelta)
+			dps = metric.Sum().DataPoints()
+		default:
+			// Type is unset/unspecified


lets record the dp type as a counter (count/gauge/rate/unknown). can be done in a later pr

Are you talking about the entire switch case? I believe its better if metrics are properly identified from the start.

oh I meant it would be nice to have this receiver itself export a metric like otelcol_datadog_series_total{type="count|gauge|rate|undefined"}

receiver/datadogreceiver/metrics_translator.go

sh0rez · 2024-07-09T10:17:03Z

receiver/datadogreceiver/metrics_translator.go

+			ts, ok := mt.streamHasTimestamp(stream)
+			if ok {
+				dp.SetStartTimestamp(ts)
+			}


Suggested change

ts, ok := mt.streamHasTimestamp(stream)

if ok {

dp.SetStartTimestamp(ts)

}

if ts, ok := mt.streamHasTimestamp(stream); ok {

dp.SetStartTimestamp(ts)

}

jpkrohling · 2024-07-11T08:53:42Z

receiver/datadogreceiver/batcher.go

+	"github.com/open-telemetry/opentelemetry-collector-contrib/internal/exp/metrics/identity"
+)
+
+type Batcher struct {


this should probably be inside an internal package, otherwise it's part of this component's public API. Unless you meant to expose this to external consumers?

jpkrohling · 2024-07-11T08:53:58Z

receiver/datadogreceiver/batcher.go

+// Dimensions stores the properties of the series that are needed in order
+// to unique identify the series. This is needed in order to batch metrics by
+// resource, scope, and datapoint attributes
+type Dimensions struct {


jpkrohling · 2024-07-11T08:58:43Z

receiver/datadogreceiver/receiver.go

+	}
+
+	w.WriteHeader(http.StatusAccepted)
+	_, err = w.Write([]byte("OK"))


this err should probably be ignored (_)

Co-authored by: Jesus Vazquez <[email protected]>:

Co-authored by: Federico Torres <[email protected]>:

jpkrohling

@jesusvazquez shared in private that they are committing to moving the items that would be part of the public API into an internal package. I think it's fine, especially if this happens within the next two weeks, which is when the next Collector should be released.

…ackage (#34160) **Description:** This PR is a follow-up to #33957. It refactors the Datadog receiver files to remove internal methods and structures from the public API and into an internal directory. **Link to tracking Issue:** #18278 **Testing:** This is a refactor, so no new unit tests have been added.

**Description:** This PR adds support for Datadog V2 series. Follow up of #33631 and #33957. The full version of the code can be found in the `cedwards/datadog-metrics-receiver-full` branch, or in Grafana Alloy: https://github.com/grafana/alloy/tree/main/internal/etc/datadogreceiver **Link to tracking Issue:** #18278 **Testing:** Unit tests, as well as an end-to-end test, have been added.

Description: This PR adds support for Datadog Service Checks. Follow up of #33631 , #33957 and #34180. The full version of the code can be found in the cedwards/datadog-metrics-receiver-full branch, or in Grafana Alloy: https://github.com/grafana/alloy/tree/main/internal/etc/datadogreceiver Link to tracking Issue: #18278 Testing: Unit tests, as well as an end-to-end test, have been added. --------- Signed-off-by: alexgreenbank <[email protected]> Co-authored-by: Carrie Edwards <[email protected]> Co-authored-by: Juraci Paixão Kröhling <[email protected]>

**Description:** This PR adds support for translating Datadog sketches into Exponential Histograms. Follow up of #33631, #33957 and #34180. The full version of the code can be found in the `cedwards/datadog-metrics-receiver-full` branch, or in Grafana Alloy: https://github.com/grafana/alloy/tree/main/internal/etc/datadogreceiver **Link to tracking Issue:** #18278 **Testing:** Unit tests, as well as an end-to-end test, have been added. --------- Signed-off-by: Federico Torres <[email protected]> Signed-off-by: György Krajcsovits <[email protected]> Co-authored-by: Federico Torres <[email protected]> Co-authored-by: György Krajcsovits <[email protected]>

…34180) **Description:** This PR adds support for Datadog V2 series. Follow up of open-telemetry#33631 and open-telemetry#33957. The full version of the code can be found in the `cedwards/datadog-metrics-receiver-full` branch, or in Grafana Alloy: https://github.com/grafana/alloy/tree/main/internal/etc/datadogreceiver **Link to tracking Issue:** open-telemetry#18278 **Testing:** Unit tests, as well as an end-to-end test, have been added.

…etry#34474) Description: This PR adds support for Datadog Service Checks. Follow up of open-telemetry#33631 , open-telemetry#33957 and open-telemetry#34180. The full version of the code can be found in the cedwards/datadog-metrics-receiver-full branch, or in Grafana Alloy: https://github.com/grafana/alloy/tree/main/internal/etc/datadogreceiver Link to tracking Issue: open-telemetry#18278 Testing: Unit tests, as well as an end-to-end test, have been added. --------- Signed-off-by: alexgreenbank <[email protected]> Co-authored-by: Carrie Edwards <[email protected]> Co-authored-by: Juraci Paixão Kröhling <[email protected]>

**Description:** This PR adds support for translating Datadog sketches into Exponential Histograms. Follow up of open-telemetry#33631, open-telemetry#33957 and open-telemetry#34180. The full version of the code can be found in the `cedwards/datadog-metrics-receiver-full` branch, or in Grafana Alloy: https://github.com/grafana/alloy/tree/main/internal/etc/datadogreceiver **Link to tracking Issue:** open-telemetry#18278 **Testing:** Unit tests, as well as an end-to-end test, have been added. --------- Signed-off-by: Federico Torres <[email protected]> Signed-off-by: György Krajcsovits <[email protected]> Co-authored-by: Federico Torres <[email protected]> Co-authored-by: György Krajcsovits <[email protected]>

…etry#34474) Description: This PR adds support for Datadog Service Checks. Follow up of open-telemetry#33631 , open-telemetry#33957 and open-telemetry#34180. The full version of the code can be found in the cedwards/datadog-metrics-receiver-full branch, or in Grafana Alloy: https://github.com/grafana/alloy/tree/main/internal/etc/datadogreceiver Link to tracking Issue: open-telemetry#18278 Testing: Unit tests, as well as an end-to-end test, have been added. --------- Signed-off-by: alexgreenbank <[email protected]> Co-authored-by: Carrie Edwards <[email protected]> Co-authored-by: Juraci Paixão Kröhling <[email protected]>

**Description:** This PR adds support for translating Datadog sketches into Exponential Histograms. Follow up of open-telemetry#33631, open-telemetry#33957 and open-telemetry#34180. The full version of the code can be found in the `cedwards/datadog-metrics-receiver-full` branch, or in Grafana Alloy: https://github.com/grafana/alloy/tree/main/internal/etc/datadogreceiver **Link to tracking Issue:** open-telemetry#18278 **Testing:** Unit tests, as well as an end-to-end test, have been added. --------- Signed-off-by: Federico Torres <[email protected]> Signed-off-by: György Krajcsovits <[email protected]> Co-authored-by: Federico Torres <[email protected]> Co-authored-by: György Krajcsovits <[email protected]>

carrieedwards requested a review from a team July 9, 2024 00:23

carrieedwards requested review from jpkrohling and MovieStoreGuy as code owners July 9, 2024 00:23

github-actions bot assigned songy23 Jul 9, 2024

github-actions bot added receiver/datadog testbed labels Jul 9, 2024

github-actions bot requested review from boostchicken and gouthamve July 9, 2024 00:24

jpkrohling assigned jpkrohling and unassigned songy23 Jul 9, 2024

sh0rez suggested changes Jul 9, 2024

View reviewed changes

carrieedwards force-pushed the cedwards/datadog-v1-metrics branch from 3dc9516 to e9b6ca8 Compare July 9, 2024 19:53

carrieedwards changed the title ~~[receiver/datadog] Add support for v1 series~~ [chore] [receiver/datadog] Add support for v1 series Jul 10, 2024

jpkrohling reviewed Jul 11, 2024

View reviewed changes

carrieedwards force-pushed the cedwards/datadog-v1-metrics branch from ca3008c to 0637b87 Compare July 15, 2024 16:49

carrieedwards and others added 11 commits July 17, 2024 07:33

Add logic to batch metrics by attribute types

4010573

Add support for V1 series endpoint

83dd84e

Co-authored by: Jesus Vazquez <[email protected]>:

Add test helper functions

d5f7724

Add tests for V1 series endpoint

0fedbe2

Co-authored by: Jesus Vazquez <[email protected]>:

Add tests for batching by attributes

e42f62f

Update vendoring

ee4e3e4

Add end-to-end test for v1 series

c8f8665

Co-authored by: Federico Torres <[email protected]>:

Refactoring of batching and v1 series translation code

9f9d651

Fix linting

7344265

Ignore err on writing header for V1 series

57c35ad

Updating go.mod

bb6240c

carrieedwards force-pushed the cedwards/datadog-v1-metrics branch from 0637b87 to bb6240c Compare July 17, 2024 14:46

jpkrohling approved these changes Jul 17, 2024

View reviewed changes

jpkrohling merged commit f3fdea3 into open-telemetry:main Jul 17, 2024
154 checks passed

github-actions bot added this to the next release milestone Jul 17, 2024

carrieedwards mentioned this pull request Jul 18, 2024

[chore] [receiver/datadog] Refactor translation files into internal package #34160

Merged

carrieedwards mentioned this pull request Jul 19, 2024

[chore] [receiver/datadog] Add support for v2 series #34180

Merged

alexgreenbank mentioned this pull request Aug 7, 2024

[chore] [receiver/datadog] Add support for Service Checks #34474

Merged

carrieedwards mentioned this pull request Aug 13, 2024

[receiver/datadog] Add support for sketches #34662

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[chore] [receiver/datadog] Add support for v1 series #33957

[chore] [receiver/datadog] Add support for v1 series #33957

carrieedwards commented Jul 9, 2024

linux-foundation-easycla bot commented Jul 9, 2024 •

edited

Loading

sh0rez Jul 9, 2024

jesusvazquez Jul 10, 2024

sh0rez Jul 10, 2024

sh0rez Jul 9, 2024

jpkrohling Jul 11, 2024

jpkrohling Jul 11, 2024

jpkrohling Jul 11, 2024

jpkrohling left a comment

[chore] [receiver/datadog] Add support for v1 series #33957

[chore] [receiver/datadog] Add support for v1 series #33957

Conversation

carrieedwards commented Jul 9, 2024

linux-foundation-easycla bot commented Jul 9, 2024 • edited Loading

sh0rez Jul 9, 2024

Choose a reason for hiding this comment

jesusvazquez Jul 10, 2024

Choose a reason for hiding this comment

sh0rez Jul 10, 2024

Choose a reason for hiding this comment

sh0rez Jul 9, 2024

Choose a reason for hiding this comment

jpkrohling Jul 11, 2024

Choose a reason for hiding this comment

jpkrohling Jul 11, 2024

Choose a reason for hiding this comment

jpkrohling Jul 11, 2024

Choose a reason for hiding this comment

jpkrohling left a comment

Choose a reason for hiding this comment

linux-foundation-easycla bot commented Jul 9, 2024 •

edited

Loading