From e126ac05256841f6dc595f55d4aaf494d10cf91e Mon Sep 17 00:00:00 2001 From: Matthew Kocher Date: Tue, 31 Oct 2023 22:29:07 +0000 Subject: [PATCH 01/14] Add blog post This blog post details an experience report for adopting OTel for metrics in Cloud Foundry. Co-authored-by: Carson Long --- content/en/blog/2023/cloud-foundry.md | 133 ++++++++++++++++++++++++++ static/refcache.json | 20 ++++ 2 files changed, 153 insertions(+) create mode 100644 content/en/blog/2023/cloud-foundry.md diff --git a/content/en/blog/2023/cloud-foundry.md b/content/en/blog/2023/cloud-foundry.md new file mode 100644 index 000000000000..3897f42b4349 --- /dev/null +++ b/content/en/blog/2023/cloud-foundry.md @@ -0,0 +1,133 @@ +--- +title: 'Experience Report: Adopting OpenTelemetry for Metrics in Cloud Foundry' +linkTitle: OpenTelemetry in Cloud Foundry +date: 2023-10-31 +author: >- + [Matthew Kocher](https://github.com/mkocher) (VMware) and [Carson + Long](https://github.com/ctlong) (VMware) +cSpell:ignore: Kocher unscalable +--- + +[Cloud Foundry](https://www.cloudfoundry.org/) recently integrated the +OpenTelemetry Collector for metrics egress and we learned a lot along the way. +We're excited about what the integration offers today and all the possibilities +it opens up for us. + +## What we were looking for + +Cloud Foundry is a large multi-tenant platform as a service that runs 12-factor +applications. Cloud Foundry platform engineering teams usually run 4 to 8 Cloud +Foundry deployments running thousands of applications and hundreds of thousands +of containers. There are many users of Cloud Foundry, and our goal when it comes +to metrics is to enable platform engineering teams to integrate with their +monitoring tool of choice, not to dictate a single solution. + +Historically adding support for a new monitoring tool was done via our own +Firehose API which required writing a new “Nozzle” for each tool, a high barrier +to entry. The Firehose API was also found to have inherent performance +limitations when being used for high volumes of logs and metrics. With an +unscalable API, and an unscalable approach to integrations, we started looking +to replace the way metrics egress from Cloud Foundry. + +We were looking for a scalable way to egress metrics from hundreds of VMs and +thousands of containers with no bottlenecks or single points of failure. We also +wanted a real community working to support the many monitoring tools that +platform engineering teams might want to use, so we could get out of the +business of writing and maintaining custom integrations. + +## What we evaluated + +It was easy to align on adding an agent to each VM sending metrics directly to +monitoring tools. Within Cloud Foundry we had already seen that this approach +worked well for logs egress, eliminating bottlenecks and single points of +failure. + +We evaluated several popular metric egress solutions. Besides OpenTelemetry, we +also looked seriously at [Fluent Bit](https://fluentbit.io/) and +[Telegraf](https://www.influxdata.com/time-series-platform/telegraf/). + +For every solution we considered, we looked at them in following ways: + +- Flexibility: How many monitoring tools can we send to? +- Performance: How efficient is the CPU and memory usage? +- Community: How engaged is the community? What languages/tools do they use? +- Deployment: Can it be deployed as a BOSH job and work within the constraints + of a Cloud Foundry deployment? Is it possible to hot reload the configuration? + +## Why we chose OpenTelemetry + +When we looked at Fluent Bit we found that it is being written in C, which may +offer some performance benefits, but we primarily write in Go. We discarded +Fluent Bit early because our ability to contribute to the codebase would be +minimal, which we worried would limit us in the future. + +We then looked more seriously at Telegraf and OpenTelemetry. Both are written in +Go, so we were good there. Telegraf has a more limited community, and does not +offer a customizable build process. We liked that OpenTelemetry allows us to +build a Collector with a curated selection of our own components and community +components. + +Additionally, when looking at OpenTelemetry we found that many tools and vendors +were adding native support for the OTLP protocol. This caused us to be confident +in adopting OpenTelemetry Collector as a widely adopted first-party +implementation of the OTLP protocol. + +We +[proposed adding the OpenTelemetry Collector to Cloud Foundry in an RFC](https://github.com/cloudfoundry/community/blob/0365df129e52ae7b784957a5569b16b7e133f97e/toc/rfc/rfc-0018-aggregate-metric-egress-with-opentelemetry-collector.md), +and solicited our community's feedback. It was accepted on 07-07-2023 and we got +to work. + +## How we integrated OpenTelemetry with our current metric system + +In Cloud Foundry we have a suite of agents responsible for forwarding logs and +metrics. The front door is a “Forwarder Agent” which accepts logs and metrics in +our own custom format and multiplexes them to the other agents. + +We added support in our agent to translate metrics to OTLP and forward them onto +the OpenTelemetry Collector. This required +[just 200 lines of Go code](https://github.com/cloudfoundry/loggregator-agent-release/blob/1fd275fe85d6190bac73dc1195007cc8726c1871/src/pkg/otelcolclient/otelcolclient.go#L108-L153), +with many of those lines simply closing curly brackets. In writing this code we +had to think about how to take our existing data model and translate it into the +OpenTelemetry protobuf data model. We found OpenTelemetry to have considered +many edge cases we had encountered in the past, and plenty we had not yet +encountered. Someday we hope to understand how to use Scopes and Resources +effectively. + +Our implementation worked well, though we found the OpenTelemetry Collector to +be using large amounts of CPU. We had done the simplest thing to start with, and +were only sending one metric per gRPC request. Our Collector's Resource +utilization dropped drastically when we +[added batching](https://github.com/cloudfoundry/loggregator-agent-release/pull/396). + +## How it works for Cloud Foundry platform engineering teams + +Platform Engineering teams can now +[optionally provide an OpenTelemetry Collector exporter configuration when deploying Cloud Foundry](https://github.com/cloudfoundry/cf-deployment/blob/fcde539a81a6b091a25d06992e16bb2fb641a329/operations/experimental/add-otel-collector.yml). +Every VM in the deployment will then run a Collector that uses that +configuration to forward metrics to the specified monitoring tools. + +We’ve started small and currently only support the OTLP and Prometheus +exporters. However, we’re looking forward to hearing what additional exporters +platform engineering teams want to use, and adding them to our OpenTelemetry +Collector distribution. + +## What's next + +When we took on this track of work we focused exclusively on metrics. There’s a +clear path to add logs as another supported signal type for our OpenTelemetry +Collector integration. We do not yet support traces as a built-in signal in +Cloud Foundry, but we are excited about the possibilities that traces could +offer for both platform components and applications running on the platform. + +Our current OpenTelemetry Collector integration offers what we call “aggregate +drains”, which egress signals generated by the platform or by applications +running on the platform. We would also like to offer “application drains”, which +would only egress signals from individual applications to the monitoring tools +of the application teams’ choice. This involves complex routing and frequent +creation and removal of exporters, which may require new OpenTelemetry Collector +features. + +If we can accomplish those goals, we could replace our entire agent suite with a +single OpenTelemetry Collector running on each VM. Those Collectors would handle +logs, metrics and traces for our system components as well as every application +running on the platform. diff --git a/static/refcache.json b/static/refcache.json index 9131ec672809..7e354705c3c6 100644 --- a/static/refcache.json +++ b/static/refcache.json @@ -2103,6 +2103,10 @@ "StatusCode": 200, "LastSeen": "2023-06-30T08:43:38.289125-04:00" }, + "https://github.com/cloudfoundry/loggregator-agent-release/pull/396": { + "StatusCode": 200, + "LastSeen": "2023-10-31T22:49:01.880645499Z" + }, "https://github.com/codeboten": { "StatusCode": 200, "LastSeen": "2023-09-19T15:39:47.072063934Z" @@ -2123,6 +2127,10 @@ "StatusCode": 200, "LastSeen": "2023-06-30T11:48:47.65097-04:00" }, + "https://github.com/ctlong": { + "StatusCode": 200, + "LastSeen": "2023-10-31T22:49:01.035555602Z" + }, "https://github.com/damemi": { "StatusCode": 200, "LastSeen": "2023-06-30T09:26:34.635172-04:00" @@ -2379,6 +2387,10 @@ "StatusCode": 200, "LastSeen": "2023-06-30T08:31:59.846931-04:00" }, + "https://github.com/mkocher": { + "StatusCode": 200, + "LastSeen": "2023-10-31T22:49:00.168214153Z" + }, "https://github.com/mlocati/docker-php-extension-installer": { "StatusCode": 200, "LastSeen": "2023-06-30T08:48:11.692308-04:00" @@ -5687,6 +5699,10 @@ "StatusCode": 200, "LastSeen": "2023-09-15T16:58:32.417789+02:00" }, + "https://www.cloudfoundry.org/": { + "StatusCode": 206, + "LastSeen": "2023-10-31T22:49:01.12971518Z" + }, "https://www.cloudscaleinc.com": { "StatusCode": 200, "LastSeen": "2023-06-30T16:24:41.542459-04:00" @@ -5875,6 +5891,10 @@ "StatusCode": 206, "LastSeen": "2023-06-29T18:36:51.442819-04:00" }, + "https://www.influxdata.com/time-series-platform/telegraf/": { + "StatusCode": 206, + "LastSeen": "2023-10-31T22:49:01.321162658Z" + }, "https://www.instana.com": { "StatusCode": 206, "LastSeen": "2023-06-30T16:25:35.002614-04:00" From 9a78517d7e8b65cda3fea42b1c2ccb31da6a935c Mon Sep 17 00:00:00 2001 From: Carson Long Date: Wed, 1 Nov 2023 20:33:48 +0000 Subject: [PATCH 02/14] Add boiler plate integration heading --- content/en/blog/2023/cloud-foundry.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/content/en/blog/2023/cloud-foundry.md b/content/en/blog/2023/cloud-foundry.md index 3897f42b4349..407e357e8738 100644 --- a/content/en/blog/2023/cloud-foundry.md +++ b/content/en/blog/2023/cloud-foundry.md @@ -8,6 +8,19 @@ author: >- cSpell:ignore: Kocher unscalable --- +{{% alert color="primary" %}} As part of +[OpenTelemetry's vision](http://localhost:1313/community/mission/#telemetry-should-be-built-in), +we believe that high-quality telemetry can be built in to the entire software +stack. Therefore we are excited, that there are more and more +[libraries, services and apps, that integrate OpenTelemetry](/ecosystem/integrations/) +to provide built in observability to their users.\_ + +_In this and future blog post, we will spot light these integrations to you! If +you want your integration to be listed and to be highlighted in a blog post as +well, +[reach out](https://github.com/open-telemetry/opentelemetry.io/issues/new)!_ +{{% /alert %}} + [Cloud Foundry](https://www.cloudfoundry.org/) recently integrated the OpenTelemetry Collector for metrics egress and we learned a lot along the way. We're excited about what the integration offers today and all the possibilities From e37ccaeacdd5ca0b438de678bf1d574aa48de049 Mon Sep 17 00:00:00 2001 From: Carson Long Date: Thu, 2 Nov 2023 13:54:42 -0700 Subject: [PATCH 03/14] Update alert with suggestions --- content/en/blog/2023/cloud-foundry.md | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/content/en/blog/2023/cloud-foundry.md b/content/en/blog/2023/cloud-foundry.md index 407e357e8738..1c907d7c2b14 100644 --- a/content/en/blog/2023/cloud-foundry.md +++ b/content/en/blog/2023/cloud-foundry.md @@ -8,17 +8,18 @@ author: >- cSpell:ignore: Kocher unscalable --- -{{% alert color="primary" %}} As part of -[OpenTelemetry's vision](http://localhost:1313/community/mission/#telemetry-should-be-built-in), -we believe that high-quality telemetry can be built in to the entire software -stack. Therefore we are excited, that there are more and more -[libraries, services and apps, that integrate OpenTelemetry](/ecosystem/integrations/) -to provide built in observability to their users.\_ - -_In this and future blog post, we will spot light these integrations to you! If -you want your integration to be listed and to be highlighted in a blog post as -well, -[reach out](https://github.com/open-telemetry/opentelemetry.io/issues/new)!_ +{{% alert title="OTel community note" color="primary" %}} As part of + +[OpenTelemetry's vision](/community/mission/#telemetry-should-be-built-in), we +believe that high-quality telemetry can be built in to the entire software +stack. We are excited that more and more +[libraries, services, and apps are integrating with OpenTelemetry](/ecosystem/integrations/) +to provide built-in observability to their users. + +In this and future blog posts, we will spotlight such integrations. If you want +your integration to be listed and to be highlighted in a blog post as well, +[reach out](https://github.com/open-telemetry/opentelemetry.io/issues/new)! + {{% /alert %}} [Cloud Foundry](https://www.cloudfoundry.org/) recently integrated the From dca0375ef92f23b758764554801b782a1372233d Mon Sep 17 00:00:00 2001 From: Carson Long Date: Mon, 6 Nov 2023 08:16:02 -0800 Subject: [PATCH 04/14] Update content/en/blog/2023/cloud-foundry.md Replace OTel Community alert with new badge Co-authored-by: Patrice Chalin --- content/en/blog/2023/cloud-foundry.md | 14 +------------- 1 file changed, 1 insertion(+), 13 deletions(-) diff --git a/content/en/blog/2023/cloud-foundry.md b/content/en/blog/2023/cloud-foundry.md index 1c907d7c2b14..0960aa3dd37d 100644 --- a/content/en/blog/2023/cloud-foundry.md +++ b/content/en/blog/2023/cloud-foundry.md @@ -8,19 +8,7 @@ author: >- cSpell:ignore: Kocher unscalable --- -{{% alert title="OTel community note" color="primary" %}} As part of - -[OpenTelemetry's vision](/community/mission/#telemetry-should-be-built-in), we -believe that high-quality telemetry can be built in to the entire software -stack. We are excited that more and more -[libraries, services, and apps are integrating with OpenTelemetry](/ecosystem/integrations/) -to provide built-in observability to their users. - -In this and future blog posts, we will spotlight such integrations. If you want -your integration to be listed and to be highlighted in a blog post as well, -[reach out](https://github.com/open-telemetry/opentelemetry.io/issues/new)! - -{{% /alert %}} +{{< blog/integration-badge >}} [Cloud Foundry](https://www.cloudfoundry.org/) recently integrated the OpenTelemetry Collector for metrics egress and we learned a lot along the way. From 59d18a840b903ca89742a1d6775c400c86edb342 Mon Sep 17 00:00:00 2001 From: Carson Long Date: Mon, 6 Nov 2023 15:01:43 -0600 Subject: [PATCH 05/14] Update content/en/blog/2023/cloud-foundry.md Update date Co-authored-by: Severin Neumann --- content/en/blog/2023/cloud-foundry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/blog/2023/cloud-foundry.md b/content/en/blog/2023/cloud-foundry.md index 0960aa3dd37d..00e39b3f6782 100644 --- a/content/en/blog/2023/cloud-foundry.md +++ b/content/en/blog/2023/cloud-foundry.md @@ -1,7 +1,7 @@ --- title: 'Experience Report: Adopting OpenTelemetry for Metrics in Cloud Foundry' linkTitle: OpenTelemetry in Cloud Foundry -date: 2023-10-31 +date: 2023-11-08 author: >- [Matthew Kocher](https://github.com/mkocher) (VMware) and [Carson Long](https://github.com/ctlong) (VMware) From 66bd3c47779badea301e38c696b9a301e26196b0 Mon Sep 17 00:00:00 2001 From: Carson Long Date: Tue, 7 Nov 2023 16:26:16 -0600 Subject: [PATCH 06/14] Update content/en/blog/2023/cloud-foundry.md Co-authored-by: Severin Neumann --- content/en/blog/2023/cloud-foundry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/blog/2023/cloud-foundry.md b/content/en/blog/2023/cloud-foundry.md index 00e39b3f6782..8692f9366e03 100644 --- a/content/en/blog/2023/cloud-foundry.md +++ b/content/en/blog/2023/cloud-foundry.md @@ -11,7 +11,7 @@ cSpell:ignore: Kocher unscalable {{< blog/integration-badge >}} [Cloud Foundry](https://www.cloudfoundry.org/) recently integrated the -OpenTelemetry Collector for metrics egress and we learned a lot along the way. +[OpenTelemetry Collector](/docs/collector/) for metrics egress and we learned a lot along the way. We're excited about what the integration offers today and all the possibilities it opens up for us. From 1d1a38807d68110e9d07187f14c1adc8482d73bf Mon Sep 17 00:00:00 2001 From: Carson Long Date: Tue, 7 Nov 2023 16:27:24 -0600 Subject: [PATCH 07/14] Update content/en/blog/2023/cloud-foundry.md Co-authored-by: Severin Neumann --- content/en/blog/2023/cloud-foundry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/blog/2023/cloud-foundry.md b/content/en/blog/2023/cloud-foundry.md index 8692f9366e03..a7cf2f8d6ac4 100644 --- a/content/en/blog/2023/cloud-foundry.md +++ b/content/en/blog/2023/cloud-foundry.md @@ -69,7 +69,7 @@ offer a customizable build process. We liked that OpenTelemetry allows us to build a Collector with a curated selection of our own components and community components. -Additionally, when looking at OpenTelemetry we found that many tools and vendors +Additionally, when looking at OpenTelemetry we found that many [tools](/ecosystem/integrations/) and [vendors](/ecosystem/vendors/) were adding native support for the OTLP protocol. This caused us to be confident in adopting OpenTelemetry Collector as a widely adopted first-party implementation of the OTLP protocol. From 2283c3f78dd94dfd3c7ce043ec933121df32c62b Mon Sep 17 00:00:00 2001 From: Carson Long Date: Tue, 7 Nov 2023 16:27:47 -0600 Subject: [PATCH 08/14] Update content/en/blog/2023/cloud-foundry.md Co-authored-by: Severin Neumann --- content/en/blog/2023/cloud-foundry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/blog/2023/cloud-foundry.md b/content/en/blog/2023/cloud-foundry.md index a7cf2f8d6ac4..b6fd9a72244e 100644 --- a/content/en/blog/2023/cloud-foundry.md +++ b/content/en/blog/2023/cloud-foundry.md @@ -70,7 +70,7 @@ build a Collector with a curated selection of our own components and community components. Additionally, when looking at OpenTelemetry we found that many [tools](/ecosystem/integrations/) and [vendors](/ecosystem/vendors/) -were adding native support for the OTLP protocol. This caused us to be confident +were adding native support for the [OpenTelemetry Protocol](/docs/specs/otlp/) (OTLP). This caused us to be confident in adopting OpenTelemetry Collector as a widely adopted first-party implementation of the OTLP protocol. From 223368e50f704f4cdc6b6a9059a4a7bbc4667e70 Mon Sep 17 00:00:00 2001 From: Carson Long Date: Tue, 7 Nov 2023 16:28:06 -0600 Subject: [PATCH 09/14] Update content/en/blog/2023/cloud-foundry.md Co-authored-by: Severin Neumann --- content/en/blog/2023/cloud-foundry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/blog/2023/cloud-foundry.md b/content/en/blog/2023/cloud-foundry.md index b6fd9a72244e..6fdcf5164b65 100644 --- a/content/en/blog/2023/cloud-foundry.md +++ b/content/en/blog/2023/cloud-foundry.md @@ -72,7 +72,7 @@ components. Additionally, when looking at OpenTelemetry we found that many [tools](/ecosystem/integrations/) and [vendors](/ecosystem/vendors/) were adding native support for the [OpenTelemetry Protocol](/docs/specs/otlp/) (OTLP). This caused us to be confident in adopting OpenTelemetry Collector as a widely adopted first-party -implementation of the OTLP protocol. +implementation of OTLP. We [proposed adding the OpenTelemetry Collector to Cloud Foundry in an RFC](https://github.com/cloudfoundry/community/blob/0365df129e52ae7b784957a5569b16b7e133f97e/toc/rfc/rfc-0018-aggregate-metric-egress-with-opentelemetry-collector.md), From bae0140f16a794b43d481ee5ab4975535afcc6f6 Mon Sep 17 00:00:00 2001 From: Carson Long Date: Tue, 7 Nov 2023 16:28:23 -0600 Subject: [PATCH 10/14] Update content/en/blog/2023/cloud-foundry.md Co-authored-by: Severin Neumann --- content/en/blog/2023/cloud-foundry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/blog/2023/cloud-foundry.md b/content/en/blog/2023/cloud-foundry.md index 6fdcf5164b65..347996cf4c80 100644 --- a/content/en/blog/2023/cloud-foundry.md +++ b/content/en/blog/2023/cloud-foundry.md @@ -76,7 +76,7 @@ implementation of OTLP. We [proposed adding the OpenTelemetry Collector to Cloud Foundry in an RFC](https://github.com/cloudfoundry/community/blob/0365df129e52ae7b784957a5569b16b7e133f97e/toc/rfc/rfc-0018-aggregate-metric-egress-with-opentelemetry-collector.md), -and solicited our community's feedback. It was accepted on 07-07-2023 and we got +and solicited our community's feedback. It was accepted on July 7, 2023 and we got to work. ## How we integrated OpenTelemetry with our current metric system From 3e40c30a40149d17f68d1dd8579a0612bd38a205 Mon Sep 17 00:00:00 2001 From: Carson Long Date: Tue, 7 Nov 2023 16:28:51 -0600 Subject: [PATCH 11/14] Update content/en/blog/2023/cloud-foundry.md Co-authored-by: Severin Neumann --- content/en/blog/2023/cloud-foundry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/blog/2023/cloud-foundry.md b/content/en/blog/2023/cloud-foundry.md index 347996cf4c80..d03585f67c67 100644 --- a/content/en/blog/2023/cloud-foundry.md +++ b/content/en/blog/2023/cloud-foundry.md @@ -97,7 +97,7 @@ effectively. Our implementation worked well, though we found the OpenTelemetry Collector to be using large amounts of CPU. We had done the simplest thing to start with, and -were only sending one metric per gRPC request. Our Collector's Resource +were only sending one metric per gRPC request. Our Collector's resource utilization dropped drastically when we [added batching](https://github.com/cloudfoundry/loggregator-agent-release/pull/396). From c1a0eb635095c4f9d6d3f2aea0bc7f9c302c7a07 Mon Sep 17 00:00:00 2001 From: Carson Long Date: Tue, 7 Nov 2023 16:29:05 -0600 Subject: [PATCH 12/14] Update content/en/blog/2023/cloud-foundry.md Co-authored-by: Severin Neumann --- content/en/blog/2023/cloud-foundry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/blog/2023/cloud-foundry.md b/content/en/blog/2023/cloud-foundry.md index d03585f67c67..921cdaa2b405 100644 --- a/content/en/blog/2023/cloud-foundry.md +++ b/content/en/blog/2023/cloud-foundry.md @@ -92,7 +92,7 @@ with many of those lines simply closing curly brackets. In writing this code we had to think about how to take our existing data model and translate it into the OpenTelemetry protobuf data model. We found OpenTelemetry to have considered many edge cases we had encountered in the past, and plenty we had not yet -encountered. Someday we hope to understand how to use Scopes and Resources +encountered. Someday we hope to understand how to use [Scopes](/docs/concepts/instrumentation-scope/) and [Resources](/docs/concepts/resources/) effectively. Our implementation worked well, though we found the OpenTelemetry Collector to From 7ffd5bf13bfebdeae80a522d13a17f53b6c97a59 Mon Sep 17 00:00:00 2001 From: Carson Long Date: Tue, 7 Nov 2023 14:36:19 -0800 Subject: [PATCH 13/14] Update content/en/blog/2023/cloud-foundry.md Co-authored-by: Severin Neumann --- content/en/blog/2023/cloud-foundry.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/en/blog/2023/cloud-foundry.md b/content/en/blog/2023/cloud-foundry.md index 921cdaa2b405..705016b7fbf9 100644 --- a/content/en/blog/2023/cloud-foundry.md +++ b/content/en/blog/2023/cloud-foundry.md @@ -64,9 +64,9 @@ Fluent Bit early because our ability to contribute to the codebase would be minimal, which we worried would limit us in the future. We then looked more seriously at Telegraf and OpenTelemetry. Both are written in -Go, so we were good there. Telegraf has a more limited community, and does not -offer a customizable build process. We liked that OpenTelemetry allows us to -build a Collector with a curated selection of our own components and community +Go, so we were good there. Our main reason to go with OpenTelemetry, was the +availability of a customizable build process: OpenTelemetry allows us to +[build a Collector](/docs/collector/custom-collector/) with a curated selection of our own components and community components. Additionally, when looking at OpenTelemetry we found that many [tools](/ecosystem/integrations/) and [vendors](/ecosystem/vendors/) From c9022573e2afcfef1cf18d764bb1feac728d7683 Mon Sep 17 00:00:00 2001 From: Carson Long Date: Wed, 8 Nov 2023 07:56:26 -0800 Subject: [PATCH 14/14] Update content/en/blog/2023/cloud-foundry.md Fix formatting. --- content/en/blog/2023/cloud-foundry.md | 28 ++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/content/en/blog/2023/cloud-foundry.md b/content/en/blog/2023/cloud-foundry.md index 705016b7fbf9..2068535ef992 100644 --- a/content/en/blog/2023/cloud-foundry.md +++ b/content/en/blog/2023/cloud-foundry.md @@ -11,9 +11,9 @@ cSpell:ignore: Kocher unscalable {{< blog/integration-badge >}} [Cloud Foundry](https://www.cloudfoundry.org/) recently integrated the -[OpenTelemetry Collector](/docs/collector/) for metrics egress and we learned a lot along the way. -We're excited about what the integration offers today and all the possibilities -it opens up for us. +[OpenTelemetry Collector](/docs/collector/) for metrics egress and we learned a +lot along the way. We're excited about what the integration offers today and all +the possibilities it opens up for us. ## What we were looking for @@ -66,18 +66,19 @@ minimal, which we worried would limit us in the future. We then looked more seriously at Telegraf and OpenTelemetry. Both are written in Go, so we were good there. Our main reason to go with OpenTelemetry, was the availability of a customizable build process: OpenTelemetry allows us to -[build a Collector](/docs/collector/custom-collector/) with a curated selection of our own components and community -components. +[build a Collector](/docs/collector/custom-collector/) with a curated selection +of our own components and community components. -Additionally, when looking at OpenTelemetry we found that many [tools](/ecosystem/integrations/) and [vendors](/ecosystem/vendors/) -were adding native support for the [OpenTelemetry Protocol](/docs/specs/otlp/) (OTLP). This caused us to be confident -in adopting OpenTelemetry Collector as a widely adopted first-party -implementation of OTLP. +Additionally, when looking at OpenTelemetry we found that many +[tools](/ecosystem/integrations/) and [vendors](/ecosystem/vendors/) were adding +native support for the [OpenTelemetry Protocol](/docs/specs/otlp/) (OTLP). This +caused us to be confident in adopting OpenTelemetry Collector as a widely +adopted first-party implementation of OTLP. We [proposed adding the OpenTelemetry Collector to Cloud Foundry in an RFC](https://github.com/cloudfoundry/community/blob/0365df129e52ae7b784957a5569b16b7e133f97e/toc/rfc/rfc-0018-aggregate-metric-egress-with-opentelemetry-collector.md), -and solicited our community's feedback. It was accepted on July 7, 2023 and we got -to work. +and solicited our community's feedback. It was accepted on July 7, 2023 and we +got to work. ## How we integrated OpenTelemetry with our current metric system @@ -92,8 +93,9 @@ with many of those lines simply closing curly brackets. In writing this code we had to think about how to take our existing data model and translate it into the OpenTelemetry protobuf data model. We found OpenTelemetry to have considered many edge cases we had encountered in the past, and plenty we had not yet -encountered. Someday we hope to understand how to use [Scopes](/docs/concepts/instrumentation-scope/) and [Resources](/docs/concepts/resources/) -effectively. +encountered. Someday we hope to understand how to use +[Scopes](/docs/concepts/instrumentation-scope/) and +[Resources](/docs/concepts/resources/) effectively. Our implementation worked well, though we found the OpenTelemetry Collector to be using large amounts of CPU. We had done the simplest thing to start with, and