Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Describe Otel log gateway PoC #843

Merged
merged 6 commits into from
Mar 4, 2024

Conversation

chrkl
Copy link
Contributor

@chrkl chrkl commented Mar 1, 2024

Description

Changes proposed in this pull request (what was done and why):

  • Describe Otel log gateway PoC

Changes refer to particular issues, PRs or documents:

Traceability

  • The PR is linked to a GitHub issue.
  • New features have a milestone set.
  • New features have defined acceptance criteria in a corresponding GitHub Issue, and all criteria are satisfied with this PR.
  • The corresponding GitHub issue has a respective area and kind label.
  • The follow-up issues (if any) are linked in the Related Issues section.
  • Adjusted the documentation if the change is user-facing.
  • The feature is unit-tested
  • The feature is e2e-tested

@chrkl chrkl requested a review from a team as a code owner March 1, 2024 14:15
@kyma-bot kyma-bot added cla: yes Indicates the PR's author has signed the CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 1, 2024
Co-authored-by: Rakesh Garimella <[email protected]>
@kyma-bot kyma-bot added the lgtm Looks good to me! label Mar 1, 2024
Copy link
Contributor

@NHingerl NHingerl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

docs/contributor/pocs/otlp-logs.md Show resolved Hide resolved
docs/contributor/pocs/otlp-logs.md Outdated Show resolved Hide resolved
docs/contributor/pocs/otlp-logs.md Outdated Show resolved Hide resolved
docs/contributor/pocs/otlp-logs.md Outdated Show resolved Hide resolved
docs/contributor/pocs/otlp-logs.md Outdated Show resolved Hide resolved
docs/contributor/pocs/otlp-logs.md Outdated Show resolved Hide resolved
docs/contributor/pocs/otlp-logs.md Outdated Show resolved Hide resolved
docs/contributor/pocs/otlp-logs.md Outdated Show resolved Hide resolved
docs/contributor/pocs/otlp-logs.md Outdated Show resolved Hide resolved
docs/contributor/pocs/otlp-logs.md Outdated Show resolved Hide resolved
@kyma-bot kyma-bot removed the lgtm Looks good to me! label Mar 4, 2024

The following figure shows the different plugins that are configured in the processing pipeline. Note the use of the batch processor in the gateway, which introduces asynchronicity to the pipeline and causes that backpressure is not propagated back to the agent. To minimize the risk of log loss due to the batch processors properties, a persistent exporter queue was introduced in the gateway, which uses a persistent volume to buffer logs in case of a backend failure.

![Otel Collector Setup](./assets/otlp-logs.drawio.svg)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This diagram looks weird when rendered:
image

Consider changing the colors and background; for example, according to our guidelines.

@kyma-bot kyma-bot added the lgtm Looks good to me! label Mar 4, 2024
@kyma-bot kyma-bot merged commit 2083eee into kyma-project:main Mar 4, 2024
7 checks passed
@chrkl chrkl deleted the otel-logging-poc branch March 4, 2024 12:31
@a-thaler a-thaler added this to the 1.11.0 milestone Mar 4, 2024
@a-thaler a-thaler added kind/feature Categorizes issue or PR as related to a new feature. area/logs LogPipeline labels Mar 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/logs LogPipeline cla: yes Indicates the PR's author has signed the CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm Looks good to me! size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants