Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support to generate OTEL logs in generate script #931

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

shantanualsi
Copy link

@shantanualsi shantanualsi commented Nov 29, 2024

Desciption:
image

This PR adds support to generate OTEL logs from the generate script. It creates another service loki-ingester-otel.
It defines another logger alongside AppLogger that pushes logs to the OTEL collector.

Few notes:

  • Dockerfile is modified that starts otel collector which is required to ingest logs
  • Loki address in otel-config is configurable as per the environment used. Can be extracted as env-var later
  • Adds slog as it provides a bridge for otel provider
  • Some refactor to move files to appropriate directories
  • There are a lot of improvements that can be made in the script related to structured metadata and labels. Can be made in subsequent PRs.

Fixes: #224

@shantanualsi shantanualsi requested a review from a team as a code owner November 29, 2024 12:32
@gtk-grafana
Copy link
Contributor

@shantanualsi Thanks for taking a look at this, getting better OTEL logs in the generator is something we've wanted for a while now: #224

But I can't seem to get this to work locally.

When running yarn generate-logs: I get

2024/12/02 09:26:48 rpc error: code = Unimplemented desc = unknown service opentelemetry.proto.collector.logs.v1.LogsService

We might want a copy of the start.sh script that is generated by the Dockerfile available to call in the generate-logs script available in the repo?

Also yarn server and yarn server:ci appear broken as well, generator container is filled with logs like the following:

2024-12-02T15:30:11.506Z	info	internal/retry_sender.go:126	Exporting failed. Will retry the request after interval.	{"kind": "exporter", "data_type": "logs", "name": "otlphttp/logs", "error": "failed to make an HTTP request: Post \"http://localhost:3100/otlp/v1/logs\": dial tcp [::1]:3100: connect: connection refused", "interval": "38.571661375s"}

Is there some additional config I need to set?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add an OTEL application to the log generator
2 participants