Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop separate logger initialization #1736

Merged
merged 3 commits into from
Oct 22, 2024
Merged

Drop separate logger initialization #1736

merged 3 commits into from
Oct 22, 2024

Conversation

sagikazarmark
Copy link
Member

Overview

This PR simplifies how logging is initialized in the application to increase conformity with standards (OpenTelemetry in this case).

There is a cost however: if the application initialization fails, there is no logger to notify us about that failure (and it will never reach any OTLP exporter, since the "real" logger doesn't get initialized until the whole application initialization is done).

As a compromise, we initialize a basic JSON logger, since JSON is used in most production environments. If anything goes wrong during app init, that logger prints the error to stderr.

The recommended approach to deal with app init failures is to have additional monitoring in place that watches over running instances of an application. If there is a crash loop, you know something is up and can directly examine pod logs.

If you have a file based log collector in place (eg. fluentbit/fluentd), you can collect the JSON logs and forward them to your log analytics system.

We may revisit this decision in the future, but for now, we feel this is an acceptable compromise.

Notes for reviewer

@sagikazarmark sagikazarmark added the kind/refactor Code refactor, cleanup or minor improvement label Oct 22, 2024
@sagikazarmark sagikazarmark merged commit 6e482f1 into main Oct 22, 2024
19 checks passed
@sagikazarmark sagikazarmark deleted the wire branch October 22, 2024 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/refactor Code refactor, cleanup or minor improvement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants