Drop separate logger initialization #1736
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
This PR simplifies how logging is initialized in the application to increase conformity with standards (OpenTelemetry in this case).
There is a cost however: if the application initialization fails, there is no logger to notify us about that failure (and it will never reach any OTLP exporter, since the "real" logger doesn't get initialized until the whole application initialization is done).
As a compromise, we initialize a basic JSON logger, since JSON is used in most production environments. If anything goes wrong during app init, that logger prints the error to
stderr
.The recommended approach to deal with app init failures is to have additional monitoring in place that watches over running instances of an application. If there is a crash loop, you know something is up and can directly examine pod logs.
If you have a file based log collector in place (eg. fluentbit/fluentd), you can collect the JSON logs and forward them to your log analytics system.
We may revisit this decision in the future, but for now, we feel this is an acceptable compromise.
Notes for reviewer