Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add connection identifier to logging output #98

Open
volcan01010 opened this issue Mar 24, 2021 · 0 comments
Open

Add connection identifier to logging output #98

volcan01010 opened this issue Mar 24, 2021 · 0 comments

Comments

@volcan01010
Copy link
Collaborator

Summary

As an ETL pipeline user, I want to know which database connection log messages correspond to so that I can have a better idea of where my process is.

Description

When messages from ETL Helper logger are consumed by other handlers they can end up like this:

16:43:26 etlhelper INFO   Fetching rows
16:43:27 etlhelper INFO   108 rows returned
16:43:30 etlhelper INFO   Executing query

If the pipeline involves multiple databases, there is no way to know which database the messages correspond to. The messages should be updated to provide some information about the connection.

(It is probably not possible to put the DbParams variable name in the message there as I'm not sure if that is accessible when the log messages are written. If it was possible, that would be best. It may be necessary to add a "name" attribute to DbParams classes.)

Acceptance criteria

Log messages look like:

  • [ ] 16:43:30 etlhelper INFO Fetching rows from MY_DB

of if that isn't possible

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant