Logger: overwrite keys while logging new stuff

[alvaropc]

Hi!

I am here to ask you about the Keys of the logger.

I see that you can update the keys with structure_logs() method, but once you put a new key and assign a value, you won't be able to change it without calling structure_logs() again.

It would be interesting to be able to overwrite some key values at the same time you are logging, think about some KPI you would like to log as a key for instance, it would be nice to easily overwrite the value.

Although I know it is posible to write keys under "message" key it feels like not the best option for me, the main keys seem to be the place for this kind of information in order to catch it easily with tools like datadog or something like that.

I don't know if I am explaining it correctly but I hope you consider this as a new feature, I would also love to know your opinion about it as well as possible alternative aproaches, maybe I am just missunderstanding the way how you want the logger to be used and wirting more keys under messages is the correct way to do it!

Thank you very much in advance!
And congrats for such an amazing tool!

[heitorlessa]

Buenas @alvaropc, and thanks for opening an issue.

I understood you perfectly :) Allow me to break the explanation down to each point you raised:

• structure_logs behaviour: This is by design, as customers wanted to either override everything previously created, or append additional keys
• add/override top keys while you log: This is the first time we're hearing this, and I'd like to hear more use cases from other customers before considering it. We support adding any object as part of the message being logged to allow for additional keys, but not top keys as that is a different behaviour of structured logging.
• difficult to catch keys under message: As we're logging as JSON, all these tools you mentioned should be able to easily access anything inside message .
  • We recommend customers to either add keys they know will be constant across their invocations via structure_logs, or to agree on a contract when logging to ensure it is well understood and always available during visualization
  • For example: log.info( { "operations": "collect_payment": "details": { "some_key": "some_value"...} })
  • Using Datadog Log Explorer or any Log Analytics tool, you can access it via @message.operations, @message.details.some_key

Hope that helps :)

We'll reopen if we hear other customers wanting this behaviour to be different while maintaining backward compatibility.

Muchas gracias Alvaro