Log enrichment and/or inserted properties

[jstafford5380]

Describe the solution you'd like
Serilog is a pretty popular logger and it has the ability to enrich logs. It does this by decomposing objects into a list of properties that can be made available to the log output both as interpolated values but also as a list of properties which I believe are format_parameters in the stackdriver schema. I think right now, scalar properties get added as format_parameters, but objects only get serialized to json, but the properties of those objects are not added as queryable properties in the collection.

It would be useful to have something similar to serilog in thsi respect, as part of this package, or at least some means to write extension packages to achieve this. I think this could be achieved by introducing some sort of middleware pipeline that can be added to configuration. This middleware (or chain of middlewares) would run on every log statement and allow a developer to insert additional properties to each log. I believe the way this is achieved in serilog is to make available, a collection of properties that can be accessed in the middleware, allowing the developer to read from and add to the collection that would ultimately be mapped into format_parameters

Describe alternatives you've considered
I considered simply adding the desired properties to the actual log statement, but this approach is a bit more obtrusive.

Additional context
None.

[jskeet]

I'll assign this to Amanda for more comment, but I don't expect this to happen. If you want something like Serilog, I'd suggest using Serilog with its Stackdriver integration.

In the future I expect our diagnostics integration to be slimmer rather than more involved - I'm hoping that when https://github.com/googleapis/google-cloud-dotnet/tree/master/apis/Google.Cloud.Logging.Console is ready, that would be the simplest integration point, with Stackdriver just collecting log entries from the console.

Designing our own fully-featured competitor to Serilog is something that I don't personally think we should invest time in.

[jstafford5380]

Designing our own fully-featured competitor to Serilog is something that I don't personally think we should invest time in.

I tend to agree. I was really just looking for something in between. This package has the log schema, and I'm really just looking for a way to insert into that log schema rather than having to implement a completely new integration. Serilog doesn't have a 1st class Cloud Logging sink, it's maintained by a random community member.

[amanda-tarafa]

There's another alternative that you can use, and that is to use directly ILogger.Log<TState>(LogLevel, EventId, TState, Exception, Func<TState, Exception, string>) instead of the helper ILogger extension methods. If TState is the same as what Microsoft.Extensions.Logging.LoggerExtensions would send the "true" format parameters as, we add all pairs as format_parameters (the string representation of the value). You could even write your own extension for ILogger that would take parameters that would differentiate between "true" format parameters and extra properties you want to add, as long as they end up in TState, we will add them.

And another type of extra info you can add to the log entries is labels. We have Google.Cloud.Diagnostics.ILogEntryLabelProvider that you can implement (as many as you'd like) and register through standard DI. Take a look at #5313 which is an open FR around allowing labels per log entry (right now there's not an easy way for label providers to get context describing the log entry, so these work better when you want to add the same label to all entries, or context is not dependent on the entry itself but on something else).

[jstafford5380]

Just leaving a note here to acknowledge the reply, thank you. I'll need to experiment with TState to see if it'll output the way I'm thinking. I'll get back soon

[amanda-tarafa]

Yep, no rush at all.

[snakefoot]

See also dotnet/runtime#35995 and https://www.nuget.org/packages/XeonApps.Extensions.Logging.WithProperty/

[amanda-tarafa]

@jstafford5380 Just wanted to check if you've had a chance to try ILogger.Log<TState>(LogLevel, EventId, TState, Exception, Func<TState, Exception, string>) to add extra properties to log entries?

[jstafford5380]

Oh I'm sorry I have not! I had to pass that project to a teammate so I could work on a different part of the cloud migration but it is still something I plan to do. I'll actually point him to this thread to see if maybe he can give it a try, or see where he's at with it.

…

On Thu, Apr 1, 2021 at 2:21 AM Amanda Tarafa Mas ***@***.***> wrote: @jstafford5380 <https://github.com/jstafford5380> Just wanted to check if you've had a chance to try ILogger.Log<TState>(LogLevel, EventId, TState, Exception, Func<TState, Exception, string>) to add extra properties to log entries? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5929 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALI4RECMKDXOSKJT6ZOCE63TGQ3KLANCNFSM4XLN6KZQ> .

[amanda-tarafa]

That's fine. I'll mark this one as needs more info and if in a few days we haven't had an upate from you we will close it. You can just reopen when you are ready to try things out again and have questions etc.