Add log.logger

[felixbarny]

As discussed in #281 (comment), event.dataset is not a good place to store the name of the logger. Let's create a more suitable field for that.

[ruflin]

This reminds me off https://github.com/elastic/beats/blob/master/filebeat/module/elasticsearch/_meta/fields.yml#L10 which we use in the Elasticsearch logs. Do you see this as specific to Java or more general? Perhaps java.component?

[felixbarny]

I'm not entirely sure whether logging implementations in other languages tend to have name for a logger which is usually scoped to one class. @elastic/apm-agent-devs do you know?

But in general, it's not specific to Java. Even if it turns out only Java loggers tend to use it I think it makes sense to have this as a general field.

[ruflin]

If we tie it to Logging but not Java, what about log.component?

[felixbarny]

that would also work for me

[ruflin]

Currently +1 on log.component then, but probably miss something ;-)

[SergeyKleyman]

Logger-per-class pattern is definitely not something specific to Java. But I don't know what are the prevalent logging approaches in languages without class concept.

[ruflin]

logger also exists in the Beats (Golang) code. And we also call it logger. Should we perhaps even call it log.logger?

[felixbarny]

simple, clean and straight to the point. +1
I'll change the PR accordingly. But also curious what the other reviewers have to say 🙂

[beniwohli]

Another +1 for log.logger, that's what it is called in Python logging.

[webmat]

I can also confirm that the logging class/component can be a thing in Ruby as well.

I wonder if we shouldn't take a step back and think about this more holistically, though. Even in normal logs, you can have the function/method name provided by some loggers (e.g. Rails' MyController#my_method). And once we're at it, it's also possible to have line numbers and file paths in logs (exceptions and otherwise). This blog post shows default Rails log samples, which contain all of these.

However, the more we dig into these details, the less it makes sense to cram all this in event.* or log.*, IMO.

Should we look into putting together a more complete application logging / debugging field set? This field set could encompass the source code details as I'm discussing here, as well as optionally exception details, if the event is about an exception.

Let's use debug.* as an example field set name. Could be something else, but I think this could work well for these two cases:

- debug.component
- debug.method (or function?)
- debug.file.path (optionally a place where someone can nest file.*, if they care?)
- debug.file.line (or directly debug.line)
- debug.exception.stack_trace
- debug.exception.class
- debug.exception.fields (for exceptions that take params)
- debug.exception.exception... (nested)

I'm excluding message in here because I think the exception message should be in the base field message.

I'm sure there's a few more attributes we could add, here. This is just a quick first draft.

These ideas are related to #154 and #501.

[felixbarny]

Good point but I think linking to source code is a separate beast and we already have an issue for that: #154.

The log.logger is in addition to, but not the same as the class name. Although they are often the same it's important to not use them interchangeably. It's often much more expensive to compute the actual class/file name (by traversing the stack). The log.logger can be a user-supplied string value, like audit_logs or foobarbaz.

[ruflin]

+1 on moving forward as is and separating the other issue.

[webmat]

Ok I'm good with log.logger.

I would flesh out the description a little more, to clarify how this field is meant to be used. People are already getting confused by the amount of fields that can be used to describe the source of events. So I'd rather be a little more verbose.

[webmat]

I'm good with this, thanks Felix!

Did you want to wait for feedback from more people from the APM side?

[webmat]

Note that you'll have to pull this change, run make and push the re-generated files as well :-)

[felixbarny]

Did you want to wait for feedback from more people from the APM side?

I just wanted to confirm that log.logger is not just a Java thing which there is agreement that it's not. So I don't think we need to wait for more APM feedback (although still appreciated ofc).

Maybe wait another week to see if there are any objections and merge if there are? How do you normally handle that in the ECS repo? I just added a bunch of people as reviewers who might have a stake in this.

[webmat]

I'm fine with waiting a little more.

For merging, my heuristic is: I merge community PRs. Elasticians are free to merge, or ask me to merge, whichever they prefer. I'll be more proactive in taking over if there's backports needed (e.g. for small clarifications to fields that were in 1.0), as I don't want the burden of understanding the intricacies of the repo's branching & management to be pushed to all Elastic contributors.

[ruflin]

Let's get it in by end of day if no objects. We always have time up to the next release to change our opinion, but I prefer not to have pending PR's.