Support for 'log.origin.module'

[peterpramb]

Using the Apache HTTPD error log as a sample use case, the log field set covers most fields, but it is missing a field in the hierarchy "above" log.origin.function to put the module name causing the log message in.

Using this sample Apache HTTPD error message

[Thu May 12 08:28:57.652118 2011] [core:error] [pid 8777:tid 4326490112] core.c(4739): [client ::1:58619] File does not exist: /usr/local/apache2/htdocs/favicon.ico

one can map the fields as follows (taking Filebeat's apache module as reference):

['@timestamp'] ['apache.error.module':'log.level'] [pid 'process.pid':tid 'process.thread.id'] 'log.origin.file.name'('log.origin.file.line'): [client 'source.ip':'source.port'] 'message'

But apache.error.module should be more unified to something like log.origin.module, e.g.:

- name: origin.module
  level: extended
  type: keyword
  example: core
  short: The module which originated the log event.
  description: >
    The name of the module which originated the log event (if the event origin has a notion of 'modules').

[webmat]

Could log.logger work for your use case? The semantics are slightly different, but perhaps it's close enough?

[peterpramb]

As far as I understand it log.logger seems to be something "global" to an application:

The name of the logger inside an application. This is usually the name of the class which initialized the logger, or can be a custom name.

Where log.origin.module would directly refer to the origin "container" of something producing a log event (a "collection" of several related log.origin.function if you want).

In the mentioned use case log.origin.module would represent all log events produced by an logical unit, either a single source code file or a directory of related source code files, e.g. core, http2, ssl.

[ebeahan]

pinging @felixbarny - Does this make sense for the application logging use case as well?

[felixbarny]

I'd also lean towards re-using log.logger

As far as I understand it log.logger seems to be something "global" to an application:

No, there would typically be multiple loggers per application. By default one per source file but if customize, they could also describe modules, like login or cart for an e-commerce application. So I think it fits quite well.

[peterpramb]

I see, then log.logger would really fit here (even though the naming is rather confusing).

Feel free to close this.

[kares]

think this might be revisited later - in a different context, consider the use-case of parsing a Java stack trace element
e.g. at com.sample.stacktrace.StackTraceExample.aMethod(StackTraceExample.java:42)

we're able to identify log.origin.file.name (as well as log.origin.file.line) and log.origin.function but there's no clear ECS field on where to put the full class-name (com.sample.stacktrace.StackTraceExample part).

[ebeahan]

I see, then log.logger would really fit here (even though the naming is rather confusing).

@peterpramb ECS is always looking for ways to improve the clarity of the specification and documentation, so the feedback here is greatly appreciated.

[ebeahan]

we're able to identify log.origin.file.name (as well as log.origin.file.line) and log.origin.function but there's no clear ECS field on where to put the full class-name (com.sample.stacktrace.StackTraceExample part).

@kares For a stack trace, the exception's content is better captured using the error.* fields with the log.* fields capturing details about the component logging the exception. Underneath error.*, error.type is intended to capture the class name of the exception.

[kares]

Thanks Eric, 👍 apm seems to be using error.* and it make sense to put the full trace under error.stack_trace.
The (legacy) case I was trying to match might have been a little 'weird' as it attempts to match a single stack-trace line (whether it's part of a full trace or not doesn't matter) so the error.type isn't really a clear fit for the class-name.

[felixbarny]

@kares ECS doesn't have a definition for a parsed stack trace yet. See also #562.
What are some of the use cases you'd like to solve by parsing out the individual stack frames?

I see, then log.logger would really fit here (even though the naming is rather confusing).

@peterpramb do you have a suggestion to make the docs a bit clearer?

closing for now but feel free to still comment on this issue

[kares]

What are some of the use cases you'd like to solve by parsing out the individual stack frames?

Sorry for the late reply not that I am aware of an actual use-case, we were trying to map a (legacy) LS grok pattern to ECS.
The JAVASTRACKTRACEPART pattern is parsing out the information into fields.