Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOCS] Adds user-facing docs for the new KP logging configuration #94993

Merged
merged 33 commits into from
Mar 30, 2021
Merged
Changes from 1 commit
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
bfd3394
Converting logging README from mdx to asciidoc for user facing docs
TinaHeiligers Mar 14, 2021
ac8b690
Converts README to asciidoc
TinaHeiligers Mar 14, 2021
a89dd12
Adds logging settings to setup settings guide
TinaHeiligers Mar 15, 2021
bd22fc2
Splits Core API dev docs into individual service pages
TinaHeiligers Mar 16, 2021
a05082f
Adds logging configuration changes to breaking changes page
TinaHeiligers Mar 18, 2021
afbdab3
Cleans up some logging documentation
TinaHeiligers Mar 18, 2021
0b028fb
Cleans up commented out section
TinaHeiligers Mar 18, 2021
b06ed3d
Removes links that will be added once the new logging docs are merged
TinaHeiligers Mar 19, 2021
b2cb201
Removes bad link
TinaHeiligers Mar 19, 2021
d4edcd9
Address PR review comments
TinaHeiligers Mar 23, 2021
1eebed4
Adds a note to the Elasticsearch service section
TinaHeiligers Mar 23, 2021
526616c
Merge branch 'master' into core-docs/logging
TinaHeiligers Mar 24, 2021
3354da8
Groups services sections together into server and or client side usag…
TinaHeiligers Mar 25, 2021
3cd78af
Addresses more review comments
TinaHeiligers Mar 25, 2021
3bd3aea
Minor fixes
TinaHeiligers Mar 25, 2021
8147e92
Adds a few phrases from the Platform Building Blocks documentation
TinaHeiligers Mar 25, 2021
4cd64c9
Removes duplicate scoped elasticsearch service
TinaHeiligers Mar 25, 2021
9a80b1d
Moves layout, appenders before logger heirarchy
TinaHeiligers Mar 29, 2021
163dcd9
Adds basic logging configuration examples
TinaHeiligers Mar 29, 2021
648f6b6
Updates general logging settings with links to new reference code sni…
TinaHeiligers Mar 29, 2021
6c3907e
Cleanup and follow up comments
TinaHeiligers Mar 29, 2021
3f675b4
Deletes file
TinaHeiligers Mar 29, 2021
c13c70d
Merge branch 'master' into core-docs/logging
TinaHeiligers Mar 29, 2021
cd26a55
Merge branch 'master' into core-docs/logging
TinaHeiligers Mar 29, 2021
83c6b6d
Update docs/developer/architecture/core/application_service.asciidoc
TinaHeiligers Mar 30, 2021
d77b4ea
Update docs/developer/architecture/core/application_service.asciidoc
TinaHeiligers Mar 30, 2021
5b7424e
Update docs/developer/architecture/core/configuration-service.asciidoc
TinaHeiligers Mar 30, 2021
7ebd4c1
Update docs/developer/architecture/core/logging-service.asciidoc
TinaHeiligers Mar 30, 2021
d8b74c7
Update docs/developer/architecture/core/logging-service.asciidoc
TinaHeiligers Mar 30, 2021
01905e3
Update docs/developer/architecture/core/logging-service.asciidoc
TinaHeiligers Mar 30, 2021
de11253
Update docs/developer/architecture/core/saved-objects-service.asciidoc
TinaHeiligers Mar 30, 2021
ac6d10d
Update docs/developer/architecture/core/saved-objects-service.asciidoc
TinaHeiligers Mar 30, 2021
c543306
Apply suggestions from code review
TinaHeiligers Mar 30, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Adds logging configuration changes to breaking changes page
  • Loading branch information
TinaHeiligers committed Mar 18, 2021
commit a05082ff3265f34f2a0693b1108b02ef3eeed90e
Original file line number Diff line number Diff line change
@@ -31,7 +31,8 @@ export const config = {
export type MyPluginConfigType = TypeOf<typeof config.schema>;
----

* Read config value exposed via `PluginInitializerContext`.
* Read config value exposed via `PluginInitializerContext`:

*my_plugin/server/index.ts*
[source,typescript]
----
@@ -42,6 +43,8 @@ export class MyPlugin {
// or if config is optional:
this.config$ = initializerContext.config.createIfExists<MyPluginConfigType>();
}
...
}
----

If your plugin also has a client-side part, you can also expose
1 change: 1 addition & 0 deletions docs/developer/architecture/core/index.asciidoc
Original file line number Diff line number Diff line change
@@ -30,6 +30,7 @@ export class MyPlugin {
The server and client services that are available are:

=== Server-side
* <<configuration-service, Configuration service>>
* <<elasticsearch-service, Elasticsearch service>>
* <<http-service, HTTP service>>
* <<logging-service, Logging service>>
Original file line number Diff line number Diff line change
@@ -90,7 +90,7 @@ NOTE: If you define an appender for a logger name, the log messages aren't handl

| @timestamp | ISO8601_TZ `2012-01-31T23:33:22.011-05:00` | Absolute `23:33:22.011`

| context name | `parent.child` | `['parent', 'child']`
| logger | `parent.child` | `['parent', 'child']`

| level | `DEBUG` | `['debug']`

@@ -108,7 +108,7 @@ NOTE: If you define an appender for a logger name, the log messages aren't handl

| @timestamp | ISO8601_TZ `2012-01-31T23:33:22.011-05:00` | ISO8601 `2012-01-31T23:33:22.011Z`

| context name | `log.logger: parent.child` | `tags: ['parent', 'child']`
| logger | `log.logger: parent.child` | `tags: ['parent', 'child']`

| level | `log.level: DEBUG` | `tags: ['debug']`

@@ -122,19 +122,20 @@ NOTE: If you define an appender for a logger name, the log messages aren't handl

|===

[[logging-cli-migration]]
=== Logging configuration via CLI migration guide

NOTE: The `--verbose` and `--silent` cli flags are supported and have build-in behavior. You can override the default behavior using the following equivalent platform logging configurations:
TinaHeiligers marked this conversation as resolved.
Show resolved Hide resolved
[options="header"]
|===

| legacy logging | {kib} Platform logging
| legacy logging | {kib} Platform logging | cli shortcuts

|--verbose| --logging.root.level=debug --logging.root.appenders[0]=default --logging.root.appenders[1]=console
|--verbose| --logging.root.level=debug --logging.root.appenders[0]=default --logging.root.appenders[1]=console | --verbose
mshustov marked this conversation as resolved.
Show resolved Hide resolved

|--quiet| --logging.root.level=error --logging.root.appenders[0]=default --logging.root.appenders[1]=console
|--quiet| --logging.root.level=error --logging.root.appenders[0]=default --logging.root.appenders[1]=console | not supported

|--silent| --logging.root.level=off
|--silent| --logging.root.level=off | --silent
|===

NOTE: you have to pass the `default` appender until the legacy logging system is removed in v8.0
TinaHeiligers marked this conversation as resolved.
Show resolved Hide resolved
16 changes: 9 additions & 7 deletions docs/developer/architecture/core/logging-service.asciidoc
Original file line number Diff line number Diff line change
@@ -62,8 +62,9 @@ Levels are ordered, so _all_ > _fatal_ > _error_ > _warn_ > _info_ > _debug_ > _
A log record is being logged by the logger if its level is higher than or equal to the level of its logger. Otherwise,
the log record is ignored.

The _all_ and _off_ levels can be used only in configuration and are just handy shortcuts that allow developer to log every
log record or disable logging entirely for the specific context name.
The _all_ and _off_ levels can be used only in configuration and are just handy shortcuts that allow you to log every
log record or disable logging entirely or for a specific logger. These levels are also configurable as <<logging-cli-migration,cli arguments>>.
TinaHeiligers marked this conversation as resolved.
Show resolved Hide resolved


[[logging-layouts]]
=== Layouts
@@ -156,7 +157,7 @@ to a terminal with color support.

[[json-layout]]
==== JSON layout
With `json` layout log messages will be formatted as JSON strings in https://www.elastic.co/guide/en/ecs/current/ecs-reference.html[ECS format] that includes a timestamp, log level, context name, message text and any other metadata that may be associated with the log message itself.
With `json` layout log messages will be formatted as JSON strings in https://www.elastic.co/guide/en/ecs/current/ecs-reference.html[ECS format] that includes a timestamp, log level, logger, message text and any other metadata that may be associated with the log message itself.

[[logging-appenders]]
=== Appenders
@@ -412,10 +413,11 @@ logging:
level: debug
----

[[dedicated-loggers]]
==== Dedicated loggers

*Root**
The `root` logger has a dedicated configuration node since this context name is special and should always exist. By default `root` is configured with `info` level and `default` appender that is also always available. This is the configuration that all custom loggers will use unless they're re-configured explicitly.
**Root**
The `root` logger has a dedicated configuration node since this logger is special and should always exist. By default `root` is configured with `info` level and `default` appender that is also always available. This is the configuration that all custom loggers will use unless they're re-configured explicitly.

For example to see _all_ log messages that fall back on the `root` logger configuration, just add one line to the configuration:

@@ -499,7 +501,7 @@ The message contains some high-level information, and the corresponding log meta

=== Usage

Usage is very straightforward, one should just get a logger for a specific context name and use it to log messages with
Usage is very straightforward, one should just get a logger for a specific context and use it to log messages with
different log level.

[source,typescript]
@@ -532,7 +534,7 @@ And assuming logger for `server` name with `console` appender and `trace` level
[2017-07-25T11:54:41.639-07:00][DEBUG][server.http] Message with `debug` log level.
----

The log will be less verbose with `warn` level for the `server` context name:
The log will be less verbose with `warn` level for the `server` logger:
[source,bash]
----
[2017-07-25T11:54:41.639-07:00][WARN ][server] Message with `warn` log level.
21 changes: 11 additions & 10 deletions docs/migration/migrate_8_0.asciidoc
Original file line number Diff line number Diff line change
@@ -58,16 +58,16 @@ for example, `logstash-*`.
logging:
appenders:
console:
kind: console
type: console
layout:
kind: pattern
type: pattern
pattern: "%date{ISO8601_TZ}{UTC}"
-------------------
See https://github.com/elastic/kibana/pull/90368 for more details.

[float]
==== Responses are never logged by default
*Details:* Previously responses would be logged if either `logging.json` was true, `logging.dest` was specified, or a `TTY` was detected. With the new logging config, these are provided by a dedicated <<logger-hierarchy,logger>>.
*Details:* Previously responses would be logged if either `logging.json` was true, `logging.dest` was specified, or a `TTY` was detected. With the new logging configuration, these are provided by a dedicated <<dedicated-loggers,logger>>.
TinaHeiligers marked this conversation as resolved.
Show resolved Hide resolved

*Impact:* To restore the previous behavior, in `kibana.yml` enable `debug` for the <<request-response-logger,`http.server.response`>> logger:
TinaHeiligers marked this conversation as resolved.
Show resolved Hide resolved
[source,yaml]
@@ -82,7 +82,7 @@ See https://github.com/elastic/kibana/pull/87939 for more details.

[float]
==== Logging destination is specified by the appender
*Details:* Previously log destination would be `stdout` and could be changed to `file` using `logging.dest`. With new {kib} logging you specify the destination using <<logging-appenders,appenders>>.
*Details:* Previously log destination would be `stdout` and could be changed to `file` using `logging.dest`. With the new logging configuration, you can specify the destination using <<logging-appenders,appenders>>.
TinaHeiligers marked this conversation as resolved.
Show resolved Hide resolved

*Impact:* To restore the previous behavior and log records to *stdout*, in `kibana.yml` use the `console` appender.
TinaHeiligers marked this conversation as resolved.
Show resolved Hide resolved
[source,yaml]
@@ -97,20 +97,21 @@ To send logs to `file` with a given file path, you should define a custom append
-------------------
logging:
appenders:
file:
file: <1>
type: file
fileName: /var/log/kibana.log
layout:
type: pattern
root:
appenders: [default, file]
-------------------
<1> custom appender name

[float]
==== Specify log verbosity with root
==== Set log verbosity with root
*Details:* Previously logging output would be specified by `logging.silent` (none), `logging.quiet` (error messages only) and `logging.verbose` (all). With the new logging configuration, set the minimum required <<log-level,log level>>.
TinaHeiligers marked this conversation as resolved.
Show resolved Hide resolved

*Impact:* To restore the previous behavior, in `kibana.yml` specify `logging.root.level` as one of `off`, `error`, `all`:
*Impact:* To restore the previous behavior, in `kibana.yml` specify `logging.root.level`:
[source,yaml]
-------------------
# suppress all logs
@@ -138,11 +139,11 @@ logging:
level: all
-------------------

NOTE: You need to specify the `root.appender` to have your log records surfaced.
NOTE: You must specify the `root` appender for your log records to show.

[float]
==== Declare log message format for each custom appender
*Details:* Previously all events would be logged in `json` format when `logging.json` was true. With the new logging configuration you can specify the output format with <<logging-layouts,layouts>>. See here for a quick look at <<logging-json-format-old-and-new-example,`json` format>> and <<logging-pattern-format-old-and-new-example,*stdout*>>.
==== Declare log message format
*Details:* Previously all events would be logged in `json` format when `logging.json` was true. With the new logging configuration you can specify the output format with <<logging-layouts,layouts>>. You can choose between <<logging-json-format-old-and-new-example,`json`>> and <<logging-pattern-format-old-and-new-example,pattern>> format depending on your needs.
TinaHeiligers marked this conversation as resolved.
Show resolved Hide resolved

*Impact:* To restore the previous behavior, in `kibana.yml` configure the logging format for each custom appender with the <<logging-layouts,`appender.layout`>> property. There is no default for custom appenders and each one must be configured expilictly.
TinaHeiligers marked this conversation as resolved.
Show resolved Hide resolved

20 changes: 11 additions & 9 deletions docs/setup/settings.asciidoc
Original file line number Diff line number Diff line change
@@ -256,36 +256,38 @@ To reload the logging settings, send a SIGHUP signal to {kib}.
| A list of logging appenders to forward the root level logger instance to. By default `root` is configured with the `default` appender that must be included in the list. This is the configuration that all custom loggers will use unless they're re-configured explicitly. Additional appenders, if configured, can be included in the list. See the <<logging-service,guide>> on configuring {kib} logging.

|[[logging-root-level]] `logging.root.level:` {ess-icon}
| Level at which a log record should be logged. Supported levels are: _all_, _fatal_, _error_, _warn_, _info_, _debug_, _trace_, _off_. Levels are ordered from _all_ (highest) to _off_ and a log record will be logged it its level is higher than or equal to the level of its logger, otherwise the log record is ignored. Set to `all` to log all events, including system usage information and all
requests. Set to `off` to silence all logs. *Default: `info`*
| Level at which a log record should be logged. Supported <<log-level,levels>> are: _all_, _fatal_, _error_, _warn_, _info_, _debug_, _trace_, _off_. Levels are ordered from _all_ (highest) to _off_ and a log record will be logged it its level is higher than or equal to the level of its logger, otherwise the log record is ignored. Set to `all` to log all events, including system usage information and all requests. Set to `off` to silence all logs. *Default: `info`*.

|[[logging-loggers]] `logging.loggers:`
| Enables you to specify a specific context for a logger instance. The following example shows a valid logger configuration:
| Allows you to specify a specific logger instance. The following example shows a valid logger configuration:
|===

[source,text]
--
logging:
loggers:
- name: server
level: fatal
appenders: [console]
- name: server <1>
level: fatal <2>
appenders: [console] <3>
--
<1> logger configuration for the server.
<2> Log all records for the server that are fatal.
<3> Use the `console` appender to send log records to *stdout* using the <<pattern-layout,default pattern>> layout.

[cols="2*<"]
|===

| `logging.loggers.name:`
| Context for the log record.
| Specific logger instance.

| `logging.loggers.level:`
| Level at which a log record should be shown. The same levels as for the <<logging-root-level,root logger>> are available for custom appenders.
| Level at which a log record should be shown. Supported <<log-level,levels>> are: _all_, _fatal_, _error_, _warn_, _info_, _debug_, _trace_, _off_.

| `logging.loggers.appenders:`
| Specific appender format to apply for a particular logger context.

| `logging.appenders:`
| Define how and where log messages are displayed (eg. stdout or console) and stored (eg. file on the disk)
| Define how and where log messages are displayed (eg. *stdout* or console) and stored (eg. file on the disk).

| `logging.appenders.console:`
| Appender to use for logging records to *stdout*. By default, uses the `[%date][%level][%logger] %message` <<pattern-layout,`pattern`>> layout. To use a <<json-layout,`json` layout>>, set the layout type to `json`.