[8.0] Remove legacy logging #112305

pgayvallet · 2021-09-15T16:00:38Z

Summary

Merge blocked by https://github.com/elastic/cloud/issues/80187

Fix #50660
Fix #82240
Part of #103915

All in title.

Tasks

Remove @kbn/legacy-logging
Remove usages of LegacyObjectToConfigAdapter
Remove @hapi/good / @hapi/good-squeeze / @hapi/podium from kibana dependencies
Remove legacy appender src/core/server/legacy/logging/appenders
Remove validation that requires logging.root.appenders to have a default
- this can now be optional and populated with the default value if nothing is explicitly configured
Remove legacy service (src/core/server/legacy)
Remove deprecated config keys from kibana-docker file
Update src/cli/serve/serve.js to remove references to removed deprecated flags (quiet, log-file), and remove references to deprecated legacy config (used inside verbose and silent)
Remove legacy logging related config deprecations

Checklist

Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios

Release Note

Kibana's logging configuration and log output format has changed. Please refer to the documentation for more detail on how to configure the new logging system.

…legacy-logging

pgayvallet · 2021-09-17T07:02:09Z

retest

…legacy-logging

tylersmalley · 2021-09-20T20:05:47Z

@elasticmachine merge upstream

kibanamachine · 2021-09-20T20:05:49Z

merge conflict between base and head

…legacy-logging

pgayvallet

Self review.

I tried to grep all usages of the removed settings in the documentation, but may have potentially missed some, a second pair of eyes would be more than welcome.

pgayvallet · 2021-09-23T07:43:15Z

packages/kbn-cli-dev-mode/src/cli_dev_mode.ts

@@ -163,7 +162,7 @@ export class CliDevMode {
      runExamples: cliArgs.runExamples,
      cache: cliArgs.cache,
      dist: cliArgs.dist,
-      quiet: !!cliArgs.quiet,
+      quiet: false,


@elastic/kibana-operations I kept the property in the Optimizer options and just set it to false to avoid more downstream changes. This can eventually be cleaned up as a follow-up (unless we really want to do it in current PR).

packages/kbn-config/src/config_service.ts

pgayvallet · 2021-09-23T07:53:54Z

packages/kbn-config/src/env.ts

@@ -21,8 +21,6 @@ export interface EnvOptions {
 export interface CliArgs {
  dev: boolean;
  envName?: string;
-  /** @deprecated */
-  quiet?: boolean;
  silent?: boolean;
  verbose?: boolean;


Wasn't sure if I should keep silent and verbose in CliArgs. Those are effectively unused inside core, as now their only usages is to change logging.root.level from src/cli/serve/serve.js, but at the same times, they're still cli args, so we may want to have them propagated to core for telemetry of some other need. WDYT?

Do we know how many users rely on silent and verbose cli flags? What bothers me is that we allow using them as CLI flags but not as config values. Btw we need to replace all the logging.* values in kibana.yml with their Kibana Logging system counterparts.

Do we know how many users rely on silent and verbose cli flags? What bothers me is that we allow using them as CLI flags but not as config values.

Yeah this bugs me too -- IIRC when we had last discussed this (I think with @spalger ?), the case was made that configuring appenders via the CLI is more cumbersome than when working in the yaml, and that it was a minimal maintenance burden to keep silent and verbose in place, which is why we only deprecated quiet at the time.

src/cli/serve/integration_tests/invalid_config.test.ts

src/cli/serve/serve.js

src/core/server/legacy/index.ts

src/core/server/logging/logging_config.ts

src/core/test_helpers/kbn_server.ts

elasticmachine · 2021-09-23T10:19:33Z

Pinging @elastic/kibana-core (Team:Core)

…legacy-logging

mshustov

Don't forget to remove the legacy logging system mentions in

code

kibana/src/core/server/elasticsearch/elasticsearch_config.ts

Line 214 in c09776d

    
           message: `Setting [${fromPath}.logQueries] is deprecated and no longer used. You should set the log level to "debug" for the "elasticsearch.queries" context in "logging.loggers" or use "logging.verbose: true".`,

config file

kibana/config/kibana.yml

Lines 93 to 104 in c09776d

    
           # Enables you to specify a file where Kibana stores log output. 
        
           #logging.dest: stdout 
        
           # Set the value of this setting to true to suppress all logging output. 
        
           #logging.silent: false 
        
           # Set the value of this setting to true to suppress all logging output other than error messages. 
        
           #logging.quiet: false 
        
           # Set the value of this setting to true to log all events, including system usage information 
        
           # and all requests. 
        
           #logging.verbose: false

https://github.com/elastic/kibana/blob/c09776dc76cf8e7c4efa1a1dd5bac2a635d66193/docs/settings/logging-settings.asciidoc#backwards-compatibility
https://github.com/elastic/kibana/blob/c09776dc76cf8e7c4efa1a1dd5bac2a635d66193/src/core/server/logging/README.mdx#logging-config-migration
https://github.com/elastic/kibana/blob/c09776dc76cf8e7c4efa1a1dd5bac2a635d66193/docs/developer/architecture/core/logging-configuration-migration.asciidoc#logging-configuration-migration
https://github.com/elastic/kibana/blob/c09776dc76cf8e7c4efa1a1dd5bac2a635d66193/docs/user/production-considerations/production.asciidoc#use-kib-in-a-production-environment

And ping Security solution. Detection engine team to update their readme file

kibana/x-pack/plugins/security_solution/server/lib/detection_engine/README.md

Line 144 in c09776d

logging.verbose: true

src/cli/serve/integration_tests/__fixtures__/invalid_config.yml

src/core/test_helpers/kbn_server.ts

src/cli/serve/integration_tests/__fixtures__/reload_logging_config/kibana.test.yml

mshustov · 2021-09-27T13:18:33Z

src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker

-    logging.rotate.usePolling
-    logging.silent
-    logging.useUTC
-    logging.verbose


@elastic/kibana-security is xpack.security.audit.appender.legacyLoggingConfig still necessary in this file?

mshustov · 2021-09-27T13:34:02Z

packages/kbn-config/src/env.ts

@@ -21,8 +21,6 @@ export interface EnvOptions {
 export interface CliArgs {
  dev: boolean;
  envName?: string;
-  /** @deprecated */
-  quiet?: boolean;
  silent?: boolean;
  verbose?: boolean;


Do we know how many users rely on silent and verbose cli flags? What bothers me is that we allow using them as CLI flags but not as config values. Btw we need to replace all the logging.* values in kibana.yml with their Kibana Logging system counterparts.

jportner

Code review only; security plugin unit test snapshot change LGTM 👍

…legacy-logging

mistic

Changes on code under operations team code owners LGTM

config/kibana.yml

…legacy-logging

kibanamachine · 2021-10-05T09:56:32Z

💚 Build Succeeded

Metrics [docs]

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
`core`	30	29	-1

History

💚 Build #157366 succeeded d97156c
💚 Build #156676 succeeded 4957cd0
💚 Build #156057 succeeded 234f4fb
💚 Build #156036 succeeded f41b60b
💚 Build #155985 succeeded b33664b

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

pgayvallet added 9 commits September 15, 2021 14:10

remove kbn-legacy-logging package

eb6d618

remove legacy service

0771036

remove legacy appender

7bae5e5

remove LegacyObjectToConfigAdapter

7b8aeb5

gix types

3e5c71a

Merge remote-tracking branch 'upstream/master' into kbn-50660-remove-…

1ca0080

…legacy-logging

remove @hapi/good / @hapi/good-squeeze / @hapi/podium

2aa840b

remove default appender validation for root logger

0d8b958

remove old config key from kibana-docker

6b25dde

pgayvallet added backport:skip This commit does not require backporting Feature:Logging release_note:breaking Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc v8.0.0 labels Sep 16, 2021

pgayvallet added 11 commits September 16, 2021 09:51

fix FTR config

cf931a4

fix dev server

a1a1568

remove reference from readme

25460af

fix unit test

32bbc99

clean CLI args and remove quiet option

b6f7cfd

fix type

5d2c1cc

fix status test config

f13bc1f

remove from test config

4d7927f

fix snapshot

afabe9e

use another regexp

d7cc6e7

update generated doc

db4edaa

Merge remote-tracking branch 'upstream/master' into kbn-50660-remove-…

fb4ff9e

…legacy-logging

Merge remote-tracking branch 'upstream/master' into kbn-50660-remove-…

e4aac52

…legacy-logging

pgayvallet added 3 commits September 22, 2021 08:08

Merge remote-tracking branch 'upstream/master' into kbn-50660-remove-…

14be47f

…legacy-logging

Merge remote-tracking branch 'upstream/master' into kbn-50660-remove-…

abd2e6f

…legacy-logging

remove legacy logging config deprecations

7c978a6

pgayvallet commented Sep 23, 2021

View reviewed changes

pgayvallet marked this pull request as ready for review September 23, 2021 10:19

pgayvallet requested review from a team as code owners September 23, 2021 10:19

Merge remote-tracking branch 'upstream/master' into kbn-50660-remove-…

c09776d

…legacy-logging

pgayvallet mentioned this pull request Sep 24, 2021

Remove deprecated Core settings #103915

Closed

Merge remote-tracking branch 'upstream/master' into kbn-50660-remove-…

67ce208

…legacy-logging

mshustov approved these changes Sep 27, 2021

View reviewed changes

rudolf approved these changes Sep 27, 2021

View reviewed changes

jportner approved these changes Sep 27, 2021

View reviewed changes

pgayvallet added 5 commits September 28, 2021 08:16

Merge remote-tracking branch 'upstream/master' into kbn-50660-remove-…

b33664b

…legacy-logging

address some review comments

2188399

update documentation

48efa70

update kibana.yml config examples

3e34e23

Merge remote-tracking branch 'upstream/master' into kbn-50660-remove-…

f41b60b

…legacy-logging

mistic approved these changes Sep 28, 2021

View reviewed changes

mshustov reviewed Sep 28, 2021

View reviewed changes

config/kibana.yml Show resolved Hide resolved

add config example for metrics.ops

234f4fb

pgayvallet mentioned this pull request Sep 29, 2021

Improve logging deprecations #113362

Merged

1 task

Merge remote-tracking branch 'upstream/master' into kbn-50660-remove-…

4957cd0

…legacy-logging

lukeelmers mentioned this pull request Oct 1, 2021

[logging] Upgrade ECS to 8.0 #113662

Merged

pgayvallet added 2 commits October 4, 2021 07:55

Merge remote-tracking branch 'upstream/master' into kbn-50660-remove-…

d97156c

…legacy-logging

Merge remote-tracking branch 'upstream/master' into kbn-50660-remove-…

bd20385

…legacy-logging

pgayvallet merged commit a4b74bd into elastic:master Oct 5, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[8.0] Remove legacy logging #112305

[8.0] Remove legacy logging #112305

pgayvallet commented Sep 15, 2021 •

edited by lukeelmers

Loading

pgayvallet commented Sep 17, 2021

tylersmalley commented Sep 20, 2021

kibanamachine commented Sep 20, 2021

pgayvallet left a comment

pgayvallet Sep 23, 2021

pgayvallet Sep 23, 2021

mshustov Sep 27, 2021

lukeelmers Sep 27, 2021

elasticmachine commented Sep 23, 2021

mshustov left a comment

mshustov Sep 27, 2021

mshustov Sep 27, 2021

jportner left a comment

mistic left a comment

kibanamachine commented Oct 5, 2021

	# Enables you to specify a file where Kibana stores log output.
	#logging.dest: stdout

	# Set the value of this setting to true to suppress all logging output.
	#logging.silent: false

	# Set the value of this setting to true to suppress all logging output other than error messages.
	#logging.quiet: false

	# Set the value of this setting to true to log all events, including system usage information
	# and all requests.
	#logging.verbose: false

[8.0] Remove legacy logging #112305

[8.0] Remove legacy logging #112305

Conversation

pgayvallet commented Sep 15, 2021 • edited by lukeelmers Loading

Summary

Tasks

Checklist

Release Note

pgayvallet commented Sep 17, 2021

tylersmalley commented Sep 20, 2021

kibanamachine commented Sep 20, 2021

pgayvallet left a comment

Choose a reason for hiding this comment

pgayvallet Sep 23, 2021

Choose a reason for hiding this comment

pgayvallet Sep 23, 2021

Choose a reason for hiding this comment

mshustov Sep 27, 2021

Choose a reason for hiding this comment

lukeelmers Sep 27, 2021

Choose a reason for hiding this comment

elasticmachine commented Sep 23, 2021

mshustov left a comment

Choose a reason for hiding this comment

mshustov Sep 27, 2021

Choose a reason for hiding this comment

mshustov Sep 27, 2021

Choose a reason for hiding this comment

jportner left a comment

Choose a reason for hiding this comment

mistic left a comment

Choose a reason for hiding this comment

kibanamachine commented Oct 5, 2021

💚 Build Succeeded

Metrics [docs]

Public APIs missing exports

History

pgayvallet commented Sep 15, 2021 •

edited by lukeelmers

Loading