Add Central Management feature #8559

exekias · 2018-10-03T21:52:42Z

Kibana will offer Central Management for Beats, this is the workflow to use it:

Enroll the Beat into Kibana, using the new added enroll command:

metricbeat enroll http://localhost:5601 d16a2ede945b4d9395c1ce0ff6e663f1

Once running, the beat will retrieve configs from Kibana and apply the changes locally.

All state is stored under meta/management.yml

Note: this is a merge from a feature branch, all PRs to it have been reviewed already

This PR implements intial enrollment to Central Management in Kibana. After running the enrollment command, beats will have a valid access token to use when retrieving configurations. To test this: - Use the following branches: - Elasticsearch: https://github.com/ycombinator/elasticsearch/tree/x-pack/management/beats - Kibana: https://github.com/elastic/kibana/tree/feature/x-pack/management/beats - Retrieve a valid enrollment token: ``` curl \ -u elastic \ -H 'kbn-xsrf: foobar' \ -H 'Content-Type: application/json' \ -X POST \ http://localhost:5601/api/beats/enrollment_tokens ``` - Use it: ``` <beat> enroll http://localhost:5601 <enrollment_token> ``` - Check agent is enrolled: ``` curl http://localhost:5601/api/beats/agents | jq ``` This is part of elastic#7028, closes elastic#7032

* Add config manager initial skeleton Config manager will poll configs from Kibana and apply them locally. It must be started with the beat. In order to check the user is not trying to override configurations provided by central management, the Config Manager can check the exisitng configuration and return errors if something is wrong.

* Refactor configs storage to avoid YAML issues * Refactor manager loop to avoid repeated code

This should make Auditbeat or any other beat based on Metricbeat have their own namespace for confs

* Allow user/passwd based enrollment This allows to enroll using the following workflow: ``` $ <beat> enroll http://kibana:5601 --username elastic Enter password: Enrolled and ready to retrieve settings from Kibana ``` It also allows to pass the password as an env variable: ``` PASS=... $ <beat> enroll http://kibana:5601 --username elastic --password env:PASS Enrolled and ready to retrieve settings from Kibana ``` * Fix some strings after review comments

exekias · 2018-10-03T21:56:49Z

Note to self: this PR uses satori/go.uuid lib, which is being replaced in #8458, we can expect some minor conflicts depending on which one goes in first

ruflin

LGTM.

I assume config changes and docs will follow?

ruflin · 2018-10-04T11:14:01Z

libbeat/kibana/client.go

@@ -145,8 +145,10 @@ func NewClientWithConfig(config *ClientConfig) (*Client, error) {
 		},
 	}

-	if err = client.SetVersion(); err != nil {
-		return nil, fmt.Errorf("fail to get the Kibana version: %v", err)
+	if !config.IgnoreVersion {


Can you remind me in which case we need this?

basically when doing enrollment, Kibana will probably be using x-pack security, so you need user/password to access the version, we don't have it, as enrollment endpoints don't require username/password but a valid token.

This change allows to configure the client to ignore kibana version check when doing enrollment

Got it. Chicken / egg problem here :-)

exekias · 2018-10-04T11:50:34Z

Yes, there is an ongoing effort for docs here: #8470

* Beats enrollment subcommand (elastic#7182) This PR implements intial enrollment to Central Management in Kibana. After running the enrollment command, beats will have a valid access token to use when retrieving configurations. To test this: - Use the following branches: - Elasticsearch: https://github.com/ycombinator/elasticsearch/tree/x-pack/management/beats - Kibana: https://github.com/elastic/kibana/tree/feature/x-pack/management/beats - Retrieve a valid enrollment token: ``` curl \ -u elastic \ -H 'kbn-xsrf: foobar' \ -H 'Content-Type: application/json' \ -X POST \ http://localhost:5601/api/beats/enrollment_tokens ``` - Use it: ``` <beat> enroll http://localhost:5601 <enrollment_token> ``` - Check agent is enrolled: ``` curl http://localhost:5601/api/beats/agents | jq ``` This is part of elastic#7028, closes elastic#7032 * Add API client to retrieve configurations from CM (elastic#8155) * Add central management service (elastic#8263) * Add config manager initial skeleton Config manager will poll configs from Kibana and apply them locally. It must be started with the beat. In order to check the user is not trying to override configurations provided by central management, the Config Manager can check the exisitng configuration and return errors if something is wrong. * Register output for reloading (elastic#8378) * Also send beat name when enrolling (elastic#8380) * Refactor how configs are stored (elastic#8379) * Refactor configs storage to avoid YAML issues * Refactor manager loop to avoid repeated code * Use beat name var when registering confs (elastic#8435) This should make Auditbeat or any other beat based on Metricbeat have their own namespace for confs * Allow user/passwd based enrollment (elastic#8524) * Allow user/passwd based enrollment This allows to enroll using the following workflow: ``` $ <beat> enroll http://kibana:5601 --username elastic Enter password: Enrolled and ready to retrieve settings from Kibana ``` It also allows to pass the password as an env variable: ``` PASS=... $ <beat> enroll http://kibana:5601 --username elastic --password env:PASS Enrolled and ready to retrieve settings from Kibana ``` * Fix some strings after review comments * Add changelog (cherry picked from commit 4247bc3)

* Add Central Management feature (#8559) * Beats enrollment subcommand (#7182) This PR implements intial enrollment to Central Management in Kibana. After running the enrollment command, beats will have a valid access token to use when retrieving configurations. To test this: - Use the following branches: - Elasticsearch: https://github.com/ycombinator/elasticsearch/tree/x-pack/management/beats - Kibana: https://github.com/elastic/kibana/tree/feature/x-pack/management/beats - Retrieve a valid enrollment token: ``` curl \ -u elastic \ -H 'kbn-xsrf: foobar' \ -H 'Content-Type: application/json' \ -X POST \ http://localhost:5601/api/beats/enrollment_tokens ``` - Use it: ``` <beat> enroll http://localhost:5601 <enrollment_token> ``` - Check agent is enrolled: ``` curl http://localhost:5601/api/beats/agents | jq ``` This is part of #7028, closes #7032 * Add API client to retrieve configurations from CM (#8155) * Add central management service (#8263) * Add config manager initial skeleton Config manager will poll configs from Kibana and apply them locally. It must be started with the beat. In order to check the user is not trying to override configurations provided by central management, the Config Manager can check the exisitng configuration and return errors if something is wrong. * Register output for reloading (#8378) * Also send beat name when enrolling (#8380) * Refactor how configs are stored (#8379) * Refactor configs storage to avoid YAML issues * Refactor manager loop to avoid repeated code * Use beat name var when registering confs (#8435) This should make Auditbeat or any other beat based on Metricbeat have their own namespace for confs * Allow user/passwd based enrollment (#8524) * Allow user/passwd based enrollment This allows to enroll using the following workflow: ``` $ <beat> enroll http://kibana:5601 --username elastic Enter password: Enrolled and ready to retrieve settings from Kibana ``` It also allows to pass the password as an env variable: ``` PASS=... $ <beat> enroll http://kibana:5601 --username elastic --password env:PASS Enrolled and ready to retrieve settings from Kibana ``` * Fix some strings after review comments * Add changelog (cherry picked from commit 4247bc3) * Fix monitoring registry usage

urso · 2018-10-19T15:39:28Z

libbeat/publisher/pipeline/pipeline.go

@@ -151,6 +153,7 @@ type queueFactory func(queue.Eventer) (queue.Queue, error)
 // queue and outputs will be closed.
 func New(
 	beat beat.Info,
+	monitors Monitors,


Todo: monitors contains logger, metrics + telemetry registry. Why still pass 'metrics', is this a different instance?

urso · 2018-10-19T15:55:59Z

libbeat/monitoring/report/elasticsearch/elasticsearch.go

@@ -172,6 +172,7 @@ func makeReporter(beat beat.Info, settings report.Settings, cfg *common.Config)

 	pipeline, err := pipeline.New(
 		beat,
+		pipeline.Monitors{},


Why pass an empty instance here? This removes the x-pack.monitoring.output metrics from each beat.

It's also a good idea to create a logger named x-pack.monitoring.

exekias and others added 9 commits September 6, 2018 16:30

Add API client to retrieve configurations from CM (elastic#8155)

229e900

Register output for reloading (elastic#8378)

268be1a

Also send beat name when enrolling (elastic#8380)

5251fe1

Refactor how configs are stored (elastic#8379)

d3573bc

* Refactor configs storage to avoid YAML issues * Refactor manager loop to avoid repeated code

Use beat name var when registering confs (elastic#8435)

a951985

This should make Auditbeat or any other beat based on Metricbeat have their own namespace for confs

Merge branch 'master' into feature-centralmgmt

4ef8108

exekias added enhancement review libbeat labels Oct 3, 2018

Add changelog

a43e3f1

exekias requested a review from ruflin October 3, 2018 21:57

exekias added the needs_backport PR is waiting to be backported to other branches. label Oct 3, 2018

This was referenced Oct 3, 2018

Central Management #7028

Closed

Reconfigure outputs on the fly #3861

Closed

Register filebeat inputs & modules for central management #8560

Merged

ruflin approved these changes Oct 4, 2018

View reviewed changes

exekias merged commit 4247bc3 into elastic:master Oct 4, 2018

exekias mentioned this pull request Oct 4, 2018

Switch to different UUID lib due to non-random generated UUIDs #8458

Merged

1 task

exekias mentioned this pull request Oct 5, 2018

Cherry-pick #8559 to 6.x: Add Central Management feature #8584

Merged

exekias added v6.5.0 and removed needs_backport PR is waiting to be backported to other branches. labels Oct 5, 2018

This was referenced Oct 5, 2018

Cherry-pick #8584 to 6.x: Cherry-pick #8559 to 6.x: Add Central Management feature #8589

Closed

Cherry-pick #8584 to 6.x: Cherry-pick #8559 to 6.x: Add Central Management feature #8590

Closed

urso reviewed Oct 19, 2018

View reviewed changes

exekias mentioned this pull request Nov 5, 2018

Do not fail when reloading outputs #8929

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Central Management feature #8559

Add Central Management feature #8559

exekias commented Oct 3, 2018

exekias commented Oct 3, 2018

ruflin left a comment

ruflin Oct 4, 2018

exekias Oct 4, 2018

ruflin Oct 4, 2018

exekias commented Oct 4, 2018

urso Oct 19, 2018

urso Oct 19, 2018

Add Central Management feature #8559

Add Central Management feature #8559

Conversation

exekias commented Oct 3, 2018

exekias commented Oct 3, 2018

ruflin left a comment

Choose a reason for hiding this comment

ruflin Oct 4, 2018

Choose a reason for hiding this comment

exekias Oct 4, 2018

Choose a reason for hiding this comment

ruflin Oct 4, 2018

Choose a reason for hiding this comment

exekias commented Oct 4, 2018

urso Oct 19, 2018

Choose a reason for hiding this comment

urso Oct 19, 2018

Choose a reason for hiding this comment