[Elastic Agent] Fix merging of fleet.yml. Add --staging to enroll cmd. #20026

blakerouse · 2020-07-17T13:41:02Z

What does this PR do?

Fixes an issue where the elastic-agent.yml and fleet.yml configurations where being merged, but the merged config was not being used in the Unpack call.

Adds a --staging f32a4a89 option to ./elastic-agent enroll. This allows users that are testing staging builds to be able to pull artifacts from that staging build.

Why is it important?

So elastic-agent.yml and fleet.yml are merged and used correctly. To simplify testing of staging builds.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
~~[ ] I have made corresponding change to the default configuration files~~
~~[ ] I have added tests that prove my fix is effective or that my feature works~~
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

 ./elastic_agent enroll --insecure http://localhost:5601 Q1JfZVhITUJsUURwelBhQVB2WVQ6NmlFSVNwZVVSU0tlcVI5dXZNNUtFZw --staging f32a4a89

Related issues

Closes [Ingest Manager] 7.9 BC1 Agent is not able to download a corresponding 7.9 Endpoint-Security artifact from the BC location #19961

…mmand line option to enroll.

elasticmachine · 2020-07-17T13:42:41Z

Pinging @elastic/ingest-management (Team:Ingest Management)

michalpristas

LGTM

michalpristas · 2020-07-17T14:13:51Z

x-pack/elastic-agent/pkg/agent/application/managed_mode.go


-	cfg, err := configuration.NewFromConfig(config)
+	cfg, err := configuration.NewFromConfig(rawConfig)


this is a good catch

michalpristas · 2020-07-17T14:14:12Z

x-pack/elastic-agent/pkg/agent/cmd/enroll.go

@@ -67,6 +68,13 @@ func enroll(streams *cli.IOStreams, cmd *cobra.Command, flags *globalFlags, args
 			errors.M(errors.MetaKeyPath, pathConfigFile))
 	}

+	staging, _ := cmd.Flags().GetString("staging")
+	if staging != "" {


could be a single line if

EricDavisX · 2020-07-17T15:20:27Z

@blakerouse this is great! can you elaborate for the sake of over communicating on this:
./elastic_agent enroll --insecure http://localhost:5601 Q1JfZVhITUJsUURwelBhQVB2WVQ6NmlFSVNwZVVSU0tlcVI5dXZNNUtFZw --staging f32a4a89

where does the f32a4a89 come from... I assume this is a hash of the build artifacts, like this:
pulling '33ee764c' from https://staging.elastic.co/7.9.0-33ee764c/summary-7.9.0.html

which is the HTML page that comes from the 7.9 main tracking issue: https://github.com/elastic/dev/issues/1442

blakerouse · 2020-07-17T15:31:11Z

@EricDavisX Correct, it comes from the HTML page of the build.

EricDavisX · 2020-07-17T15:50:03Z

thanks. this is so great!

EricDavisX · 2020-07-17T15:52:44Z

@blakerouse Also, does this PR fix the problem of Linux Agent not ‘finding’ Endpoint even when its on disk (manually copied there with the SHA 512 file)? Not sure if we need a new bug or not.

elastic#20026) * Fix usage of merged elastic-agent.yml and fleet.yml. Add --staging command line option to enroll. * Add to docs. * Add changelog. * Fix import sorting. (cherry picked from commit cfbd81b)

EricDavisX · 2020-07-17T16:39:43Z

FYI - we found the bug on my side and it was a personal script problem, not in the code - yay. we're good to go!

#20026) (#20032) * Fix usage of merged elastic-agent.yml and fleet.yml. Add --staging command line option to enroll. * Add to docs. * Add changelog. * Fix import sorting. (cherry picked from commit cfbd81b)

#20026) (#20031) * Fix usage of merged elastic-agent.yml and fleet.yml. Add --staging command line option to enroll. * Add to docs. * Add changelog. * Fix import sorting. (cherry picked from commit cfbd81b)

…ngs-archive * upstream/master: (274 commits) Test export commands in all Beats (elastic#20016) [Ingest Manager] Allow using drop path for agent build (elastic#20019) [DOCS] Remove include for deleted file in monitoring docs (elastic#20038) Removing xpack.monitoring.* settings (elastic#18608) [Heartbeat] Add service_name option for APM integration (elastic#19932) [Elastic Agent] Fix merging of fleet.yml. Add --staging to enroll cmd. (elastic#20026) Stricter stalebot configuration for PRs (elastic#20004) [ci] Favor direct mage invocation on CI (elastic#19960) Add cloudwatch input into Filebeat configure inputs documentation (elastic#19973) [Filebeat] remove delimiter \n from log line in s3 input (elastic#19972) [Metricbeat] Update MySQL dashboard (elastic#19913) Packetbeat process monitor: Ignore missing /proc/net/tcp6 (elastic#19945) [CI] fix MODULE variable cornercases (elastic#19985) Ignore timestamp in fortinet/clientendpoint and netscout/sightline (elastic#19998) add 7.9 to .backportrc.json (elastic#19952) Update internal links to external (elastic#19947) Remove Dynamic Script Compilations warning in Cisco module (elastic#19840) [Elastic Agent] Fix RPM and DEB packaging for Elastic Agent (elastic#19959) [Ingest Manager] Do not compare err with custom type (elastic#19980) Fix nanocore sum for non default intervals on Kubernetes Overview Dashboard (elastic#19675) ...

…nly-mod * upstream/master: (75 commits) Test export commands in all Beats (elastic#20016) [Ingest Manager] Allow using drop path for agent build (elastic#20019) [DOCS] Remove include for deleted file in monitoring docs (elastic#20038) Removing xpack.monitoring.* settings (elastic#18608) [Heartbeat] Add service_name option for APM integration (elastic#19932) [Elastic Agent] Fix merging of fleet.yml. Add --staging to enroll cmd. (elastic#20026) Stricter stalebot configuration for PRs (elastic#20004) [ci] Favor direct mage invocation on CI (elastic#19960) Add cloudwatch input into Filebeat configure inputs documentation (elastic#19973) [Filebeat] remove delimiter \n from log line in s3 input (elastic#19972) [Metricbeat] Update MySQL dashboard (elastic#19913) Packetbeat process monitor: Ignore missing /proc/net/tcp6 (elastic#19945) [CI] fix MODULE variable cornercases (elastic#19985) Ignore timestamp in fortinet/clientendpoint and netscout/sightline (elastic#19998) add 7.9 to .backportrc.json (elastic#19952) Update internal links to external (elastic#19947) Remove Dynamic Script Compilations warning in Cisco module (elastic#19840) [Elastic Agent] Fix RPM and DEB packaging for Elastic Agent (elastic#19959) [Ingest Manager] Do not compare err with custom type (elastic#19980) Fix nanocore sum for non default intervals on Kubernetes Overview Dashboard (elastic#19675) ...

EricDavisX · 2020-07-22T23:12:26Z

note - this works great, tested with BC3 - we're in business!

elastic#20026) * Fix usage of merged elastic-agent.yml and fleet.yml. Add --staging command line option to enroll. * Add to docs. * Add changelog. * Fix import sorting.

elastic#20026) (elastic#20032) * Fix usage of merged elastic-agent.yml and fleet.yml. Add --staging command line option to enroll. * Add to docs. * Add changelog. * Fix import sorting. (cherry picked from commit c84dd9e)

blakerouse added 2 commits July 17, 2020 09:35

Fix usage of merged elastic-agent.yml and fleet.yml. Add --staging co…

6cfc37b

…mmand line option to enroll.

Add to docs.

46104c4

blakerouse added the Team:Ingest Management label Jul 17, 2020

blakerouse self-assigned this Jul 17, 2020

botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jul 17, 2020

Add changelog.

65a7905

blakerouse marked this pull request as ready for review July 17, 2020 13:42

Fix import sorting.

e6b637c

michalpristas approved these changes Jul 17, 2020

View reviewed changes

blakerouse merged commit cfbd81b into elastic:master Jul 17, 2020

blakerouse deleted the agent-enroll-cmd-staging branch July 17, 2020 15:57

blakerouse mentioned this pull request Jul 17, 2020

Cherry-pick #20026 to 7.x: [Elastic Agent] Fix merging of fleet.yml. Add --staging to enroll cmd. #20031

Merged

4 tasks

blakerouse added the v7.10.0 label Jul 17, 2020

blakerouse mentioned this pull request Jul 17, 2020

Cherry-pick #20026 to 7.9: [Elastic Agent] Fix merging of fleet.yml. Add --staging to enroll cmd. #20032

Merged

4 tasks

blakerouse added the v7.9.0 label Jul 17, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Elastic Agent] Fix merging of fleet.yml. Add --staging to enroll cmd. #20026

[Elastic Agent] Fix merging of fleet.yml. Add --staging to enroll cmd. #20026

blakerouse commented Jul 17, 2020 •

edited

Loading

elasticmachine commented Jul 17, 2020

michalpristas left a comment

michalpristas Jul 17, 2020

michalpristas Jul 17, 2020

EricDavisX commented Jul 17, 2020

blakerouse commented Jul 17, 2020

EricDavisX commented Jul 17, 2020

EricDavisX commented Jul 17, 2020 •

edited

Loading

EricDavisX commented Jul 17, 2020

EricDavisX commented Jul 22, 2020


		cfg, err := configuration.NewFromConfig(config)
		cfg, err := configuration.NewFromConfig(rawConfig)

[Elastic Agent] Fix merging of fleet.yml. Add --staging to enroll cmd. #20026

[Elastic Agent] Fix merging of fleet.yml. Add --staging to enroll cmd. #20026

Conversation

blakerouse commented Jul 17, 2020 • edited Loading

What does this PR do?

Why is it important?

Checklist

How to test this PR locally

Related issues

elasticmachine commented Jul 17, 2020

michalpristas left a comment

Choose a reason for hiding this comment

michalpristas Jul 17, 2020

Choose a reason for hiding this comment

michalpristas Jul 17, 2020

Choose a reason for hiding this comment

EricDavisX commented Jul 17, 2020

blakerouse commented Jul 17, 2020

EricDavisX commented Jul 17, 2020

EricDavisX commented Jul 17, 2020 • edited Loading

EricDavisX commented Jul 17, 2020

EricDavisX commented Jul 22, 2020

blakerouse commented Jul 17, 2020 •

edited

Loading

EricDavisX commented Jul 17, 2020 •

edited

Loading