[Elastic Agent] Enroll with Fleet Server #23865
Conversation
botelastic
bot
added
needs_team
|
Pinging @elastic/agent (Team:Agent) |
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
Trends 🧪💚 Flaky test reportTests succeeded. Expand to view the summary
Test stats 🧪
|
|
|
||
| // enroll should use localhost as fleet-server is now running | ||
| // it must also restart | ||
| c.options.URL = "http://localhost:8000" |
There was a problem hiding this comment.
Yes as it will communicate with the Fleet Server locally. There is currently not a way to setup SSL or run it on a different port through the enroll command.
Definitely things we need to look into, but not in this PR. This is just enough to get it up and running.
| } | ||
|
|
||
| if c.options.NoRestart { | ||
| return err |
| if c.daemonReload(ctx) != nil { | ||
| c.log.Info("Elastic Agent might not be running; unable to trigger restart") | ||
| } | ||
| c.log.Info("Successfully triggered restart on running Elastic Agent.") |
There was a problem hiding this comment.
nit: thought we agreed on lower-case logs. as long as it is consistent.
There was a problem hiding this comment.
That was in Fleet Server repository. This is more consistent in Elastic Agent, even though I am not a fan of it.
| c.log.Info("Elastic Agent might not be running; unable to trigger restart") | ||
| } | ||
| c.log.Info("Successfully triggered restart on running Elastic Agent.") | ||
| return err |
| } | ||
|
|
||
| func (c *EnrollCmd) fleetServerBootstrap(ctx context.Context) error { | ||
| c.log.Debug("verifying communication with running elastic-agent daemon") |
There was a problem hiding this comment.
nit: consistency
running Elastic Agent
and
running elastic-agent
in the same file
| // Degraded status means something minor is preventing agent to work properly. | ||
| Degraded | ||
| // Failed status means agent is unable to work properly. | ||
| Failed | ||
| ) | ||
|
|
||
| var ( | ||
| humanReadableStatuses = map[AgentStatus]string{ | ||
| humanReadableStatuses = map[AgentStatusCode]string{ |
There was a problem hiding this comment.
remove this map
and use:
func (s AgentStatusCode) String() string {
return []string{"online", "degraded", "error"}[s]
}
it's more idiomatic
| notifyChangeFunc: r.updateStatus, | ||
| } | ||
|
|
||
| r.lock.Lock() |
| } | ||
| rep.lock.Unlock() | ||
| i++ | ||
| } |
There was a problem hiding this comment.
nit: shorter, no i++
apps := make([]AgentApplicationStatus, 0, len(r.appReporters))
for key, rep := range r.appReporters {
rep.lock.Lock()
apps = append(apps, AgentApplicationStatus{
ID: key,
Name: rep.name,
Status: rep.status,
Message: rep.message,
})
rep.lock.Unlock()
}
```
| @@ -125,6 +195,19 @@ func (r *controller) updateStatus() { | |||
| break | |||
| } | |||
| } | |||
| if status != Failed { | |||
| for id, rep := range r.appReporters { | |||
| s := statusToAgentStatus(rep.status) | |||
There was a problem hiding this comment.
you were using locks above
rep.lock.Lock()
but not here
| status = s | ||
| } | ||
|
|
||
| r.log.Debugf("'%s' has status '%s'", id, humanReadableStatuses[s]) |
There was a problem hiding this comment.
once you change the enum code above, then could do just
r.log.Debugf("'%s' has status '%s'", id, s)
|
Pinging @elastic/ingest-management (Team:Ingest Management) |
blakerouse
force-pushed
the
enroll-with-fleet-server
branch
from
7d57806 to
50966c6
Compare
|
/package |
|
@blakerouse Do we have to specify the policy id if we use and enrollment token (the token should contain a policy id) |
|
@nchaulet At the moment you need both, if we could simplify it to only one that would be better. |
Yes I think we can simplify it an enrollment key is always linked to a policy so it could work without the policy id |
* Add test and changelog. * Add ability to enroll through a local Fleet Server started by the running Elastic Agent daemon. * Fix tests. * Fix changelog. * Fixes from code review. * Cleanup from merge into master. (cherry picked from commit ae0f29e)
* Add test and changelog. * Add ability to enroll through a local Fleet Server started by the running Elastic Agent daemon. * Fix tests. * Fix changelog. * Fixes from code review. * Cleanup from merge into master. (cherry picked from commit ae0f29e)
…-arm * upstream/master: [CI] install docker-compose with retry (elastic#24069) Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes elastic#24051 (elastic#24052) updating manifest files for filebeat threatintel module (elastic#24074) Add Zeek Signatures (elastic#23772) Update Beats to ECS 1.8.0 (elastic#23465) Support running Docker logging plugin on ARM64 (elastic#24034) Fix ec2 metricset fields.yml and add integration test (elastic#23726) Only build targz and zip versions of Beats if PACKAGES is set in agent (elastic#24060) [Filebeat] Add field definitions for known Netflow/IPFIX vendor fields (elastic#23773) [Elastic Agent] Enroll with Fleet Server (elastic#23865) [Filebeat] Convert logstash logEvent.action objects to strings (elastic#23944) [Ingest Management] Fix reloading of log level for services (elastic#24055) Add Agent standalone k8s manifest (elastic#23679)
…dows-7 * upstream/master: (332 commits) Use ECS v1.8.0 (elastic#24086) Add support for postgresql csv logs (elastic#23334) [Heartbeat] Refactor config system (elastic#23467) [CI] install docker-compose with retry (elastic#24069) Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes elastic#24051 (elastic#24052) updating manifest files for filebeat threatintel module (elastic#24074) Add Zeek Signatures (elastic#23772) Update Beats to ECS 1.8.0 (elastic#23465) Support running Docker logging plugin on ARM64 (elastic#24034) Fix ec2 metricset fields.yml and add integration test (elastic#23726) Only build targz and zip versions of Beats if PACKAGES is set in agent (elastic#24060) [Filebeat] Add field definitions for known Netflow/IPFIX vendor fields (elastic#23773) [Elastic Agent] Enroll with Fleet Server (elastic#23865) [Filebeat] Convert logstash logEvent.action objects to strings (elastic#23944) [Ingest Management] Fix reloading of log level for services (elastic#24055) Add Agent standalone k8s manifest (elastic#23679) [Metricbeat][Kubernetes] Extend state_node with more conditions (elastic#23905) [CI] googleStorageUploadExt step (elastic#24048) Check fields are documented for aws metricsets (elastic#23887) Update go-concert to 0.1.0 (elastic#23770) ...
What does this PR do?
This adds the ability to enroll the Elastic Agent with Fleet Server executed locally on the same machine. To get this work a few things needed to be added to Elastic Agent.
status.Controllerto the socket control protocol.Note: This has a breaking change in the parameters to
enroll.kibana_urlandenrollment_tokenmove to being parameters instead of positional arguments. This makesinstallandenrolltake the same parameters, and closes #21897.Why is it important?
So that Fleet Server can be bootstrapped on a machine with Fleet Server also running on that same machine.
How does it work?
The
enrollcommand handles the coordination of controlling the running Elastic Agent daemon. Theinstallcommand proxies to theenrollcommand so this can be ran from theinstallor from the DEB/RPM.Breakdown of the steps that are completed to handle the bootstrap:
--fleet-serverparameter. This parameter is a connection string for Fleet Server to communicate to elasticsearch. (Example:--fleet-server http://elastic:changeme@localhost:9200)fleet.ymlwithfleet.serverconfiguration, withfleet.server.bootstrap: true.statusGRPC of the Elastic Agent until Fleet Server is started and is in degraded state (should be degraded, because the Elastic Agent is not enrolled yet).fleet.ymlwith enrollment information and thefleet.serverinformation. Thefleet.server.bootstrapis removed (aka. False).fleet.serveris set in thefleet.yml).Checklist
[ ] I have made corresponding changes to the documentation[ ] I have made corresponding change to the default configuration filesCHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.Author's Checklist
--fleet-serverworks.How to test this PR locally
Run the latest
8.0.0-SNAPSHOTof elasticsearch and Kibana. Start Kibana with thexpack.fleet.agents.fleetServerEnabled: true.Add the
Fleet Serverintegration to a policy.Look up the policy ID (as this is currently needed until a default policy for Fleet Server is added to Kibana).
Start Elastic Agent.
Run the following command to bootstrap and enroll the Elastic Agent.
./elastic-agent enroll --insecure --url http://localhost:8000 --enrollment-token {token} --fleet-server http://elastic:changeme@localhost:9200 --fleet-server-policy {policy_id}Related issues