Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run only providers that have references in the policy #3609

Closed
blakerouse opened this issue Oct 16, 2023 · 3 comments · Fixed by #6169
Closed

Run only providers that have references in the policy #3609

blakerouse opened this issue Oct 16, 2023 · 3 comments · Fixed by #6169
Labels
Team:Elastic-Agent Label for the Agent team Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team

Comments

@blakerouse
Copy link
Contributor

Describe the enhancement:

Currently the Elastic Agent runs all of its providers all the time. Context providers are rather low overhead, but dynamic providers are not and when they are not even being used its adding load for no benefit to the running policy.

We should adjust the Elastic Agent to only run providers that are referenced in the policy. Variable referencing is always prefixed with a provider name aka ${kubernetes.*} that means its rather easy to identify which providers should be enabled based on what is in the policy.

This also removes the need to add any configuration to Fleet to disable or enable a provide for the Elastic Agent. If the policy from Fleet doesn't use a provider then the Elastic Agent will not run it.

Describe a specific use case for the enhancement or feature:

Reduce load of an Elastic Agent, prevents it from trying to communicate with Docker or Kubernetes in the case that its not even running a Docker or Kubernetes inputs in the policy.

What is the definition of done?

I propose the following change:

  1. Parse the policy into the AST.
  2. Read the AST to determine the providers that are referenced.
  3. Start/stop the providers that are referenced or no longer referenced.
@blakerouse blakerouse added the Team:Elastic-Agent Label for the Agent team label Oct 16, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

@cmacknz
Copy link
Member

cmacknz commented Oct 16, 2023

An additional problem we discovered recently is that the .enabled configuration for a provider is only read once at startup when the controller is first created:

// New creates a new controller.
func New(log *logger.Logger, c *config.Config, managed bool) (Controller, error) {
l := log.Named("composable")
var providersCfg Config
if c != nil {
err := c.Unpack(&providersCfg)
if err != nil {
return nil, errors.New(err, "failed to unpack providers config", errors.TypeConfig)
}
}
// build all the context providers
contextProviders := map[string]*contextProviderState{}
for name, builder := range Providers.contextProviders {
pCfg, ok := providersCfg.Providers[name]
if ok && !pCfg.Enabled() {
// explicitly disabled; skipping
continue
}

This means disabling a provider in the policy has no effect unless you also restart the agent. The change proposed here would also fix this problem.

For context we were attempting to disable the k8s leader election provider with a Fleet policy override and found this wasn't possible without restarting the agent manually afterwards.

@elasticmachine
Copy link
Contributor

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Elastic-Agent Label for the Agent team Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants