[Fleet] Requirement for Kibana API Key for Agent subprocess

[simitt]

APM Server requires access to Kibana for supporting the APM Server Central Configuration Management feature. The API Key needs to have spaces read privileges for the APM app.

[elasticmachine]

Pinging @elastic/fleet (Feature:Fleet)

[simitt]

It looks like the endpoint integration is receiving a Kibana API Key by requesting fleet.access_api_key and fleet.kibana already, see spec/endpoint.yml.

pinging @scunningham and @ruflin as you have been involved in previous conversations around this.

[elasticmachine]

Pinging @elastic/ingest-management (Team:Ingest Management)

[scunningham]

@blakerouse

[blakerouse]

@simitt Elastic Agent does send the Endpoint the access_api_key and kibana connection information. But this is not custom generated for Endpoint. This is the same connection and API key information that Elastic Agent uses to contact Kibana.

[simitt]

Thanks @blakerouse; do you know which privileges that API Key has (or could you link to the code)?

I am not sure it makese sense to reuse this access_api_key for specific APM features (which would require privileges for the APM Kibana space). @jalvz has already put more thoughts and efforts into this and probably has an opinion on what would make the most sense/might be a short term solution.

[blakerouse]

@simitt For the access_api_key it is created at enroll time by Fleet Server with the following roles:

https://github.com/elastic/fleet-server/blob/8fb511292c4c62d2b0fd0291a31100be86e3a55c/cmd/fleet/schema.go#L25

[simitt]

Update: pulled out the requirements for ES API Keys in inputs and created a dedicated issue #91704; changed the description to make this issue focused on Kibana API Key requirements for APM.

[scunningham]

I am concerned that the apm server will not have a network route to Kibana when running under the Fleet Agent. Reading the ticket, this seems like a possible candidate for policy integration. Unclear if that would solve the problem. It would depend on the nature of the configuration data:

• Does it make sense on a per policy basis, or is it more granular than that.
• How often does it change? Constantly changing policy will probably break things.

An artifact implementation could work as well, but that too is tied to policy.

[jalvz]

I am concerned that the apm server will not have a network route to Kibana when running under the Fleet Agent

Oh, I understood otherwise from elastic/beats#23856 (comment), but I am not familiar enough with fleet server.

What is a "policy integration"? We just need one connection to Kibana with an API key with the right privileges, we don't need more granularity than that - and I don't expect apm policies to change often (if that helps).

[ruflin]

My suggest is what we should do for now is that APM includes the necessary Kibana URL and API key directly inside the APM input config. Like this we have a temporary solution and the Elastic Agent does not require any knowledge about it.

[graphaelli]

@simitt and I discussed - we still need to determine what needs to be done here to figure out who will do it, depending on the outcome of the indexing permissions discussion. As of now this is not slated for a specific milestone.

[simitt]

Depending on elastic/apm-server#4573 (comment) we might not need to solve this for APM for 7.13.

[ruflin]

Update: What @simitt posted above with elastic/apm-server#4573 (comment) is what should be done.

I don't think this depends on the index permission discussion as we are discussing here access to Kibana. Long term, apm-server should not require access to Kibana at all. Instead, the central config changes for the APM Agents should be pushed as part of the integration policy. There should be only 1 delivery mechanism for Cloud.

Until we get there, APM should create its own API Key for Kibana + URL and add it to the APM Integration policy.

[simitt]

Closing this in favor of #93420