[Fleet] Add .fleet-secrets system index

[hop-dev]

Part of elastic/kibana#154715

Closes #95143

The .fleet-secrets index will contain agent policy secrets. An example of a secret is a password or API key. Kibana has write access only to the index, and fleet server has read access.

I have kept it in the fleet area as it is managed by us.

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[elasticsearchmachine]

Hi @hop-dev, I've created a changelog YAML for you.

[rjernst]

I've asked @williamrandolph to take a look, but in the meantime, I have one general thought. While I understand the note about the intention to use this index with enterprise search in the future, I think the name is too generic. It could easily be confused by developers as a general purpose index within Elasticsearch internals, such as .security. I think the name should be narrowed.

One idea is .external-secrets. Another (and my preferred) is to use .fleet-secrets here, and use another system index name for enterprise search. In any case, these are system indices, so the preferred approach is to wrap these indices with rest apis which have complete control over them. Then the particular index name matters even less.

[jimczi]

One idea is .external-secrets. Another (and my preferred) is to use .fleet-secrets here, and use another system index name for enterprise search. In any case, these are system indices, so the preferred approach is to wrap these indices with rest apis which have complete control over them. Then the particular index name matters even less.

+1 for .fleet-secrets, the sharing is not mandatory and we might want different rest APIs for fleet and enterprise-search.

[hop-dev]

@rjernst @jimczi on reflection I think you are completely right, thanks for the feedback.

I will test the change now and get back to you 👍

[hop-dev]

@rjernst @jimczi I have moved to .fleet-secrets now

[elasticsearchmachine]

Hi @hop-dev, I've updated the changelog YAML for you.

[juliaElastic]

@rjernst @jimczi Are you ready to approve? We are waiting for this to go ahead with the Fleet feature.

[juliaElastic]

@hop-dev updated a few remaining occurrences of .secrets.

[hop-dev]

@juliaElastic Thanks for that! I have re-posted in #es-core-infra to try and get a review

[rjernst]

LGTM