Manage fleet server indices as system indices in the Elastic Stack

[scunningham]

Describe the enhancement:

Fleet Server relies on a set of indices to store operational data about Fleet. These indices must be created early in the system lifecycle. Currently the indices are instantiated in Kibana. This enhancement will move the instantiation to the Elastic Stack where the fleet indices will be treated as managed system indices. Moving forward, updates and migrations of these indices will be managed by the system indices plugins.

Describe a specific use case for the enhancement or feature:

We 7 indices we are dependent on, and one data stream with an ILM policy:

Indices

.fleet-actions
.fleet-agents
.fleet-enrollment-api-keys
.fleet-policies
.fleet-policies-leader
.fleet-servers
.fleet-artifacts

DataStream

.fleet-actions-results and ILM policy for the data stream.

Note: The indices creation had moved over to Kibana, however the data stream was still being created via package management. That was an oversight.

Known issues

• The indices/datastreams are not instantiated in the system indices plugin until the first document is written to them. The code in Kibana does not currently expect that. Is there a way to "touch" each of the indices at Kibana boot so we don't have to fix the reads of indices in Kibana to handle the "does not yet exist" case?
• System Indices does not yet support the concept of a system managed ILM policy. So the ILM policy for .fleet-action-results will likely remain in the package, or have to put into Kibana. Question on race condition: what happens if the package is installed before the data stream is instantiated? Does trying to add an ILM policy to a system managed data stream cause it to be created?

@jaymode will do the initial implementation in the Elastic Stack for this integration. Moving forward, the Fleet Team will pick up maintenance. This is expected to land for 7.13.

[jaymode]

I've opened elastic/elasticsearch#70689 for the indices and will follow-up on the data stream.

[ruflin]

Here is the related PR for data streams: elastic/elasticsearch#71667

[ruflin]

@scunningham @jaymode Going to close this as I think all changes landed. Can you confirm?

[scunningham]

Landed.