[Fleet] Remove fleetServerEnabled feature flag and use fleet system indices

[nchaulet]

Summary

Resolve #92422

In 7.13 we are going to make Fleet Server the default way to use Elastic Agent, in preparation of this, that PR:

• Remove the fleetServerEnabled flag
• Make using the fleet system indices
• remove unused code that used old Saved object.

This PR should not break anything as the kibana agents endpoint are still here (I will remove this endpoints in a following PR, after syncing with folks from endpoint and elastic package to ensure we do not break anything)

What to test?

You should be able to enroll agent throught Kibana as usual.

You should be able to enroll agent through Fleet server using the default Policy server see (TODO add link to fleet server readme)

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[jen-huang]

I added release note breaking tag, but actually I am not sure. Looks like the Kibana API routes will remain in place during this PR, are there actually any breaking changes here? Will Kibana API routes be removed during this release too?

[nchaulet]

@jen-huang thanks for adding the label, that PR will not remove the API routes, I would create a follow up PR that remove APIs as soon we are in sync with other teams that use Fleet, I will make sure to add the release note here.

[skh]

🚀 for the collector change, thanks!

[nchaulet]

@mtojek @EricDavisX I do not think agents saved object are directly used in tests or in elastic-package so we should be good right?

[mtojek]

@mtojek @EricDavisX I do not think saved object are directly used in tests or in elastic-package so we should be good right?

elastic-package:
We used to visit/fetch saved objects for exporting purposes (elastic-package export) and checking installed assets (asset tests). /cc @ycombinator

[nchaulet]

elastic-package:
We used to visit/fetch saved objects for exporting purposes (elastic-package export) and checking installed assets (asset tests).

Sorry I should have precised Fleet saved object like agents, enrolment api keys.

[EricDavisX]

@mtojek @EricDavisX I do not think [Fleet] agents saved object [like Agents, enrollment api keys] are directly used in tests or in elastic-package so we should be good right?

The e2e-testing scenarios do use the API to enroll / install agents and manipulate / apply policy, I suppose that would be impacted? I regret we don't have e2e-testing embedded into Kibana CI yet. But any manual testing done should cover the same usage right? should we use Observability 'deploy my pr' and test on this a bit to be sure?

[paul-tavares]

Thanks @nchaulet for making the changes in endpoint - those in this PR look good.

There are, however, several others that need to be done. Here is what I found:

• search for fleet- (the prefix to all saved object types): you will find several KQL filters (kuery=) that need to have the SO type removed from it.
• search for _SAVED_OBJECT_TYPE (the suffix of the const's that hold saved object type names): you will see several other areas that use those and thus also need to be adjusted

I found these in both security solution as well as in the functional test folders.

[mtojek]

@nchaulet

Sorry I should have precised Fleet saved object like agents, enrolment api keys.

Speaking about agents, I think the deepest level the elastic-package is interacting is FleetAPI :) For reference: https://github.com/elastic/elastic-package/blob/master/internal/kibana/agents.go#L100

BTW I really appreciate the heads up, thanks!

[nchaulet]

@paul-tavares I just did a pass to remove the fleet-agents: from endpoint, there is still one in the data collector, but I know there is already some work on your side to change that (#92311)

[paul-tavares]

Sorry for the delay @nchaulet . Looks good. Thanks for making the changes in Endpoint

[nchaulet]

@elasticmachine merge upstream

[nchaulet]

@elasticmachine merge upstream

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 4f0776c
• Storybooks Preview

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
fleet	755.6KB	754.7KB	-956.0B
securitySolution	7.8MB	7.8MB	-39.0B
total			-995.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
fleet	367.6KB	367.3KB	-259.0B

History

• 💔 Build #111506 failed 5d489d4
• 💔 Build #111436 failed 3ad9ae1
• 💔 Build #111180 failed 6826872
• 💚 Build #110788 succeeded fef3eec
• 💚 Build #110439 succeeded 77b5bad

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @nchaulet