APM Server on cloud is broken for 8.4.0-SNAPSHOT

[simitt]

APM Server version (apm-server version): 8.4.0-SNAPSHOT

Description of the problem including expected versus actual behavior:
APM Server version 8.4.0-SNAPSHOT cannot startup on ESS. There are a couple of unexpected log entries:

• RUM is logged as disabled: RUM endpoints disabled.
• SSL is logged as disabled: SSL disabled.
• ES output is set to default value instead of the cloud injected ES value : elasticsearch url: http://localhost:9200
• when changing some config following error is logged:

Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data).

It looks like something has been broken with respect to configuration/reloading.

Steps to reproduce:

1. Create deployment on ESS and observe that APM Server is down and logs errors

[marclop]

@pazone the benchmarks seem to be failing for this reason. Still unknown why this is happening since we've got some tests covering this behavior locally and it seems to work on main for me (not on ESS).

[axw]

I'm pretty sure this is related to the recent build changes I made, which inadvertently have us building without the Fleet management code. This means APM Server doesn't pick up any config from Fleet.

[pazone]

Question: do we have a system test(s) which is supposed to fail in this case?

[simitt]

@marclop's the newly introduced smoke tests (#8477) shave have caught this right?

[marclop]

@simitt they would have, if they had been running on a daily schedule.

[pazone]

@simitt @marclop We can do that. Not sure about daily schedule. Probably we can run the tests after a merge to main or on snapshot release. Let's create an issue?

[pazone]

I see it's already there #8303