Load and initialize all plugins concurrently

[afharo]

Summary

After #170856, we may want to parallelize the load of the plugin's code in an attempt to speed up Kibana's startup.

We are only parallelizing the part where we load the plugin and call the constructor but we still maintain the expected order of calling the plugins' setup and start methods.

Risk Matrix

Risk	Probability	Severity	Mitigation/Notes
Concurrently performing this action may cause unexpected spikes in CPU (and Memory?), leading to momentarily high Event Loop Delays.	Low	Low	Memory-wise, it shouldn't be a problem because we already hold all the plugins in memory. I wouldn't expect this approach to use much more. Regarding CPU, it might require higher processing power since it's performing parallel processing, while earlier, it was sequential. We need to test it, but I doubt this will cause any issues.

For maintainers

• This was checked for breaking API changes and was labeled appropriately

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[afharo]

According to our APM data, there doesn't seem to improve things. Which makes sense, given that loading the files from disk is the only thing that can be parallelized. Parsing and evaluating the JS files is synchronous.

Closing the PR

[kibana-ci]

💔 Build Failed

• Buildkite Build
• Commit: 0209c4e
• Interpreting CI Failures

Failed CI Steps

• FTR Configs #3
• FTR Configs #82
• FTR Configs #82

Test Failures

• [job] [logs] FTR Configs #82 / Monitoring Collection Prometheus endpoint returns prometheus scraped metrics
• [job] [logs] FTR Configs #82 / Monitoring Collection Prometheus endpoint returns prometheus scraped metrics

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @afharo