[Security Solution] Fix performance issues affecting rules management

[xcrzx]

Summary

This PR fixes some of the performance issues identified here #134826.

• Set the stale time of installed integration responses to 15 minutes, so we don't re-fetch them too frequently.
• Refactor retrieval of the status of the prepackaged rules to use useQuery. That allows us to fetch the prepackaged rules statuses only once and re-fetch them only after server state mutations.
• Refactored the installPrepackagedRules method to use the promise pool and respect the MAX_RULES_TO_UPDATE_IN_PARALLEL setting.
• Wrapped some code paths in withSecuritySpan for better discoverability in APM.

Results

Before the rules management table took 950ms to render for 95% of users

Now, the rules management table takes less than 300ms to render for 95% of users (3x faster)

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[spong]

Appreciate the comment/context here. This is one of those things the team just needs to know about when working with mutations as it would be easy to add a mutation operation and not know you have to invalidate the PrePackagedStatus. If we guaranteed all rule access was going through a specific hook/API, we could invalidate at that layer, but not much else can be done, so thanks for the context here 👍

[spong]

The tech debt cleanup here is fantastic @xcrzx! Like finally paying off 150 year mortgage that has been following your family for generations, haha! 😅 Thanks for tackling this one -- so much cleaner and easier to grok the actual control flow with this, usePrePackagedRulesStatus, and useInstallPrePackagedRules. 💯💯💯

[spong]

Checked out, tested locally, and performed code review -- LGTM! 👍

Couple nits/questions around cache expiration timing (and will have to resolve conflicts with main coming from the recent schema changes), but everything else looks great @xcrzx! Thanks for the massive tech debt cleanup here, further APM instrumentation, and general perf cleanup as well. 🎉 🙌 🚀

[YulNaumenko]

LGTM!

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: fcd603f
• Storybooks Preview
• Webpack Bundle Analyzer

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
securitySolution	3114	3116	+2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
securitySolution	5.2MB	5.2MB	-568.0B

History

• 💔 Build #56576 failed 0278c47c0614a08a7af1c61aa0af6a30d65a07c2
• 💚 Build #56360 succeeded 05f7b3589c7ccfbafad317b8d25705f391d5ce90
• 💔 Build #56289 failed 99eb6a36236f575d5f5d08e56906adc1a20ce35c
• 💔 Build #56245 failed 7dbedac78c0a515e859e91ebaadd407b335ed7a3
• 💛 Build #54529 was flaky 5671e9ce71e4559530190d864304be6a5810cf77

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @xcrzx