[ML] AIOps: Reduces rerenders when streaming analysis results

[walterra]

Summary

Part of #181111.

Improves the @kbn/ml-response-stream package to trigger less rerendering.

• Updates the types in streamFactory(). Instead of an function overload to handle the different stream types (plain string or NDJSON based), a conditional type is now used to handle both types within the same function.
• The way the state gets updated with throttling in useFetchStream() was improved. Previously useReducer could cause a lot of rerenders, even with the useThrottle we had in place. We now use a ref instead that holds intermediate state without causing rerenders and update the "real" state periodically while the the stream is running. For the Kibana Sample Logs dataset this bring down rerenders from ~16 to ~7. The pgbench dataset comes down from ~20 to ~11.
• Part of the previous approach to reduce rerenders was to use both useThrottle and a special way to accept arrays of actions within reducers. This is a bit non-standard compared to redux reducers. Allowing arrays of actions is now no longer necessary with the improved rerendering so its support was removed.
• For the Security AI Assistant we now export StreamResponseWithHeaders to simplify the types a bit.

Checklist

• Unit or functional tests were updated or added to match the most common scenarios
• Flaky Test Runner was used on any tests changed
• This was checked for breaking API changes and was labeled appropriately

[elasticmachine]

Pinging @elastic/ml-ui (:ml)

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: d0f2a69

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
aiops	492	489	-3

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
aiops	422.6KB	421.9KB	-750.0B
dataVisualizer	671.8KB	671.8KB	-12.0B
total			-762.0B

History

• 💛 Build #208352 was flaky 958cf18

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @walterra

[qn895]

Code LGTM 🎉

[qn895]

Should the consumer of this useFetchStream hook also have the ability to pass in the duration they want to throttle? Not necessary for this PR but would be nice to have in a follow up.

[stephmilovic]

Thanks for the improvement!

[alvarezmelissa87]

LGTM ⚡