[Security Solution][Detections] Ensure ML Detection Rules/Jobs function with introduction of Space Aware ML Jobs

[spong]

With the introduction of Space Aware ML Jobs (#77916), there will be a few tweaks we'll need to do on the Detections side to ensure everything continues to function as intended. Since new jobs will be scoped to the current space when created, we'll need to update these requests to specify that the new jobs be scoped to the * space to maintain the current behavior of ML Rules/Jobs between spaces.

Additionally, since we're sticking to global ML Jobs for this release, ML may potentially add a warning to the ML Job/Spaces UI to warn users that they may break existing Detection Rules when changing the configured space for ML Jobs with the security group.

[rylnd]

A few updates here:

TL;DR: this all works, but I don't believe there's any UI to distinguish space-aware from -agnostic (aka global, '*' space) jobs, which is potentially confusing to the user.

• The ML popover correctly creates global jobs. That change was introduced in this commit
  • These jobs are visible in any space via the ML API: in the ML popover, Anomalies tabs, etc.
• If desired, users can also create custom jobs that are space-aware.
  • This is the default when creating new jobs via the ML UI, and I did not see any way to override that behavior.
• In a given space, ML Rules can be created for both global jobs and/or space-specific jobs (assuming the job is visible in that space)

Potential UX improvements

• Adding a note to the ML Popover that jobs will be created in the * space
• Distinguishing the space of a given job in the ML Popover
• Distinguishing the space within the ML Jobs table

A global and a space-specific job in the Jobs table (expanded views are similarly identical):

A global and a space-specific job in the ML Popover:

[rylnd]

@jgowdyelastic RE the above: is there a way to determine the space of the job in the ML UI that I've overlooked?

[jgowdyelastic]

@rylnd a job's spaces can be viewed in Stack Management, under the Machine Learning Jobs section.
Spaces can be assigned and removed from jobs.
We do not list a job's spaces in the main Machine Learning plugin.
Space management was added in this PR #83320

[rylnd]

Thanks @jgowdyelastic, I knew I'd seen that data somewhere! @spong here's what that looks like:

[spong]

Closing this issue as this functionality has been shipped and verified.