[Infra UI] Support using Data views in UI

[miltonhultgren]

Similar to the Logs UI, we should support Data views in the Infra UI

• Cross-Kibana linking such as between Discover and Infra
• Linking to Infra UI with a data view (or index pattern string) on the fly (in the URL) to match a data view specified in an alert
  • Allow user to override index pattern for Metric Threshold Rule #149989
  • [OBS-UX-MNGMT] Source links for Custom Threshold rule in Alert Details are broken when DataView doesn't match the ones in the App config e.g. Inventory, Metric  #179053
• Allowing the user to save different data views that they can use later for querying. They might also want to attach a particular data view to a "Saved View".

[elasticmachine]

Pinging @elastic/infra-monitoring-ui (Team:Infra Monitoring UI)

[roshan-elastic]

Hey @miltonhultgren - given the split between observability and search + security, do you think Data Views will still be a thing for us?

[miltonhultgren]

Deferring to @neptunian

[neptunian]

Hey @miltonhultgren - given the split between observability and search + security, do you think Data Views will still be a thing for us?

@roshan-elastic I updated the description with a few use cases for supporting Data Views

[roshan-elastic]

Thanks @neptunian - I'd love to do this but will likely hold off until the direction of Infra UI is becoming clearer (esp w relation to server-less).

I'm not sure whether switching to Data Views something which be required long-term (not saying it isn't - I just don't know given serverless)...if it is then I'd really like to get this done but for now I don't have enough clarity on how serverless will impact the UI work we do

[roshan-elastic]

Hey @neptunian @jasonrhodes - do you think this will be important once we move to be asset-mgt powered?

Checking to see if this investment will be redundant once we move to that world...maybe we don't know yet...

[neptunian]

I don't see how being asset-mgt powered affects the use cases in the description.

[roshan-elastic]

Cool @neptunian - just checking in case being asset-powered would change anything.

I get that this would be nice to do and be a good user experience but I'm trying to get a sense of the value of this to understand how this should be prioritised, e.g. what would the user be able to do tomorrow if we were to release this today?

Is it best for me to speak with @miltonhultgren about this or maybe we should chat about this?

[neptunian]

what would the user be able to do tomorrow if we were to release this today

There are various use cases for what being able to load custom Data Views into the UI. I tried to capture some in the description of this issue.

I don't think it's highest priority or that we should focus on it now.

[roshan-elastic]

Cool - sounds good @neptunian. Put this at the bottom of the backlog until it becomes worthwhile for us to focus on this.

[elasticmachine]

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

[neptunian]

@roshan-elastic @smith I added #179053 to the description. Looks like this is a problem where users are trying view the asset being alerted on in our UIs using the data view they selected in the alert but will get errors due to us not supporting loading our UI with a provided data view.

[jasonrhodes]

For extra clarity, I'll just add that to solve the linked problems coming from alerts, we don't necessarily need the ability to specify a data view in the Infra UI settings screen (the way you can in Logs). (In fact, that likely wouldn't solve this problem, unless it also provided a way to dynamically change the settings.) What we need is a way to specify either a data view or a string "index pattern", "on the fly", so that the linked UI knows where to find the requested data.

For example, if a Custom Threshold Rule is running in an Elastic deployment, and the rule is using "My Custom Data View" (ID 123xyz) and grouping by host name, each host-related alert would contain links to view the host at /app/metrics/host/detail/{hostname}. If a user clicks that link, the Infra UI will look for that hostname in its configured indices only, which may lead to a 404. If we could instead link to something like /app/metrics/host/detail/{hostname}?data-source=dataview:123xyz or something along those lines, that would be enough to solve the issue.

[neptunian]

@jasonrhodes Thanks. That makes sense. Should be a more straightforward change addressing only supporting it on the fly.

[roshan-elastic]

Thanks for this everyone.

Just summarising this in my head, do we know which of the below use cases need data view support to work?

• Cross-Kibana linking such as between Discover and Infra

• [OBS-UX-MNGMT] Source links for Custom Threshold rule in Alert Details are broken when DataView doesn't match the ones in the App config e.g. Inventory, Metric [OBS-UX-MNGMT] Source links for Custom Threshold rule in Alert Details are broken when DataView doesn't match the ones in the App config e.g. Inventory, Metric  #179053

• Linking to Infra UI with a data view on the fly (in the URL) to match a data view specified in an alert
  Allow user to override index pattern for Metric Threshold Rule #149989

• Allowing the user to save different data views that they can use later for querying. They might also want to attach a particular data view to a "Saved View".

I think the first 3 are going to be the most helpful for users right now...

[neptunian]

The first three could be solved with the "on the fly" data view. The last one involves the user being able to save it to their settings (source config).

[jasonrhodes]

I agree with what @neptunian said. And just to be super clear, this kind of "on the fly" data source support would need to handle string index patterns OR data views.

[roshan-elastic]

FYI I've asked a question around the custom threshold rule workflows @neptunian @jasonrhodes

[smith]

This isn't something we're prioritizing at this time.