[DataUsage][Serverless] Data usage charts enhancements

[ashokaditya]

Summary

follow up of:

• [DataUsage][Serverless] Data usage metrics page enhancements #195556

Adds a lot of enhancements to the datastream dropdown including:

• shows storage sizes on the data stream dropdown
• preselects all data streams on the first page load
• updates selected data streams to URL params
• selects data streams based on URL load
• doesn't allow deselecting all data streams
• cancels older API requests

screen

clip

related PRs

• [DataUsage] Add internal API/UX hooks to interact with serverless project metrics API #193966

Checklist

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Unit or functional tests were updated or added to match the most common scenarios
• Flaky Test Runner was used on any tests changed
• Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)
• This renders correctly on smaller devices using a responsive layout. (You can test this in your browser)
• This was checked for cross-browser compatibility

[neptunian]

Whenever i select or unselect a data stream I get two requests to /metrics with the same payload. I'm also still getting the initial 400 error when i load the page due to sending the empty data streams array.

[ashokaditya]

Whenever i select or unselect a data stream I get two requests to /metrics with the same payload. I'm also still getting the initial 400 error when i load the page due to sending the empty data streams array.

yeah, I fixed that as well.

[neptunian]

Looks great! I do still get the issue where if I select a data stream (not deselect), i get two of the same metrics requests/responses. It doesn't do it if I'm only selecting a total of 1 data streams, though (from zero).

[ashokaditya]

Looks great! I do still get the issue where if I select a data stream (not deselect), i get two of the same metrics requests/responses. It doesn't do it if I'm only selecting a total of 1 data streams, though (from zero).

Oh right. I see that too. It's cause the list is being sorted and that is a new query key so it makes a new request. fixed in 6cd61ba

[neptunian]

We can do this in a followup - but I'm thinking there is no reason to make the /metrics request when removing a datastream from the filter dropdown. We aren't getting any new data unless we are adding a datastream or the time range has changed. Given we default by selecting all, if a user wants to remove several data streams, making requests for no reason and causing the charts to have to wait and re render would not be a great experience.

[neptunian]

It was easy for me to create many requests (and likely race conditions) by selecting or deselecting data streams from the dropdown. Can we make sure to abort any inflight requests when another request happens to avoid race conditions?

[ashokaditya]

It was easy for me to create many requests (and likely race conditions) by selecting or deselecting or some combination of data streams from the dropdown. Can we make sure to abort any inflight requests when another request happens to avoid race conditions?

Not sure I follow. Are you saying that you were able to create parallel requests? Do you have a screen clip to illustrate what you mean by race conditions you're seeing on your end. We create a metrics API request every time we change the datastreams list or date ranges. That said, we can ensure that we don't make a metrics API request if we're also fetching a list of data streams.

[neptunian]

Are you saying that you were able to create parallel requests?

yes

Use a settimeout or throttle your connection and click on and off the the data streams:

If you have a look at other fetcher hooks they usually use the AbortController to avoid this.

[ashokaditya]

Are you saying that you were able to create parallel requests?

yes

Use a settimeout or throttle your connection and click on and off the the data streams:

Ah I see what you mean. Fixed in 47d22ee

[ashokaditya]

We can do this in a followup - but I'm thinking there is no reason to make the /metrics request when removing a datastream from the filter dropdown.

I think we should allow users to see less data for fewer indices/datastreams should they like. They might want to see fewer indices on the charts instead of being overwhelmed by looking at data for several at a time.

We aren't getting any new data unless we are adding a datastream or the time range has changed.

I would expect to get different or less data based on changes to time-range and which data streams are selected. Is that not the case with the auto-ops API?

Given we default by selecting all, if a user wants to remove several data streams, making requests for no reason and causing the charts to have to wait and re-render would not be a great experience.

Yeah, we need to make the chart UX better. I would expect to see chart grids while data is loading for the first time. On subsequent requests, I would expect the chart to be updated if there's a delta. I imagine the bars changing in height/frequency on subsequent requests. What do you think? @neptunian @YulNaumenko

[neptunian]

I think we should allow users to see less data for fewer indices/datastreams should they like. They might want to see fewer indices on the charts instead of being overwhelmed by looking at data for several at a time.

I agree. My question is why do we need to make a request to /metrics when they remove data streams? We are not getting any new data in that case? We need to update the UI state but we don't need to make a request. If we didn't have to make a request as they removed data streams the chart would update immediately instead of having to wait for the request. This is a performance optimization I'm getting at and it adds a little bit of complexity. I mainly mention it because this request can take some time due to the external call - so it may be worth it.

[neptunian]

wow, that was easy 😄

[ashokaditya]

Yeah, one of many reasons a lot of us moved away from redux to manage server state and use tanstack-query.

[ashokaditya]

I think we should allow users to see less data for fewer indices/datastreams should they like. They might want to see fewer indices on the charts instead of being overwhelmed by looking at data for several at a time.

I agree. My question is why do we need to make a request to /metrics when they remove data streams? We are not getting any new data in that case? We need to update the UI state but we don't need to make a request. If we didn't have to make a request as they removed data streams the chart would update immediately instead of having to wait for the request. This is a performance optimization I'm getting at and it adds a little bit of complexity. I mainly mention it because this request can take some time due to the external call - so it may be worth it.

Ah, I see what you mean by improving performance. That makes sense if we can correctly assume that "there's no new data" meaning non of the metrics data changes after the intiial request.

I would expect that ingestion rate and storage rate is in fact constantly changing while (any arbitrary of time) users are looking at the charts.

[neptunian]

I would expect that ingestion rate and storage rate is in fact constantly changing while (any arbitrary of time) users are looking at the charts.

That's true. I was thinking that so long as the time range didn't change, nothing would change, but since we default to a relative time range, it's probably an unlikely scenario and not worth pursuing.

[ashokaditya]

I would expect that ingestion rate and storage rate is in fact constantly changing while (any arbitrary of time) users are looking at the charts.

That's true. I was thinking that so long as the time range didn't change, nothing would change, but since we default to a relative time range, it's probably an unlikely scenario and not worth pursuing.

Oh right. Thanks for clarifying. Yeah if the end date is not set to now then it makes sense to optimize requests. I can try and test it out when I work on the new PR.

[kibanamachine]

Starting backport for target branches: 8.16

https://github.com/elastic/kibana/actions/runs/11404361214

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: dda9e8d

Failed CI Steps

• Jest Tests #19

Test Failures

• [job] [logs] Jest Tests #19 / useGetCaseUserActionsStats returns proper state on getCaseUserActionsStats

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
dataUsage	237.8KB	237.6KB	-230.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
dataUsage	4.6KB	4.5KB	-54.0B

History

• 💔 Build #243757 failed 7e4db9d
• 💔 Build #243756 failed c99ad95
• 💚 Build #243688 succeeded e5d4fe5
• 💚 Build #243657 succeeded 47d22ee
• 💚 Build #243601 succeeded 6cd61ba

cc @ashokaditya

[kibanamachine]

💚 All backports created successfully

Status	Branch	Result
✅	8.16

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

[jbudz]

@ashokaditya there's a version gap with this backport. Can you check if this should also be backported to 8.x/8.17?

[ashokaditya]

@ashokaditya there's a version gap with this backport. Can you check if this should also be backported to 8.x/8.17?

yeah it should be backported to 8.x/8.17