Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lazy field list loading #134306

Closed
flash1293 opened this issue Jun 14, 2022 · 4 comments
Closed

Lazy field list loading #134306

flash1293 opened this issue Jun 14, 2022 · 4 comments
Labels
discuss Feature:Data Views Data Views code and UI - index patterns before 8.0 impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.

Comments

@flash1293
Copy link
Contributor

Related to #134176

Long fields lists can be a problem because currently we always load them up front in their entirety. A lazy loading approach only requesting relevant parts of information and paginating should help with this and speed up the UI even in suboptimal situations (Mapping explosion, Elasticsearch under heavy load, ...)

In general this would require us to treat _field_caps more like a search and less like a way to fetch configuration state: Response might be large, response might be slow, graceful handling of errors keeping the UI functional in a reduced way, ...

I can imagine the following steps:

  • Change Elasticsearch to allow pagination on field caps or something comparable and filtering on all properties (type, searchable, …)
  • In Kibana, be smart about the fields loaded - do only load those up front that are needed for the vis (but don't cache them, this is causing problems because they could get out of sync)
  • For lists (e.g. Lens left hand panel) only load fields page by page while the user is scrolling
  • When a field name is searched for, load the fields for this prefix
  • When field list filter is applied, do a search for field names

These challenges are similar to those presented by flattened fields - by their very nature there are a large number of them so they have to be loaded on demand.

cc @thomasneirynck

@flash1293 flash1293 added discuss Feature:Data Views Data Views code and UI - index patterns before 8.0 Team:AppServicesSv labels Jun 14, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app-services (Team:AppServicesSv)

@exalate-issue-sync exalate-issue-sync bot added the impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. label Sep 19, 2022
@kertal kertal added bug Fixes for quality problems that affect the customer experience and removed bug Fixes for quality problems that affect the customer experience labels Nov 18, 2022
@petrklapka petrklapka added Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. and removed Team:AppServicesSv labels Nov 28, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

@kertal
Copy link
Member

kertal commented Feb 1, 2023

FYI @mattkime, should we ask for the first step, pagination of field caps in elastic search?

@kertal
Copy link
Member

kertal commented Jun 20, 2023

FYI @mattkime closing in favor of #152159 some good thought here worth to considering

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discuss Feature:Data Views Data Views code and UI - index patterns before 8.0 impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.
Projects
None yet
Development

No branches or pull requests

4 participants