Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DataViews] Implement DataViewLazy class based on AbstractDataView #167750

Closed
Tracked by #167595
kertal opened this issue Oct 2, 2023 · 2 comments
Closed
Tracked by #167595

[DataViews] Implement DataViewLazy class based on AbstractDataView #167750

kertal opened this issue Oct 2, 2023 · 2 comments
Assignees
Labels
Feature:Data Views Data Views code and UI - index patterns before 8.0 impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. loe:large Large Level of Effort Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.

Comments

@kertal
Copy link
Member

kertal commented Oct 2, 2023

When DataViewAbstract is implemented (#167736), the decoupling of the fields fetching for DataViews can continued by implementing a new DataViewLazy class. This will provide consumers of this class to get DataViews without automatic field fetching. This is beneficial to improve the performance when data views are consumed.

  • Create DataViewLazy class.
  • Account for class methods that assume field list.
  • Provide method to request only fields needed.
  • Provide methods that convert to and from DataView class.

A potential improvement

In Discover/Lens we currently fetch a data view with all fields, and additionally there's a request for the field list with an indexFilter, aiming the find the fields of indices that match a give query. This fields are then displayed in the "Available fields" list. With the new DataViewLazy class, we can optimize this, by e.g. initially just fetch the fields that match the indexFilter, and load all fields on demand. What's more, currently the fetching of fields is blocking the UI when Discover is loaded, or the data view is being switched to a new one. Since field loading with the new class fields is decoupled, we can provide a more responsive UX.

@kertal kertal added loe:needs-research This issue requires some research before it can be worked on or estimated Feature:Data Views Data Views code and UI - index patterns before 8.0 impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. labels Oct 2, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

@kertal kertal changed the title [DataViews] Implement DataViewAsync / DataViewLazy class [DataViews] Implement DataViewAsync / DataViewLazy class based on AbstractDataView Oct 2, 2023
@kertal kertal changed the title [DataViews] Implement DataViewAsync / DataViewLazy class based on AbstractDataView [DataViews] Implement DataViewLazy class based on AbstractDataView Oct 6, 2023
@kertal kertal added loe:large Large Level of Effort and removed loe:needs-research This issue requires some research before it can be worked on or estimated labels Feb 7, 2024
mattkime added a commit that referenced this issue Apr 3, 2024
## Summary

Part 1 of #178594

Initial implementation of DataViewLazy class - an alternative DataView
implementation that doesn't load the full field list on initial
creation.

```
Implements DataViewLazy (variations from DataView class)
- getFields ({ mapped = true, scripted = true, runtime = true, fieldName, forceRefresh = false, unmapped, indexFilter, metaFields = true }) => 
    Promise<{ getFieldMap: () => Record<string, DataViewField>; getFieldMapSorted: () => Record<string, DataViewField> }>
- getRuntimeFields({ fieldName = ['*'] }) => Record<string, DataViewField>
- getScriptedFields({ fieldName = ['*'] }) => Record<string, DataViewField>
- getFieldByName (string) => Promise<DataViewField>
- toSpec (same params as getFields) => Promise<DataViewSpec>

other notable changes using DataViewLazy over DataView
- won't throw if index pattern fails to match
- by default toSpec doesn't return fields, must pass in parameters for requested fields

---

DataViews service additions
- toDataView: (toDataView: DataViewLazy) => Promise<DataView>;
- toDataViewLazy: (dataView: DataView) => Promise<DataViewLazy>;
- getAllDataViewLazy: () => Promise<DataViewLazy[]>;
- getDataViewLazy: (id: string) => Promise<DataViewLazy>;
- createDataViewLazy: (spec: DataViewSpec) => Promise<DataViewLazy>;
- createAndSaveDataViewLazy: (spec: DataViewSpec, override?: boolean) => Promise<DataViewLazy>;
- getDefaultDataViewLazy: () => Promise<DataViewLazy | null>;
```

Closes #176426 and
#167750

---------

Co-authored-by: kibanamachine <[email protected]>
@kertal
Copy link
Member Author

kertal commented Apr 3, 2024

Closed by #173948

@kertal kertal closed this as completed Apr 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Data Views Data Views code and UI - index patterns before 8.0 impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. loe:large Large Level of Effort Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.
Projects
None yet
Development

No branches or pull requests

3 participants