Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make SavedObjectFinder backward compatible #162904

Merged
merged 21 commits into from
Aug 8, 2023

Conversation

Dosant
Copy link
Contributor

@Dosant Dosant commented Aug 1, 2023

Summary

close #161545
close #153257

This PR makes SavedObjectFinder component backward compatible. It is achieved by going through content- management layer, more technical details here

Testing

SavedObjectFinder is this component that allows to pick a saved object (supports: search index-pattern map visualization lens event-annotation-group:

Screenshot 2023-08-07 at 16 53 32

It is used in the following places:

  • Dashboard
    • Add panel
    • Replace panel
  • Discover - Open Search
  • Visualization - Select search as a source for new viz
  • Graph - select source
  • Cases - markdown editor add lens
  • ML (3 places)
  • Canvas - select embeddable panel
  • Transform
  • Lens > select event annotation

Risks / Follow up

The SavedObjectFinder should stay mostly the same, the only notable functional change is that now SavedObjectFinder doesn't support includeFields which allowed partial saved object returns, this was done to make the call backward-compatible without making the system even more complicated as otherwise we'll need a way to abstract includeFields from so attributes and allow to run migrations on it before making a search. follow up issue to bring it back #163043

The risk with that is that some client that have a lot of large objects might run into performance issues when using SavedObjectFinder. This can be mitigated by changing listing limit in advanced setting from default 1000 to something lower

@Dosant Dosant force-pushed the d/2023-07-28-saved-object-finder-msearch branch from 5b7e630 to 8f08e60 Compare August 2, 2023 11:42
Dosant added 2 commits August 2, 2023 15:51
…ved-object-finder-msearch

# Conflicts:
#	src/plugins/discover/tsconfig.json
Dosant added a commit that referenced this pull request Aug 3, 2023
## Summary

These examples are outdated and don't show recent embeddable best
practices. They also use client-side saved object client and block
making `SavedObjectFinder` backward compatible
#162904 as the `foobar` saved
objects need to be added to content management. We decided that it is
better to clean them up, as fixing them is not a small effort and it is
not worth it on this point as a large embeddable refactor is coming.
@Dosant
Copy link
Contributor Author

Dosant commented Aug 3, 2023

@elasticmachine merge upstream

@Dosant Dosant changed the title D/2023 07 28 saved object finder msearch Make SavedObjectFinder backward compatible Aug 3, 2023
@Dosant Dosant added release_note:skip Skip the PR/issue when compiling release notes Team:SharedUX Team label for AppEx-SharedUX (formerly Global Experience) Feature:Content Management User generated content (saved objects) management labels Aug 3, 2023
@Dosant Dosant marked this pull request as ready for review August 3, 2023 12:15
@Dosant Dosant requested review from a team as code owners August 3, 2023 12:15
@elasticmachine
Copy link
Contributor

Pinging @elastic/appex-sharedux (Team:SharedUX)

@Dosant Dosant requested review from sebelga and jloleysens August 3, 2023 12:15
Copy link
Contributor

@jloleysens jloleysens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a smoke test in Dashboard app and the everything worked as expected. I left a question about using TS interface but otherwise everything looks great!

Nice job @Dosant !


Discussed offline about not supporting the includeFields option at first, happy with the trade off that we may be sending more data over the wire in these searches -- seems to mitigated a bit by CM's processing of allowed fields already?

})) as FindResponseHTTP<FinderAttributes>;
const types = visibleTypes ?? Object.keys(metaDataMap);

const response = await contentClient.mSearch<SavedObjectCommon<FinderAttributes>>({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought the content management would not return the actual SO interface, perhaps we should rather use SOWithMetadata?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I see SavedObjectCommon is SavedObject)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My feeling is indeed that we should have our own interface here with the minimum set of fields that are consumed by apps (for ex is coreMigrationVersion or typeMigrationVersion or namespaces actually needed/consumed in the UI?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jloleysens, I changed from SavedObject to SOWithMetadata, still far from ideal, but better as we more decoupled form saved objects. thanks

@sebelga,

My feeling is indeed that we should have our own interface here with the minimum set of fields that are consumed by apps

Agree, yes, if/when we move forward with cm this could be the direction.
But for now msearch is just a different way to query saved objects with intermediate bwc layer. If to do this typing technically correct right now, then seems we can only type this as generic object for now :(
Or we could do an intersection of all supported types and their interfaces

@botelastic botelastic bot added the Feature:Embedding Embedding content via iFrame label Aug 3, 2023
Copy link
Contributor

@ThomThomson ThomThomson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM! Smoke tested Canvas add from library, Dashboard add from library, and the replace panel action registered in the Embeddable plugin. Also looked through the code and all of the service additions look good to me!

Copy link
Contributor

@walterra walterra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ML/Transform changes LGTM. Smoked tested affected UIs (AD new job, DFA wizard, transform wizard).

Copy link
Contributor

@sebelga sebelga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I tested locally the example app and works great 👍
I left a few comments around being more strict in the typing (fields to be searched on and response). No blocker though

@@ -45,8 +39,8 @@ export interface SavedObjectMetaData<T = unknown> {
getTooltipForSavedObject?(savedObject: SavedObjectCommon<T>): string;
showSavedObject?(savedObject: SavedObjectCommon<T>): boolean;
getSavedObjectSubType?(savedObject: SavedObjectCommon<T>): string;
/** @deprecated doesn't do anything, the full object is returned **/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we remove it then?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try to spend some time and see if it will be easy and worth it adding this back #163043 , didn't want to delete, as there is a chance we can bring this functionality back without changing the component outside outside

})) as FindResponseHTTP<FinderAttributes>;
const types = visibleTypes ?? Object.keys(metaDataMap);

const response = await contentClient.mSearch<SavedObjectCommon<FinderAttributes>>({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My feeling is indeed that we should have our own interface here with the minimum set of fields that are consumed by apps (for ex is coreMigrationVersion or typeMigrationVersion or namespaces actually needed/consumed in the UI?)

@@ -135,6 +135,7 @@ export interface SOContentStorageConstrutorParams<Types extends CMCrudTypes> {
updateArgsToSoUpdateOptions?: UpdateArgsToSoUpdateOptions<Types>;
searchArgsToSOFindOptions?: SearchArgsToSOFindOptions<Types>;
enableMSearch?: boolean;
mSearchAdditionalSearchFields?: string[];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am still a bit puzzeled why we allow any field name to be passed here instead of having a set of fixed fields to be added to the search (e.g. 'name' | 'description' )

Copy link
Contributor Author

@Dosant Dosant Aug 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

title and description are common, but only DataView has name and wants it searchable. This mSearchAdditionalSearchFields is for DataView to configure the name that belongs to them without hardcoding it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So why not set this type as mSearchAdditionalSearchFields?: Array<'name'> ?

@Dosant Dosant requested a review from mattkime August 4, 2023 13:30
Copy link
Contributor

@drewdaemon drewdaemon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested
✅ Event annotation library
✅ Aggs-based search source modal
✅ Graph search source modal

Code LGTM, too!

@Dosant
Copy link
Contributor Author

Dosant commented Aug 7, 2023

@elasticmachine merge upstream

Copy link
Contributor

@mattkime mattkime left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Data views work well with the saved object finder.

@Dosant
Copy link
Contributor Author

Dosant commented Aug 8, 2023

@elasticmachine merge upstream

@kibana-ci
Copy link
Collaborator

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
dashboard 387 388 +1

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/content-management-utils 122 123 +1
embeddable 437 438 +1
eventAnnotation 192 191 -1
savedObjectsFinder 44 25 -19
savedObjectsManagement 140 147 +7
total -11

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
canvas 994.1KB 994.1KB -36.0B
cases 421.2KB 421.1KB -21.0B
dashboard 360.1KB 360.2KB +98.0B
discover 549.3KB 549.3KB -37.0B
embeddable 12.0KB 11.9KB -16.0B
eventAnnotation 176.7KB 176.7KB -48.0B
graph 407.9KB 407.8KB -70.0B
ml 3.4MB 3.4MB -103.0B
savedObjectsFinder 5.3KB 5.0KB -228.0B
transform 404.7KB 404.7KB -7.0B
visualizations 262.9KB 262.8KB -104.0B
total -572.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
discover 31.4KB 31.4KB +41.0B
embeddable 64.6KB 64.6KB +24.0B
eventAnnotation 21.5KB 21.4KB -152.0B
graph 7.9KB 7.9KB +38.0B
ml 74.0KB 74.0KB +38.0B
savedObjectsFinder 3.2KB 3.3KB +134.0B
savedObjectsManagement 19.3KB 19.4KB +128.0B
visualizations 56.9KB 56.8KB -55.0B
total +196.0B
Unknown metric groups

API count

id before after diff
@kbn/content-management-utils 187 188 +1
embeddable 535 536 +1
eventAnnotation 192 191 -1
savedObjectsFinder 44 25 -19
savedObjectsManagement 154 161 +7
total -11

References to deprecated APIs

id before after diff
graph 88 89 +1
visualizations 77 79 +2
total +3

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@Dosant Dosant merged commit 304cb25 into elastic:main Aug 8, 2023
@kibanamachine kibanamachine added v8.10.0 backport:skip This commit does not require backporting labels Aug 8, 2023
bryce-b pushed a commit to bryce-b/kibana that referenced this pull request Aug 9, 2023
## Summary

close elastic#161545
close elastic#153257

This PR makes `SavedObjectFinder` component backward compatible. It is
achieved by going through content- management layer, more technical
details
[here](https://docs.google.com/document/d/1ssYmqSEUPrsuCR4iz8DohkEWekoYrm2yL4QR_fVxXLg/edit)

### Testing

`SavedObjectFinder` is this component that allows to pick a saved object
(supports: `search` `index-pattern` `map` `visualization` `lens`
`event-annotation-group`:

![Screenshot 2023-08-07 at 16 53
32](https://github.com/elastic/kibana/assets/7784120/5c283ea5-3682-4dc8-a8ff-422e6f4f3195)


It is used in the following places: 

- Dashboard 
  - Add panel
  - Replace panel 
- Discover - Open Search 
- Visualization - Select search as a source for new viz
- Graph - select source
- Cases - markdown editor add lens 
- ML (3 places) 
- Canvas - select embeddable panel 
- Transform 
- Lens > select event annotation 




### Risks / Follow up 

The `SavedObjectFinder` should stay mostly the same, the only notable
functional change is that now `SavedObjectFinder` doesn't support
`includeFields` which allowed partial saved object returns, this was
done to make the call backward-compatible without making the system even
more complicated as otherwise we'll need a way to abstract
`includeFields` from so attributes and allow to run migrations on it
before making a search. follow up issue to bring it back
elastic#163043

The risk with that is that some client that have a lot of large objects
might run into performance issues when using `SavedObjectFinder`. This
can be mitigated by changing listing limit in advanced setting from
default 1000 to something lower
@Dosant Dosant self-assigned this Aug 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting Feature:Content Management User generated content (saved objects) management Feature:Embedding Embedding content via iFrame release_note:skip Skip the PR/issue when compiling release notes Team:SharedUX Team label for AppEx-SharedUX (formerly Global Experience) v8.10.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CM] Migrate SavedObectsFinder to CM mSearch [CM] Onboard AX group content