Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Embeddable] Clientside Migration System #158677

Closed
ThomThomson opened this issue May 30, 2023 · 2 comments · Fixed by #162986
Closed

[Embeddable] Clientside Migration System #158677

ThomThomson opened this issue May 30, 2023 · 2 comments · Fixed by #162986
Assignees
Labels
Feature:Embeddables Relating to the Embeddable system impact:critical This issue should be addressed immediately due to a critical level of impact on the product. loe:x-large Extra Large Level of Effort Project:Serverless Work as part of the Serverless project for its initial release Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas

Comments

@ThomThomson
Copy link
Contributor

ThomThomson commented May 30, 2023

Current State

Currently the Embeddable migration system consists of the following pieces:

  • Migrations are registered as part of the server-side Embeddable Factory(example)
  • The Persistable State Service's migrate function which calls the migration functions registered in the embeddable factory when called with a type argument and the input.
  • A persister, which actually persists the embeddable state, and calls the embeddable migrate function. This usually happens during the saved object migrate time, but could technically happen anywhere. Here is an example from Dashboard.

This system works okay as long as:

  • We can update values of fields arbitrarily during the saved object migration step.
  • We can keep the saved object migrations and the embeddable migrations in sync.
  • Every persister of embeddable state remembers to call the migrate function properly.

ZDT Complications

The existing embeddable migration system creates a new saved object migration for every persister. In that persister, it mutates the value of an existing field. In Dashboard for example, every migration added to the serverside embeddable factory for any embeddable type results in a Dashboard saved object migration where the dashboard's PanelsJSON is mutated.

In our new zero downtime environment this will not be allowed.

Next state

The next Embeddable Migration system should be clientside. In practice this looks like:

  • Every embeddable type comes with a unique version. (not related to the Kibana version).
  • Migrations are registered in the public embeddable factory.
  • A similar migrate-to-latest function exists in the public directory of the embeddable plugin.
  • The Embeddable frame runs the migrations on mount before creating the Embeddable Instance.
  • The embeddable instance is always given the latest state.

Considerations / Benefits

Benefit: More flexibility

With a clientside migration system, embeddable inputs will be storable anywhere and will work out of the box without any extra code considerations. Consider the following examples:

  • Cases want to store Lens Charts in their comment saved objects. With a serverside migration system the comments plugin would be responsible for calling a migrate function during its saved object migrations. With a clientside migration system, the Lens chart in that comment would always be a particular version, and when it is loaded into a Lens Embeddable, the input will be updated to the latest version.
  • A user bookmarks a snapshot URL for a Dashboard with 6 by value Lens Panels from version 11 of Kibana. Kibana updates to version 12 and the user attempts to load that URL. With a clientside migration system the version 11 lens panels will update properly. With a serverside only migration system the Lens embeddable will not be able to interpret the older input.
  • Security wants to store per-user customized dashboards in the user's browser storage (or in the user saved object). With clientside migrations they can store this state anywhere with the confidence that it will get migrated at runtime.

Consideration: Up migrations only

  • During a zero downtime migration when the Kibana server is a higher version than the user's browser we will be unable to migrate down. Imagine that:
    • User A's browser is running Kibana version 11 on the Dashboard Listing Page.
    • The Kibana server updates to Kibana version 12
    • User B, running version 12, saves a Dashboard with many version 12 Lens panels
    • User A does not refresh, and opens that Dashboard. The version 12 saved object is passed to a version 11 client.
    • The version 11 client does not contain any code to migrate 12 -> 11
    • In this case we should prompt the user to refresh the page. At that time they will download the newer client code from the server, and their browser will be able to interpret the version 12 lens panels.

Consideration: Jest tests

Ideally, the solution to this issue will have some way to ensure that all newly added saved object migrations for embeddables that can be either by value or by reference are also added to the clientside embeddable migrations registry. We will write jest tests to cover this case.

@ThomThomson ThomThomson added Feature:Embedding Embedding content via iFrame Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas loe:x-large Extra Large Level of Effort impact:critical This issue should be addressed immediately due to a critical level of impact on the product. Project:Serverless Work as part of the Serverless project for its initial release labels May 30, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-presentation (Team:Presentation)

@ThomThomson ThomThomson self-assigned this May 30, 2023
@ThomThomson ThomThomson added Feature:Embeddables Relating to the Embeddable system and removed Feature:Embedding Embedding content via iFrame labels Jun 14, 2023
@cnasikas
Copy link
Member

cnasikas commented Jun 22, 2023

Thank you @ThomThomson for this effort. It is very important for Cases. As you said we store Lens visualizations in our cases-comments and cases-user-actions saved objects and we need to run the migrations on the server side. In our saved objects we persist the attributes (by value not by reference) and the timeRange and we use the EmbeddableComponent to render the visualization. As we are not allowed to add any more migrations (we block the lens team with this restriction), having a clientside migration system is a great solution for us.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Embeddables Relating to the Embeddable system impact:critical This issue should be addressed immediately due to a critical level of impact on the product. loe:x-large Extra Large Level of Effort Project:Serverless Work as part of the Serverless project for its initial release Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants