Fail saved object migrations when encountering embeddable state from an unknown embeddable factory

[rudolf]

When a saved object contains persistable state from an unknown embeddable, the persisted state won't be migrated when Kibana gets upgraded.

This is because embeddables.getEmbeddableFactory returns an id: 'unknown' factor with migrations: {} when an embeddable isn't known https://github.com/elastic/kibana/blob/main/src/plugins/embeddable/server/plugin.ts#L139-L147

Even if the embeddable is later registered again, the migration might not run because the saved objects have already been marked as being up to date. I think the embeddable state might be migrated once it's loaded on the browser, but this means that extractReferences was potentially skipped and that because of _id regeneration the references in the embeddable might contain the old _id's.

In #107678 we decided that starting from 8.0 Kibana will fail to upgrade if there are saved objects of an unknown type to prevent silent data loss with _id regeneration.

It feels like the same reasoning could be applied to persistable state too.

If implemented this would enable a use case where a customer uses Kibana Cloud (because it's required for APM/Fleet), but has a on-prem Kibana with a custom plugin that registers an embeddable with extractReferences or migrations.

[Dosant]

@elastic/kibana-app-services

[elasticmachine]

Pinging @elastic/kibana-presentation (Team:Presentation)

[ThomThomson]

@rudolf, do you know of any clients that this applies to? I haven't heard of any cases where clients have built a custom plugin that registers a new type of embeddable which has been around long enough / is stable enough to require migrations.

While it does seem like the right thing to do to throw an error in this case - after all, unknown embeddable types are potentially a risk - but knowing that a client is actually bound to encounter this issue will increase the estimated impact.

[rudolf]

We know of customers running custom plugins in an on-prem Kibana that connects to Cloud. But we don't know that any of these plugins use embeddables.

We're also planning to do #147445 which would mean embeddables are lazily migrated and would likely require us to separate the saved objects migrationVersion and the embeddable's migrationVersion. I think this work might solve this problem.

Overall though, without data that custom plugins registers embeddables I agree this doesn't feel like a high priority.

[ThomThomson]

Closing this for the time being. We will be moving towards a client-side embeddable migration system that will make this obsolete.