Skipping major versions during upgrade

[MorrisJobke]

I had some rough ideas about what we need to do until this is ready to go. Here is a first braindump. Feel free to comment on this. I can update this first entry here to have the current consensus covered.

Stuff we need to take care about

• run repair steps only once
• define the version a repair step has to be run on (to also provide an order and skip old ones)
• provide a way to describe database table transitions (doctrine migrations?) including the version they were added

How to do the update then

Skipping old steps: We don't need to run steps with a version number equal or lower than the (previously) installed version that is listed in the database.

When to run live repair steps: But we need to question if the repair step v15 already has finished, right? Or (what I really prefer) we simply don't rely on the live migration and simply add them as usual as background job and make this very clear, that the live repair step could happen during a future version which then needs to take care to cleanup repair steps that doesn't apply or work anymore (maybe because of a changed database structure - idea removing the repair step code should be fine - then the code shouldn't be executed at all and be removed from the background jobs just fine)

Apps with dependencies to the server: When and in which order should updates of apps happen that has a dependency to the server - filesystem or user backend apps.

Specific scenario

• app1 v4
  • db migration v2
  • live repair step v2
  • db migration v3
  • repair step v3
  • live repair step v3
• ldap_app v20
  • db migration v16
  • live repair step v16
  • db migration v17
  • repair step v17
  • live repair step v18
• server v20
  • db migration v15
  • live repair step v15
  • db migration v17
  • live repair step v17
  • db migration v19
  • repair step v19
  • db migration v20

previous installed versions: app1 v1, ldap_app v16 and server v16

• no need to run the v15 steps of the server and the v16 steps of ldap_app

Run them

• server first and then the apps (currently apps are in no specific order) - here db migrations of a version are sorted before a repair step for that version
  • server db migration v15
  • server live repair step v15
  • server db migration v17
  • server db migration v19
  • server repair step v19
  • server db migration v20
  • ldap_app db migration v16
  • ldap_app live repair step v16
  • ldap_app db migration v17
  • ldap_app repair step v17
  • app1 db migration v2
  • app1 db migration v3
  • app1 repair step v3
• live steps are added at the very end (is currently also the case)
  • server live repair step v17
  • ldap_app live repair step v18
  • app1 live repair step v2
  • app1 live repair step v3

@LukasReschke @rullzer @karlitschek @schiessle @icewind1991 @nickvergessen @blizzz Comments are welcome - this is only my first quick draft and we need to specify a bit more to not run into issues here.

See oc/core#14851 and oc/activity#546

[blizzz]

server first and then the apps (currently apps are in no specific order) - here db migrations of a version are sorted before a repair step for that version

What if in between steps of other apps require different (in between) server state?

[nickvergessen]

Yeah, we need some kind of dependency chain. We had the very same problem in phpBB as well: https://github.com/phpbb/phpbb/blob/master/phpBB/phpbb/db/migration/data/v31x/v312rc1.php#L23-L29
It has one migration for each release. And that one should be listed as a dependency of the app then, so they can only "depend" on releases, not on inbetween steps. However there is no "run before", because that just makes no sense.

Preventing the "future" however is really hard. Just remember one could simply have disabled an app while running NC 12, so when you reenable it in NC 14, the inbetween steps are not available because they happened long ago.

[MorrisJobke]

Preventing the "future" however is really hard. Just remember one could simply have disabled an app while running NC 12, so when you reenable it in NC 14, the inbetween steps are not available because they happened long ago.

For this case we should introduce a "delete app with all settings" way. And then make it mandatory to run it if the versions are too different.

[MorrisJobke]

When to run live repair steps: But we need to question if the repair step v15 already has finished, right? Or (what I really prefer) we simply don't rely on the live migration and simply add them as usual as background job and make this very clear, that the live repair step could happen during a future version which then needs to take care to cleanup repair steps that doesn't apply or work anymore (maybe because of a changed database structure - idea removing the repair step code should be fine - then the code shouldn't be executed at all and be removed from the background jobs just fine)

#2566

[MorrisJobke]

@karlitschek sadly we need to move this to 13

[karlitschek]

Thats ok. Thanks

[jospoortvliet]

An alternative approach would be: nextcloud/updater#321