Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Define how to fix resumption of facility syncs to allow cancellation of sync tasks #7372

Closed
bjester opened this issue Jul 23, 2020 · 1 comment
Closed

Define how to fix resumption of facility syncs to allow cancellation of sync tasks #7372

bjester opened this issue Jul 23, 2020 · 1 comment
Assignees
@jamalex @bjester
Labels
APP: Device Re: Device App (content import/export, facility-syncing, user permissions, etc.) DEV: backend Python, databases, networking, filesystem... P0 - critical Priority: Release blocker or regression TAG: performance User-facing performance TAG: tech update / debt Change not visible to user TODO: needs decisions Design or specifications are necessary
Milestone
0.15.0

Comments

@bjester
Copy link
Member

bjester commented Jul 23, 2020

Observed behavior

Morango currently doesn't have the capability to retry sync sessions. As a result, a table which holds transferred data can grow out of hand, creating issues like #7270. We've disabled task cancellation for sync tasks for Kolibri 0.14 release to avoid exacerbating this issue.

Expected behavior

A device admin should be able resume incomplete sync sessions, whether interrupted by a network issue or user's cancellation request.

User-facing consequences

Bloated databases, depending on how many incomplete syncs and the amount of data being transferred.

References

#7270
#7313
learningequality/morango#86
learningequality/morango#59

Steps to reproduce

  1. Create a new Kolibri instance
  2. Attempt to sync a large facility from a Kolibri device, from the command line:
    kolibri manage sync --facility <...> --no-push --chunk-size 75 --user <...> --password <...>
  3. Press Ctrl+C to kill it while it's pulling
  4. Re-attempt the sync
  5. Observe the pulling transfer starts from the beginning
  6. The morango_buffers table will also not be empty (it usually is after successful syncs)

Context

Kolibri 0.14.0
@bjester bjester added TAG: tech update / debt Change not visible to user P0 - critical Priority: Release blocker or regression TAG: performance User-facing performance DEV: backend Python, databases, networking, filesystem... labels Jul 23, 2020
@bjester bjester added this to the upcoming patch milestone Jul 23, 2020
@bjester bjester self-assigned this Jul 23, 2020
@bjester bjester mentioned this issue Jul 23, 2020
Merged
9 tasks
@jonboiser jonboiser modified the milestones: upcoming patch, 0.15.0 Oct 8, 2020
@jonboiser jonboiser added the APP: Device Re: Device App (content import/export, facility-syncing, user permissions, etc.) label Feb 20, 2021
@rtibbles rtibbles added the TODO: needs decisions Design or specifications are necessary label Mar 30, 2021
@rtibbles rtibbles changed the title Fix resumption of facility syncs to allow cancellation of sync tasks Define how to fix resumption of facility syncs to allow cancellation of sync tasks Mar 30, 2021
@bjester
Copy link
Member Author

bjester commented Apr 20, 2021

Added follow up features:
learningequality/morango#103
#8019
learningequality/morango#104
learningequality/morango#105

@bjester bjester closed this as completed Apr 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jamalex jamalex

@bjester bjester

Labels
APP: Device Re: Device App (content import/export, facility-syncing, user permissions, etc.) DEV: backend Python, databases, networking, filesystem... P0 - critical Priority: Release blocker or regression TAG: performance User-facing performance TAG: tech update / debt Change not visible to user TODO: needs decisions Design or specifications are necessary
Projects
None yet
Milestone
0.15.0
Development

No branches or pull requests
4 participants
@jamalex @bjester @rtibbles @jonboiser