Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support the flexibility to configure tasks emission frequency within a time range or endlessly #577

Open
senseibara opened this issue Aug 26, 2020 · 2 comments
Assignees
Labels
retro-action-item Type: Improvement Make something better Type: Performance Make something faster

Comments

@senseibara
Copy link

Is your feature request related to a problem? Please describe.
The title of this request is broad on purpose, but the problem that lead to this is more specific. It is the challenges that we are having right now to find a clean way to configure daily tasks (tasks that needs to be triggered everyday until the end of time).
We tried different things such as generating tasks events in a limited time frame , the problem of this solution is that, if we generate few events, for example 30 events. We would need to update the configuration monthly. But if we generate too many events, it will cause performances issues due to tasks docs.
The current workaround we are using is generating 8 new tasks each week after cache expiry.

Describe the solution you'd like
In a nutshell, we would want the task feature to support generating endless events with an efficient way to handle it in the background which avoid lots of tasks documents generated in one time. If we see it through the lens of daily tasks, it would be great for example if we could configure emission frequency ( one per day, one per week, one per month etc... ) could be similar to the way we are planning purge with a cron setting.The tasks should be then generated endlessly following the given frequency but would be also nice to have the possibility to mention an optional date when the emission should stop, if the date is not mentioned the tasks should be triggered endlessly.

Additional context

A lot of additional context can be found here : https://docs.google.com/document/d/1-G3vPsX-2LhgBkVr6ZsjwXoupvYsMsYtxPMEWURSHwo/edit
. You'll find in this document how we experienced a live incident following a bad daily tasks configuration (10K tasks upwards). With Covid-19 few projects expressed the need to have daily tasks for symptoms screening for example, in that we could see how much value a feature like this would be helpful for these kind of use cases.

@kennsippell
Copy link
Member

kennsippell commented Aug 28, 2020

One somewhat straight-forward solution might be to handle this in the declarative configuration system by putting the solution based around task expiry (linked above) behind an intuitive declarative interface like events: { recurring: true, startingDate: '01/01/2020', endDate: '01/01/2021', interval: '2 days' } (likely not the right interface, but communicates the mechanic).

If this solution seems reasonable, this issue might move to the medic-conf repo.

@MaxDiz MaxDiz added Type: Improvement Make something better Type: Performance Make something faster labels Sep 10, 2020
@kennsippell
Copy link
Member

kennsippell commented Mar 1, 2022

This issue was listed in a retrospective as a priority action-item after two different live-site incidents. For both incidents, recurring tasks scheduled to trigger once per day (recurring forever) were written in a manner resulting in significant degradation of user performance and server performance. In both cases, tens of thousands of unnecessary task documents were created - flooding the server and user's devices. aka task document explosion

Having a user interface to make recurring tasks easier would be an improvement which can help avoid future incidents.

@kennsippell kennsippell self-assigned this Sep 26, 2023
@kennsippell kennsippell transferred this issue from medic/cht-core Oct 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
retro-action-item Type: Improvement Make something better Type: Performance Make something faster
Projects
None yet
Development

No branches or pull requests

3 participants