Avoid spawning the same scheduled task multiple times #252
Conversation
This reverts commit d4f78c5. The same bug exists not only in schedule.once() but also schedule.every() so a better fix is necessary.
The threadpool already has a dedicated thread to run scheduler tasks which is not the same as running in the main thread.
This forces code to run sequentially on the scheduler thread. fixes attzonko#231 (again)
|
Code Climate has analyzed commit e393992 and detected 0 issues on this pull request. View more on Code Climate. |
|
After looking at the schedule repo, there are a few issues/PRs proposing support for async/threaded execution but this doesn't seem to be a feature maintainers are interested in. See:
I propose merging the current PR for sequential execution until a better solution is found. |
There was a problem hiding this comment.
Let's get rid of this schedule library altogether. I don't like how it's structured under the hood anyway. Not necessarily related to this repo, but I think it would be nice to create a subclass of concurrent.futures.Executor that supports scheduling tasks. It could use the python sched library under the hood. Might be something for https://github.com/jneeven/practipy, but I have never needed to schedule tasks from Python so I probably won't be working on it the coming year 😅
|
I'm using scheduled tasks extensively, and will use more in the near future, so if we want to replace it we need a compatible candidate. Anything that supports cronjob-like execution and one-time-tasks would fit for me. I haven't yet run into a bottleneck with too many active tasks. |
|
I think |
|
Looking back at the discussion, I agree that option 2 would have been ideal if the other project was willing to change. Since that seems like a dead-end, I am inclined to merge this change in as is, and if future improvements are needed (i.e. change to using native |
TLDR: This is not a complete solution and currently two of the tests are not fixable. Relates to #231 .
Currently scheduled tasks are repeatedly executed causing all kinds of unexpected behavior.
Above we can see two iterations of
schedule.every(60).seconds.do(...)for a task that takes roughly 2.2 seconds to complete. Given the schedule loop is pre-defined to 1 second, the task is launched 3 times, the first at second 0, and the last at second 2. At second 3 the task is marked complete and is not relaunched.In the above log
admin taskshould twice, once every minute, but instead ends up running 6 times.We have two possible solutions here:
The current PR implements 1. but renders the
schedule.do().tag("process")approach useless since the schedule thread (Thread-1) still blocks. The main thread (MainThread) runs normally.The two schedule tests that verify if tasks can run in parallel are therefore marked as skipped for the time being.