Hang caused by basic Flask application layout when using a Jobstore

[RobertDeRose]

If you follow a basic Flask tutorial and don't use a Application Factory pattern and use the basic global app development, following the docs on the README and root of the documentation can lead to a Hang that is hard to figure out.

Example

__init__.py

from flask import Flask
from flask_apscheduler import APScheduler
from apscheduler.jobstores.sqlalchemy import SQLAlchemyJobStore

app = Flask(__name__)
app.config.update(
    SCHEDULER_JOBSTORES={
        "default": SQLAlchemyJobStore(
            url="sqlite:///{}/jobs.db".format(app.instance_path)
        )
    }
)

scheduler = APScheduler()
scheduler.init_app(app)
scheduler.start()

@scheduler.task("interval", id="sample", seconds=10)
def sample():
  print("I'll work on the first load, but probably not the second")

The docs should highlight this issue as it has been seen by a decent number of people as seen here agronholm/apscheduler#250

Solution

Simply move scheduler.start() to the bottom of __init__.py or follow the Application Factory pattern

[viniciuschiele]

Hello,

I suspect this issue has been caused by #140 which was released a couple of days ago, I've reverted the PR just in case.
Can you try again using the latest version (1.12.1)?

Thanks

[RobertDeRose]

@viniciuschiele Hey, thanks for the reply, but I don't think they are related.

[ginsburgnm]

@viniciuschiele interestingly version 1.12.1 causes my flask app to hang, whereas reverting to 1.12.0 works

[RobertDeRose]

@viniciuschiele Yes I wish you didn't revert that PR, it was a good PR, the reason I opened this issue was to document that the issues is with neither your project nor the APScheduler itself, but how most people develop their Flask apps based on the simple Flask tutorials without using an Application Factory.

I myself am simply mocking up a demo so I didn't bother and discovered that if I call start with a JobStore, it would attempt to use the __import__ on the name of the module from the JobStore before the module is fully loaded. This could be solved a few ways:

• Document that schedule.start() should be call after all your job functions have been defined
• Add a delay in the start function so that it can ensure that the entire module is loaded first, though timing is always a tricky thing to guess at.

Thanks!

[christopherpickering]

@RobertDeRose hey, have you successfully gotten an application factory working you can link to? It would be great to add to the docs here!

Is there an advantage to the factory vs the basic way when only one wsgi work can run anyways?

I fought it for a while and ended up using the basic flask way, but would like to change to the factory at some point.

[viniciuschiele]

@RobertDeRose I will do another round of tests with #140 to make sure it doesn't have any impact.

Regarding your issue, I haven't been able to reproduce it, I tried running it with uwsgi and app.run and both worked.

My recommendation is always to have the job definitions in a separate file, that prevents this and other issues.

[viniciuschiele]

Ok, now I was able to replicate the issue.

It isn't a good practice to start the scheduler and have the job definitions in the same file, I think we could change the documentation to explain a better way to organize the structure of the files.

Something along those lines:

init.py

app = App()
scheduler = APScheduler()

tasks.py

from . import scheduler

@scheduler.task()
def job1():
   ...

main.py

from . import app, scheduler, tasks

scheduler.init_app(app)
scheduler.start()
app.run()

[RobertDeRose]

@christopherpickering I solved my issue by just ensuring that I did not call scheduler.start until after my tasks were defined, basic order of operations solution.

@viniciuschiele Yes, I think documenting that pattern and adding a warning in the docs would be sufficient to prevent this issue hopefully for others.

[christopherpickering]

@RobertDeRose yeah, I just rewrote my app as an application factory, and it works when debug=false, I think same as #148.

[Gkirito]

My Flask apps using an Application Factory, but I use function like scheduler.add_job to add my job dynamically. If APScheduler didn't start in werkzeug reloader process, I can't let my job run by using scheduler.add_job , when the debug mode is on. So I think #140 have some impact, And my suggestion is APScheduler should start in werkzeug reloader process when the debug mode is on. My description maybe not clearly, therefore I offer a demo that let us test my Job environment

flaskProject.zip

I use poetry to manage my package

poetry install

I use [email protected] which means remove #140

This is my create_app()

def is_debug_mode():
    val = os.environ.get("FLASK_DEBUG")
    if not val:
        return os.environ.get("FLASK_ENV") == "development"
    return val.lower() not in ("0", "false", "no")


def is_werkzeug_reloader_process():
    return os.environ.get("WERKZEUG_RUN_MAIN") == "true"


def create_app():
    app = Flask(__name__)
    app.config.from_object(DevelopmentConfig)
    apis.init_app(app)
    # models.init_app(app)

    # ⬇chose one mode

    # if you use #140
    if is_werkzeug_reloader_process():
        pass
    else:
        tasks.init_app(app)

    # This is my solution
    # if is_debug_mode() and not is_werkzeug_reloader_process():
    #     pass
    # else:
    #     tasks.init_app(app)

    return app

task/__init__.py

from app.extensions import scheduler

def init_app(app):
    scheduler.init_app(app)
    scheduler.start()

Make sure debug mode is on. Then let APScheduler do not start in werkzeug reloader process.
If you add a job, it will not run because the APScheduler didn't run in werkzeug reloader process

Then choose my solution

if is_debug_mode() and not is_werkzeug_reloader_process():
        pass
    else:
        tasks.init_app(app)

Do again

The job run normally

This PR has same way #151

[christopherpickering]

nice! I copied the way to detect debug, and put a pull request w/ it in new docs : ) Its a first try, probably needs tweaked more. #154

I use poetry as well, but left it out of these docs.. its a doc in itself 😁

[viniciuschiele]

@Gkirito Thanks for providing a full example, that helps a lot. I will push a PR to fix that.