Making scheduled tasks run on the start_date

[kimtore]

It took me a while to figure out why my DAGs weren't run on schedule, but then I found this in the docs:

Note that if you run a DAG on a schedule_interval of one day, the run stamped 2016-01-01 will be trigger soon after 2016-01-01T23:59. In other words, the job instance is started once the period it covers has ended.

I find this counter-intuitive, and in my opinion it would be better to have the task run exactly on the start_date, and not delayed by the schedule_interval time delta.

Is it possible to configure Airflow so that this functionality is achieved?

[cgelman]

+1 this. If you look in models.py: is_queueable():
https://github.com/airbnb/airflow/blob/master/airflow/models.py#L642
There's this line:

  if self.execution_date > datetime.now() - self.task.schedule_interval:
            return False

This makes sense for hourly or daily jobs - you need to wait for the day to finish to start this. But if want to have a weekly job that runs at the end of the week - this will wait until 7 days after the end of the week.

I guess you can get around this by making your schedule based on a start date of the first day of the week...

[mistercrunch]

Yes, that's the convention. Maybe it needs to be more prominent in the docs.