Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it possible to change the default cron timetable #34851

Merged
merged 11 commits into from
Jan 4, 2024
19 changes: 19 additions & 0 deletions airflow/config_templates/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2375,6 +2375,25 @@ scheduler:
type: string
example: ~
default: "^[A-Za-z0-9_.~:+-]+$"
create_cron_data_intervals:
description: |
Whether to create DAG runs that span an interval or one single point in time for cron schedules, when
a cron string is provided to `schedule` argument of a DAG. If True,
CronDataIntervalTimetable is used, which is the legacy Airflow behavior suitable
for DAGs with well-defined data_interval you get contiguous intervals from the end of the previous
interval up to the scheduled datetime. If False, CronTriggerTimetable is used,
which is closer to the behavior of cron itself.

Notably, for CronTriggerTimetable, the logical_date is the same as the time the DAG Run will try to
schedule, while for CronDataIntervalTimetable, the logical_date is the beginning of the data interval,
but the DAG Run will try to schedule at the end of the data interval. For more differences
between the two Timetables, see
https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/timetable.html#differences-between-the-two-cron-timetables
collinmcnulty marked this conversation as resolved.
Show resolved Hide resolved
version_added: 2.9.0
type: boolean
example: ~
default: "True"
see_also: ":ref:`Differences between the two cron timetables`"
triggerer:
description: ~
options:
Expand Down
8 changes: 6 additions & 2 deletions airflow/models/dag.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@
NullTimetable,
OnceTimetable,
)
from airflow.timetables.trigger import CronTriggerTimetable
from airflow.utils import timezone
from airflow.utils.dag_cycle_tester import check_cycle
from airflow.utils.dates import cron_presets, date_range as utils_date_range
Expand Down Expand Up @@ -226,8 +227,11 @@ def create_timetable(interval: ScheduleIntervalArg, timezone: Timezone) -> Timet
if isinstance(interval, (timedelta, relativedelta)):
return DeltaDataIntervalTimetable(interval)
if isinstance(interval, str):
return CronDataIntervalTimetable(interval, timezone)
raise ValueError(f"{interval!r} is not a valid interval.")
if airflow_conf.getboolean("scheduler", "create_cron_data_intervals"):
return CronDataIntervalTimetable(interval, timezone)
else:
return CronTriggerTimetable(interval, timezone=timezone)
raise ValueError(f"{interval!r} is not a valid schedule_interval.")


def get_last_dagrun(dag_id, session, include_externally_triggered=False):
Expand Down