Skip to content

Commit

Permalink
Make it possible to change the default cron timetable (#34851)
Browse files Browse the repository at this point in the history
---------

Co-authored-by: Tzu-ping Chung <[email protected]>
  • Loading branch information
collinmcnulty and uranusjr authored Jan 4, 2024
1 parent a95d0e6 commit ac4073b
Show file tree
Hide file tree
Showing 2 changed files with 25 additions and 2 deletions.
19 changes: 19 additions & 0 deletions airflow/config_templates/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2375,6 +2375,25 @@ scheduler:
type: string
example: ~
default: "^[A-Za-z0-9_.~:+-]+$"
create_cron_data_intervals:
description: |
Whether to create DAG runs that span an interval or one single point in time for cron schedules, when
a cron string is provided to `schedule` argument of a DAG. If True,
CronDataIntervalTimetable is used, which is the legacy Airflow behavior suitable
for DAGs with well-defined data_interval you get contiguous intervals from the end of the previous
interval up to the scheduled datetime. If False, CronTriggerTimetable is used,
which is closer to the behavior of cron itself.
Notably, for CronTriggerTimetable, the logical_date is the same as the time the DAG Run will try to
schedule, while for CronDataIntervalTimetable, the logical_date is the beginning of the data interval,
but the DAG Run will try to schedule at the end of the data interval. For more differences
between the two Timetables, see
https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/timetable.html#differences-between-the-two-cron-timetables
version_added: 2.9.0
type: boolean
example: ~
default: "True"
see_also: ":ref:`Differences between the two cron timetables`"
triggerer:
description: ~
options:
Expand Down
8 changes: 6 additions & 2 deletions airflow/models/dag.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@
NullTimetable,
OnceTimetable,
)
from airflow.timetables.trigger import CronTriggerTimetable
from airflow.utils import timezone
from airflow.utils.dag_cycle_tester import check_cycle
from airflow.utils.dates import cron_presets, date_range as utils_date_range
Expand Down Expand Up @@ -226,8 +227,11 @@ def create_timetable(interval: ScheduleIntervalArg, timezone: Timezone) -> Timet
if isinstance(interval, (timedelta, relativedelta)):
return DeltaDataIntervalTimetable(interval)
if isinstance(interval, str):
return CronDataIntervalTimetable(interval, timezone)
raise ValueError(f"{interval!r} is not a valid interval.")
if airflow_conf.getboolean("scheduler", "create_cron_data_intervals"):
return CronDataIntervalTimetable(interval, timezone)
else:
return CronTriggerTimetable(interval, timezone=timezone)
raise ValueError(f"{interval!r} is not a valid schedule_interval.")


def get_last_dagrun(dag_id, session, include_externally_triggered=False):
Expand Down

0 comments on commit ac4073b

Please sign in to comment.