Skip to content

Commit

Permalink
added explaining concept of logical date in DAG run docs (#21433)
Browse files Browse the repository at this point in the history
GitOrigin-RevId: 752d53860e636ead2be7c3f2044b9b312ba86b95
  • Loading branch information
howardyoo authored and Cloud Composer Team committed Jul 10, 2022
1 parent 3d9b739 commit 91441d7
Show file tree
Hide file tree
Showing 3 changed files with 25 additions and 0 deletions.
16 changes: 16 additions & 0 deletions docs/apache-airflow/concepts/dags.rst
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,8 @@ The ``schedule_interval`` argument takes any value that is a valid `Crontab <htt
For more information on ``schedule_interval`` values, see :doc:`DAG Run </dag-run>`.

If ``schedule_interval`` is not enough to express the DAG's schedule, see :doc:`Timetables </howto/timetable>`.
For more information on ``logical date``, see :ref:`data-interval` and
:ref:`faq:what-does-execution-date-mean`.

Every time you run a DAG, you are creating a new instance of that DAG which
Airflow calls a :doc:`DAG Run </dag-run>`. DAG Runs can run in parallel for the
Expand All @@ -177,6 +179,20 @@ In much the same way a DAG instantiates into a DAG Run every time it's run,
Tasks specified inside a DAG are also instantiated into
:ref:`Task Instances <concepts:task-instances>` along with it.

A DAG run will have a start date when it starts, and end date when it ends.
This period describes the time when the DAG actually 'ran.' Aside from the DAG
run's start and end date, there is another date called *logical date*
(formally known as execution date), which describes the intended time a
DAG run is scheduled or triggered. The reason why this is called
*logical* is because of the abstract nature of it having multiple meanings,
depending on the context of the DAG run itself.

For example, if a DAG run is manually triggered by the user, its logical date would be the
date and time of which the DAG run was triggered, and the value should be equal
to DAG run's start date. However, when the DAG is being automatically scheduled, with certain
schedule interval put in place, the logical date is going to indicate the time
at which it marks the start of the data interval, where the DAG run's start
date would then be the logical date + scheduled interval.

DAG Assignment
--------------
Expand Down
2 changes: 2 additions & 0 deletions docs/apache-airflow/dag-run.rst
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,8 @@ scheduled one interval after ``start_date``.

If ``schedule_interval`` is not enough to express your DAG's schedule,
logical date, or data interval, see :doc:`/concepts/timetable`.
For more information on ``logical date``, see :ref:`concepts:dag-run` and
:ref:`faq:what-does-execution-date-mean`

Re-run DAG
''''''''''
Expand Down
7 changes: 7 additions & 0 deletions docs/apache-airflow/faq.rst
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,8 @@ This allows for a backfill on tasks that have ``depends_on_past=True`` to
actually start. If this were not the case, the backfill just would not start.


.. _faq:what-does-execution-date-mean:

What does ``execution_date`` mean?
----------------------------------

Expand Down Expand Up @@ -277,6 +279,11 @@ misunderstandings.
Note that ``ds`` (the YYYY-MM-DD form of ``data_interval_start``) refers to
*date* ***string***, not *date* ***start*** as may be confusing to some.

.. tip::

For more information on ``logical date``, see :ref:`data-interval` and
:ref:`concepts:dag-run`.


How to create DAGs dynamically?
-------------------------------
Expand Down

0 comments on commit 91441d7

Please sign in to comment.