Skip to content

Commit

Permalink
Update python.rst with multi-Scheduler LocalExecutor info (#32310)
Browse files Browse the repository at this point in the history
Co-authored-by: Pankaj Koti <[email protected]>
(cherry picked from commit e53320d)
  • Loading branch information
pragnareddye authored and ephraimbuddy committed Jul 6, 2023
1 parent 3e4c5e8 commit 288fb12
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions docs/apache-airflow/core-concepts/executor/local.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,3 +47,11 @@ Arguably, :class:`~airflow.executors.sequential_executor.SequentialExecutor` cou
parallelism of just 1 worker, i.e. ``self.parallelism = 1``.
This option could lead to the unification of the executor implementations, running
locally, into just one :class:`~airflow.executors.local_executor.LocalExecutor` with multiple modes.

.. note::

When multiple Schedulers are configured with ``executor = LocalExecutor`` in the ``[core]`` section of your ``airflow.cfg``, each Scheduler will run a LocalExecutor. This means tasks would be processed in a distributed fashion across the machines running the Schedulers.

One consideration should be taken into account:

- Restarting a Scheduler: If a Scheduler is restarted, it may take some time for other Schedulers to recognize the orphaned tasks and restart or fail them.

0 comments on commit 288fb12

Please sign in to comment.