Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docs around when to use TaskGroup vs SubDag and potentially listing PROs and CONS. #12298

Closed
kaxil opened this issue Nov 11, 2020 · 3 comments · Fixed by #20700
Closed

Add docs around when to use TaskGroup vs SubDag and potentially listing PROs and CONS. #12298

kaxil opened this issue Nov 11, 2020 · 3 comments · Fixed by #20700

Comments

@kaxil
Copy link
Member

kaxil commented Nov 11, 2020

It would be great for users to know when they should use TaskGroup vs SubDag. A section somewhere in docs would be great or even better a Matrix / table to explain difference would be aweomse.

What are the PROs and CONs of each

TaskGroup: https://airflow.readthedocs.io/en/latest/concepts.html#taskgroup
SubDags: https://airflow.readthedocs.io/en/latest/concepts.html#subdags

@VeenaArv
Copy link

Hi, can I work on this as my first issue? Should this go under concepts or somewhere else?

@kaxil
Copy link
Member Author

kaxil commented Nov 17, 2020

@VeenaArv Yes somewhere in Concepts, in or after https://github.com/apache/airflow/blob/master/docs/concepts.rst#taskgroup section

@turbaszek
Copy link
Member

@kaxil @VeenaArv should we also add information about TriggerDagRunOperator as this is also a possibility to manage large DAGs?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants