Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return None if an XComArg fails to resolve in a multiple_outputs Task #32027

Merged
merged 2 commits into from
Jun 21, 2023

Conversation

ephraimbuddy
Copy link
Contributor

Tasks with multiple_outputs set to True returns XComs with different keys which are not known to Airflow. Because they have multiple_outputs set, we should return None if we can't find the XCom, just like we return None when the key is equal to XCOM_RETURN_KEY known to Airflow.

Closes: #29199

@ephraimbuddy ephraimbuddy requested a review from uranusjr as a code owner June 20, 2023 11:45
Tasks with multiple_outputs set to True returns XComs with different keys
which are not known to Airflow. Because they have multiple_outputs set, we
should return None if we can't find the XCom, just like we return None when
the key is equal to XCOM_RETURN_KEY known to Airflow.

Closes: apache#29199
@ephraimbuddy ephraimbuddy added this to the Airlfow 2.6.3 milestone Jun 21, 2023
@ephraimbuddy ephraimbuddy added the type:bug-fix Changelog: Bug Fixes label Jun 21, 2023
@ephraimbuddy ephraimbuddy merged commit 79eac76 into apache:main Jun 21, 2023
@ephraimbuddy ephraimbuddy deleted the multioutputs-error branch June 21, 2023 09:55
ferruzzi pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Jun 27, 2023
…apache#32027)

* Return None if an XComArg fails to resolve in a multiple_outputs Task

Tasks with multiple_outputs set to True returns XComs with different keys
which are not known to Airflow. Because they have multiple_outputs set, we
should return None if we can't find the XCom, just like we return None when
the key is equal to XCOM_RETURN_KEY known to Airflow.

Closes: apache#29199

* Apply suggestions from code review
ephraimbuddy added a commit that referenced this pull request Jul 6, 2023
…#32027)

* Return None if an XComArg fails to resolve in a multiple_outputs Task

Tasks with multiple_outputs set to True returns XComs with different keys
which are not known to Airflow. Because they have multiple_outputs set, we
should return None if we can't find the XCom, just like we return None when
the key is equal to XCOM_RETURN_KEY known to Airflow.

Closes: #29199

* Apply suggestions from code review

(cherry picked from commit 79eac76)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:bug-fix Changelog: Bug Fixes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

TaskFlow AirflowSkipException causes downstream step to fail when multiple_outputs is true
2 participants