-
Notifications
You must be signed in to change notification settings - Fork 14.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ECSOperator returns last logs when ECS task fails #17209
ECSOperator returns last logs when ECS task fails #17209
Conversation
(Forgot to comment when hitting submit: the implementation itself looks good to me, but the tests can be improved.) |
Co-authored-by: Tzu-ping Chung <[email protected]>
Co-authored-by: Tzu-ping Chung <[email protected]>
Co-authored-by: Tzu-ping Chung <[email protected]>
Co-authored-by: Tzu-ping Chung <[email protected]>
Co-authored-by: Tzu-ping Chung <[email protected]>
Thanks for your review @uranusjr ! |
I added some other tests for _last_log_messages and making sure it works if there are no messages at all. I wanted to do all of that in the same test instead of splitting it into 2 (one for _last_log_messages() and one for the exception message) but unfortunately there are issues with mocks and generators (can only be called once because then it's empty) and fixing this with https://stackoverflow.com/a/65019164/5706548 might be confusing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think Flake8 is complaining about these; logic-wise this looks good to me.
(I could have missed something. You should run pre-commit locally to pass the linters; see https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#contribution-workflow) |
Co-authored-by: Tzu-ping Chung <[email protected]>
Co-authored-by: Tzu-ping Chung <[email protected]>
Thanks @uranusjr I just ran the flake8 tests locally now after your changes and it works. |
The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease. |
closes: #17038
This PR changes the message in the AirflowException when the ECS task launched by ECSOperator is stopped.
Before:
The message when it failed was:
This task is not in success state {<huge JSON from AWS containing all the ECS task details>}
Now:
The message is:
which makes it much more useful to understand what failed in the underlying code directly from the alert.
The number of logs can be customized with the parameter
number_logs_exception
.FYI @uranusjr