-
Notifications
You must be signed in to change notification settings - Fork 14.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Less verbose logging in ssh operator #24915
Conversation
a3e7032
to
a639ae5
Compare
Static checks are faiing. |
@potiuk I will try to fix them tomorrow. btw, do you think the current change is valid? |
Yep. if error message is already printed it makes no sense to print it again. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes make sense to me, should be ready after fixing static check errors.
Checks are successful now. @uranusjr |
Just in time for the next provider's wave |
Hi @potiuk, @hsnprsd! We need to see stderr in exception - it is viewable via error email reporting and significantly reduces the time to analyse the problem for supports. What about return stderr to exception? I can implement it. So ideally i want to see something like this:
|
Yes. Might be a good idea @morooshka - feel free to implement it. |
Sorry for commenting on an old PR. Before this change, exceptions thrown by paramiko (eg. host not reachable, etc) were caught and raised forward as an AirflowException. After this change, that's not the case. Was that intentional @hsnprsd? If not, I can raise a separate issue if you agree this is not desired behaviour. |
Airlfow Exception is only relevant as base exception for some special behaviours (timeout etc.). There is no particular reason why you should expect provider to throw only Airflow Exception, at any point in time any unhandled exception can be raised so your code should be ready to handle any of those. |
Current logs of
SSHOperator
are too verbose.SSHHook
already logs stdout and stderr of the running command in task logs, so there is no need for including stderr inAirflowException
message returned byrun_ssh_client_command
. Also there is no need to catch and then raise the same exception inexecute
method ofSSHOperator
.Example of current logs: