SimpleHttpOperator aborts connection after 5 minutes #8160
Comments
|
Thanks for opening your first issue here! Be sure to follow the issue template! |
|
It seems that the server to which SimpleHttpOperator connects breaks the connection after 5 minutes. I've tried to reproduce this bug using simple flask app: from flask import Flask
from time import monotonic, sleep
app = Flask(__name__)
@app.route('/')
def hello_world():
t0 = monotonic()
sleep(302)
t1 = monotonic()
return "{}".format(t1-t0)Everything works great: |
|
I don't think it is a server issue, because I made the same request on Postman, and it worked. |
|
Are you running with the SequentialExecutor? |
No, I'm using CeleryExecutor |
|
This one appears to be something specific to your environment or the request you are making, as I can't reproduce this behavour, nor could khyurri. Without more detailed reproduction steps that we can try ourselves we won't be able to help with this one. |
|
I faced a similar issue, for me it was a bad request that was sent using |
|
@anirudhbagri Could you give me details as to what the root cause of your issue was and how you fixed it? I'm currently experiencing the same issue and I think it would help a lot. |
|
For me, the error was because of wrong input. The API was expecting a list of string but I was sending only a single string. httpOperator(conn="{{ ti.xcom_pull("{0}", "{1}") }}".format(someval, otherval),...)Instead this one worked: endpoint = "{{ ti.xcom_pull("{0}", "{1}") }}".format(someval, otherval)
httpOperator(conn=endpoint,...) |
|
@BrunoDamacena Thanks |
|
This is very likely already handled in the upcoming https://pypi.org/project/apache-airflow-providers-http/4.0.0rc1/ (it will be likely released tomorrow / Monday latest). Can you please @ADITI7499 install it locally and check if it fixes the problem? The fix was implemented in #24967 |
|
Also anyone in this thread - if you face similar issue Please try 4.0.0rc1 HTTP provider! |
|
@potiuk I tried this new http provider with my airflow 2.2 and as it picks default value as tcp_keep_alive true. It should be sufficient to tackle this issue right but again my remote connection got closed |
|
You might want to increase the frequency of default settings. If that's your firewall (or whatever is between you and the host) it seems overly agressive. I would expect closing idle connection after an hour but 5 minutes is unheard of. So maybe someone at your company has very agressive policy. If you look at the description of the KeepAlive feature, they mention that firewalls might be also agressively closing connections where keep alive does not come frequently enough. Ideally - find out who is doing it and what are the rules, then adjust the settings. Or experiment. Or maybe this is completely different problem. But finding out who is doing it is the only way forward (it's not Airflow for sure). |
Apache Airflow version: v1.10.4
Kubernetes version (if you are using kubernetes) (use
kubectl version):Environment: puckel/docker-airflow
What happened:
The HTTP request from the API aborts connection after 5 minutes.
I'm trying to run a long request, but everytime this error occurs:
What you expected to happen:
The operator should wait for the request response without timeout
How to reproduce it:
Run a long request using SimpleHttpOperator
Anything else we need to know:
The text was updated successfully, but these errors were encountered: