API Timeout Issue in a Self-Hosted Web Workflow

[MILKMILKKING]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

Dify version

0.15.0

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Description

I’ve deployed the Dify project on my server and tried using the HTTP module in workflows to add content to the knowledge base through an API. While API calls work seamlessly with curl on the server and terminal, identical requests via Dify's HTTP module consistently result in timeouts.

Despite numerous tests and troubleshooting attempts, the problem remains unresolved. Any advice or solutions would be greatly appreciated.

---

Symptoms Observed

1. Server-side, Code, and Terminal Tests:

   • API requests tested with the curl command on the server terminal respond as expected:

     Outcome: Successfully returned a response.

2. HTTP Module in Dify Workflows:

   • API calls result in timeouts, typically showing extended connection waits before failing.

3. Additional Notes:

   • HTTPS and reverse proxy configurations are correctly enabled in the Nginx setup.

---

Troubleshooting Steps Taken

1. Validated API Functionality:

   • API requests using local IP (127.0.0.1) and domain names returned expected results.
   • Verified the service is bound to 0.0.0.0 and its ports are externally accessible.

2. Redirection and HTTPS Tests:

   • Checked for potential issues with HTTP-to-HTTPS redirections.
   • Used the --location parameter in curl tests, and all redirections completed without issue.

3. CORS Configuration Review:

   • Confirmed the following headers are included in the Nginx configuration:

     add_header 'Access-Control-Allow-Origin' '*';  
     add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';  
     add_header 'Access-Control-Allow-Headers' 'Authorization, Content-Type';  

   • This did not resolve the timeout issue on the web front.

4. Network Environment Validation:

   • Ensured the client can resolve the domain to the server’s public IP.
   • Used ping and curl to confirm that domain resolution and connectivity are operational.

5. Browser Developer Tools:

   • Network panel logs show no evident 3xx redirections.
   • The primary issue reported is a connection timeout.

✔️ Expected Behavior

Expected Outcome

• The HTTP module in workflows should perform API calls successfully and return proper responses.

❌ Actual Behavior

No response

[dosubot]

Hey @MILKMILKKING! 👋 I'm here to help you with your bug report and answer any questions you have. Let's squash those bugs together!

There are a few similar discussions and issues related to HTTP request timeouts in self-hosted Dify environments that might be helpful:

1. Node HTTP Request run failed: Reached maximum retries (3) for URL - This discussion is still open and might provide insights into similar timeout issues ^[1].

2. HTTP Request utility does not work (timeout) on a self-hosted environment - This closed issue suggests increasing the GUNICORN_TIMEOUT environment variable to 360 seconds in the docker-compose.yaml file and removing the SSRF_PROXY_HTTP_URL and SSRF_PROXY_HTTPS_URL environment variables from the API container as potential solutions ^[2].

Additionally, the default timeout for HTTP requests in Dify's workflow module is 15 seconds, which might be causing the issue if the requests take longer than this duration ^[3].

You might want to try adjusting the timeout settings and checking the environment variables as suggested in the closed issue.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}