EOF occured in violation of protocol -- From EC2 AWS

[bertello72]

I got this error message executing the code in Ec2 AWS Linux Ubuntu 22.04.2 (not locally): [SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1007)

Replicate v. 0.23.1

Error log:

Traceback (most recent call last):
  File "/home/ubuntu/News/LLM_confronto.py", line 42, in generate_summary
    for event in replicate.stream(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/replicate/stream.py", line 200, in stream
    with client._client.stream("GET", url, headers=headers) as response:
  File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__
    return next(self.gen)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/httpx/_client.py", line 871, in stream
    response = self.send(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/httpx/_client.py", line 915, in send
    response = self._send_handling_auth(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/httpx/_client.py", line 943, in _send_handling_auth
    response = self._send_handling_redirects(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/httpx/_client.py", line 980, in _send_handling_redirects
    response = self._send_single_request(request)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/httpx/_client.py", line 1016, in _send_single_request
    response = transport.handle_request(request)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/replicate/client.py", line 258, in handle_request
    response = self._wrapped_transport.handle_request(request)  # type: ignore
  File "/home/ubuntu/.local/lib/python3.10/site-packages/httpx/_transports/default.py", line 230, in handle_request
    with map_httpcore_exceptions():
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/httpx/_transports/default.py", line 84, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectError: [SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1007)

[mattt]

Hi, @bertello72. Is this happening every time? Intermittently? Just a one-off?

[bertello72]

It varies a lot. First attempt: ~600 words in the prompt, ~400 words in output, and then crashed Second attempt: ~600 words in the prompt, and then crashed (no output) Third attempt: three consecutive prompts with ok outputs, and then crashed Il giorno mar 30 gen 2024 alle ore 17:59 Mattt ***@***.***> ha scritto:

…

Hi, @bertello72 <https://github.com/bertello72>. Is this happening every time? Intermittently? Just a one-off? — Reply to this email directly, view it on GitHub <#237 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BFTUJONAJJCIL77YQHABP5DYRERGPAVCNFSM6AAAAABCRKTIVOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJXGQ4DSMZZGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[mattt]

@bertello72 Thanks for clarifying. Searching around for that error message, it sounds like the problem can be resolved by upgrading your installation of OpenSSL or nginx (if you're using that). See this SO thread.

[bertello72]

i haven't used nginx and I've checked OpenSSL; it's the latest available in AWS EC2 Amazon Linux.

AWS Service Support gave me this advice:

<<I did researched about the error "[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1007)" and found that it might be related to SSL/TLS mismtach between server and hosts. I have got information from below external source:
https://community.cisco.com/t5/radkit-discussions/getting-conda-error-encountered-an-ssl-error-most-likely-a/td-p/4873809>>

Does it rise in your mind some insights on how to fix the problem?

[mattt]

@bertello72 According to python/cpython#110467, this appears to be a problem with Python 3.10. I'd recommend upgrading to a newer version of Python.

Since this doesn't appear to be a problem with the Replicate client library itself, I'll go ahead and close this issue for now.