the basic code causes a bug. drives me crazy!!!

[dongrixinyu]

What is the problem?

*Ray version: 1.2.0
Python version:3.8 (anaconda),
OS: linux ubuntu 18:04

pip install ray[serve]

Reproduction (REQUIRED)

the basic code:

import ray
from ray import serve
import requests

ray.init(num_cpus=4)
client = serve.start()


def say_hello(request):
    return "hello " + request.query_params["name"] + "!"


# Form a backend from our function and connect it to an endpoint.
client.create_backend("my_backend", say_hello)
client.create_endpoint("my_endpoint", backend="my_backend", route="/hello")

# Query our endpoint in two different ways: from HTTP and from Python.
print(requests.get("http://127.0.0.1:8000/hello?name=serve").text)
# > hello serve!
print(ray.get(client.get_handle("my_endpoint").remote(name="serve")))
# > hello serve!

and the below is the bug and error reported:

(pid=15846) WARNING:  Invalid HTTP request received.
(pid=15846) Traceback (most recent call last):
(pid=15846)   File "/home/cuichengyu/anaconda3/envs/py38/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 172, in handle_events
(pid=15846)     event = self.conn.next_event()
(pid=15846)   File "/home/cuichengyu/anaconda3/envs/py38/lib/python3.8/site-packages/h11/_connection.py", line 443, in next_event
(pid=15846)     exc._reraise_as_remote_protocol_error()
(pid=15846)   File "/home/cuichengyu/anaconda3/envs/py38/lib/python3.8/site-packages/h11/_util.py", line 76, in _reraise_as_remote_protocol_error
(pid=15846)     raise self
(pid=15846)   File "/home/cuichengyu/anaconda3/envs/py38/lib/python3.8/site-packages/h11/_connection.py", line 425, in next_event
(pid=15846)     event = self._extract_next_receive_event()
(pid=15846)   File "/home/cuichengyu/anaconda3/envs/py38/lib/python3.8/site-packages/h11/_connection.py", line 367, in _extract_next_receive_event
(pid=15846)     event = self._reader(self._receive_buffer)
(pid=15846)   File "/home/cuichengyu/anaconda3/envs/py38/lib/python3.8/site-packages/h11/_readers.py", line 68, in maybe_read_from_IDLE_client
(pid=15846)     raise LocalProtocolError("illegal request line")
(pid=15846) h11._util.RemoteProtocolError: illegal request line
(pid=15846) 2021-03-08 15:13:53,996     INFO router.py:248 -- Endpoint my_endpoint doesn't exist, waiting for registration.

Could you tell me how to avoid the basic bug?

[dongrixinyu]

plus there is always a log (pid=15846) 2021-03-08 15:13:53,996 INFO router.py:248 -- Endpoint my_endpoint doesn't exist, waiting for registration.
could you tell me if the endpoint registered successfully?!!!???

[dongrixinyu]

Package                  Version
------------------------ -------------------
aiohttp                  3.7.4
aiohttp-cors             0.7.0
aioredis                 1.3.1
async-timeout            3.0.1
attrs                    20.3.0
blessings                1.7
cachetools               4.2.1
certifi                  2020.12.5
chardet                  3.0.4
click                    7.1.2
colorama                 0.4.4
colorful                 0.5.4
ConfigArgParse           1.3
cssselect                1.1.0
cycler                   0.10.0
docker-py                1.10.6
docker-pycreds           0.4.0
filelock                 3.0.12
Flask                    1.1.2
Flask-BasicAuth          0.2.0
gevent                   21.1.2
geventhttpclient         1.4.5
google-api-core          1.26.0
google-auth              1.27.0
googleapis-common-protos 1.53.0
gpustat                  0.6.0
greenlet                 1.0.0
grpc-channelz            0.0.1
grpcio                   1.35.0
h11                      0.12.0
hiredis                  1.1.0
idna                     2.10
itsdangerous             1.1.0
Jinja2                   2.11.3
jsonschema               3.2.0
kiwisolver               1.3.1
locust                   1.4.3
lxml                     4.6.2
MarkupSafe               1.1.1
matplotlib               3.3.4
msgpack                  1.0.2
multidict                5.1.0
numpy                    1.20.1
nvidia-ml-py3            7.352.0
nvidia-pyindex           1.0.6
opencensus               0.7.12
opencensus-context       0.1.2
opencv-python            4.5.1.48
packaging                20.9
Pillow                   8.1.0
pip                      21.0.1
prometheus-client        0.9.0
protobuf                 3.15.2
psutil                   5.8.0
py-spy                   0.3.4
pyasn1                   0.4.8
pyasn1-modules           0.2.8
pydantic                 1.6.1
pyparsing                2.4.7
pyquery                  1.4.3
pyrsistent               0.17.3
python-dateutil          2.8.1
python-rapidjson         1.0
pytz                     2021.1
PyYAML                   5.4.1
pyzmq                    22.0.3
ray                      1.2.0
redis                    3.5.3
requests                 2.25.1
rsa                      4.7.2
setuptools               52.0.0.post20210125
six                      1.15.0
starlette                0.14.2
torch                    1.7.1+cu110
torchaudio               0.7.2
torchvision              0.8.2+cu110
tritonclient             2.3.0
typing-extensions        3.7.4.3
urllib3                  1.26.3
uvicorn                  0.13.4
websocket-client         0.57.0
Werkzeug                 1.0.1
wheel                    0.36.2
yarl                     1.6.3
zope.event               4.5.0
zope.interface           5.2.0

can not stand up with this tool

[simon-mo]

Hi! Googling the error message leads to this link encode/httpx#96, which seems like a long standing bug that's hard to reproduce. Does this still happen when you try install ray[serve] in a clean conda env?

[dongrixinyu]

Hi! Googling the error message leads to this link encode/httpx#96, which seems like a long standing bug that's hard to reproduce. Does this still happen when you try install ray[serve] in a clean conda env?

yes, it mistangled with pytorch and opencv.
and this bug occuered from nowhere.

(pid=20053) 2021-03-09 10:17:13,750     INFO http_state.py:67 -- Starting HTTP proxy with name 'rgMWBf:SERVE_CONTROLLER_ACTOR:SERVE_PROXY_ACTOR-node:192.168.35.2-0' on node 'node:192.168.35.2-0' listening on '192.168.35.2:8000'
(pid=20056) INFO:     Started server process [20056]
> /home/cuichengyu/ray_server.py(31)<module>()
-> client.create_endpoint("my_endpoint", backend="my_backend", route="/hello")
(Pdb) (pid=20056) WARNING:  Invalid HTTP request received.
(pid=20056) Traceback (most recent call last):
(pid=20056)   File "/home/cuichengyu/anaconda3/envs/ray_server/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 172, in handle_events
(pid=20056)     event = self.conn.next_event()
(pid=20056)   File "/home/cuichengyu/anaconda3/envs/ray_server/lib/python3.8/site-packages/h11/_connection.py", line 443, in next_event
(pid=20056)     exc._reraise_as_remote_protocol_error()
(pid=20056)   File "/home/cuichengyu/anaconda3/envs/ray_server/lib/python3.8/site-packages/h11/_util.py", line 76, in _reraise_as_remote_protocol_error
(pid=20056)     raise self
(pid=20056)   File "/home/cuichengyu/anaconda3/envs/ray_server/lib/python3.8/site-packages/h11/_connection.py", line 425, in next_event
(pid=20056)     event = self._extract_next_receive_event()
(pid=20056)   File "/home/cuichengyu/anaconda3/envs/ray_server/lib/python3.8/site-packages/h11/_connection.py", line 367, in _extract_next_receive_event
(pid=20056)     event = self._reader(self._receive_buffer)
(pid=20056)   File "/home/cuichengyu/anaconda3/envs/ray_server/lib/python3.8/site-packages/h11/_readers.py", line 68, in maybe_read_from_IDLE_client
(pid=20056)     raise LocalProtocolError("illegal request line")
(pid=20056) h11._util.RemoteProtocolError: illegal request line
--KeyboardInterrupt--
(Pdb) --KeyboardInterrupt--
(Pdb) --KeyboardInterrupt--
(Pdb) q
Traceback (most recent call last):
  File "ray_server.py", line 31, in <module>
    client.create_endpoint("my_endpoint", backend="my_backend", route="/hello")
  File "ray_server.py", line 31, in <module>
    client.create_endpoint("my_endpoint", backend="my_backend", route="/hello")
  File "/home/cuichengyu/anaconda3/envs/ray_server/lib/python3.8/bdb.py", line 88, in trace_dispatch
    return self.dispatch_line(frame)
  File "/home/cuichengyu/anaconda3/envs/ray_server/lib/python3.8/bdb.py", line 113, in dispatch_line
    if self.quitting: raise BdbQuit
bdb.BdbQuit

it can not co-existed with pytorch and opencv

torch                    1.8.0+cu111
torchaudio               0.8.0
torchvision              0.9.0+cu111
opencv-python            4.5.1.48

[dongrixinyu]

plus, this bug occured in every 20 seconds. and can not be handled invisibly.

[grechaw]

Based on the PR above that references this bug, i think we should resolve this one as fixed.