vLLM: RayWorkerVllm.execute_method failed with error 'TCPStore is not available'

[nkwangleiGIT]

Ray v2.9.3
vLLM 0.3.3

[RayWorkerVllm.execute_method]
failed with callstack below:

Error Type: TASK_EXECUTION_EXCEPTION

home/ray/anaconda3/lib/python3.9/site-packages/cupyx/distributed/_nccl_comm.py", line 97, in _init_with_tcp_store
self._store_proxy.barrier()
File "/home/ray/anaconda3/lib/python3.9/site-packages/cupyx/distributed/_store.py", line 152, in barrier
self._send_recv(_store_actions.Barrier())
File "/home/ray/anaconda3/lib/python3.9/site-packages/cupyx/distributed/_store.py", line 142, in _send_recv
raise RuntimeError('TCPStore is not available')
RuntimeError: TCPStore is not available

Related issues on Ray and vLLM:
ray-project/ray#43756
vllm-project/vllm#3334

[nkwangleiGIT]

add --enforce-eager to EXTRA_ARGS to disable cupy, and vllm mode can work normally.

[bjwswang]

should be fixed by #884