Update Readme for FastChat docker demo

[ATMxsp01]

Update Readme for FastChat docker demo

[liu-shaojun]

To set up model serving using IPEX-LLM as the backend with FastChat, you can refer to this Quickstart guide or follow these quick steps to deploy a demo.

Quick Setup for FastChat with IPEX-LLM

1. Start the Docker Container

   Run the following command to launch a Docker container with device access:

   #/bin/bash
   export DOCKER_IMAGE=intelanalytics/ipex-llm-serving-xpu:latest
   
   sudo docker run -itd \
       --net=host \
       --device=/dev/dri \
       --name=demo-container \
       -v /LLM_MODELS/:/llm/models/ \   # Example: map host model directory to container
       --shm-size="16g" \
       -e http_proxy=... \              # Optional: set proxy if needed
       -e https_proxy=... \
       -e no_proxy="127.0.0.1,localhost" \
       $DOCKER_IMAGE

2. Start the FastChat Service

   Enter the container and start the FastChat service:

   #/bin/bash
   
   # Stop any existing FastChat processes
   ps -ef | grep "fastchat" | awk '{print $2}' | xargs kill -9
   
   # Install the required Gradio version
   pip install -U gradio==4.43.0
   
   # Launch the FastChat controller
   python -m fastchat.serve.controller &
   
   # Set environment variables for CCL
   export TORCH_LLM_ALLREDUCE=0
   export CCL_DG2_ALLREDUCE=1
   export CCL_WORKER_COUNT=2
   # Optional: Pin CCL workers to specific cores
   # export CCL_WORKER_AFFINITY=32,33,34,35
   export FI_PROVIDER=shm
   export CCL_ATL_TRANSPORT=ofi
   export CCL_ZE_IPC_EXCHANGE=sockets
   export CCL_ATL_SHM=1
   
   # Load Intel CCL settings
   source /opt/intel/1ccl-wks/setvars.sh
   
   # Start the model worker (replace "Yi-1.5-34B" with your model name)
   python -m ipex_llm.serving.fastchat.vllm_worker \
       --model-path /llm/models/Yi-1.5-34B \
       --device xpu \
       --enforce-eager \
       --dtype float16 \
       --load-in-low-bit fp8 \
       --tensor-parallel-size 4 \
       --gpu-memory-utilization 0.9 \
       --max-model-len 4096 \
       --max-num-batched-tokens 8000 &
   
   # Wait for initialization
   sleep 120
   
   # Start the Gradio web server for FastChat
   python -m fastchat.serve.gradio_web_server &

This quick setup allows you to deploy FastChat with IPEX-LLM efficiently.

[glorysdj]

LGTM

[liu-shaojun]

LGTM