docs: Add BentoML deployment doc (vllm-project#3336)

Signed-off-by: Sherlock113 <[email protected]>
Temirulan · Mar 12, 2024 · 55d2491 · 55d2491
1 parent 72425a5
commit 55d2491
Show file tree

Hide file tree

Showing 2 changed files with 9 additions and 0 deletions.
diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -73,6 +73,7 @@ Documentation
    serving/run_on_sky
    serving/deploying_with_kserve
    serving/deploying_with_triton
+   serving/deploying_with_bentoml
    serving/deploying_with_docker
    serving/serving_with_langchain
    serving/metrics

diff --git a/docs/source/serving/deploying_with_bentoml.rst b/docs/source/serving/deploying_with_bentoml.rst
@@ -0,0 +1,8 @@
+.. _deploying_with_bentoml:
+
+Deploying with BentoML
+======================
+
+`BentoML <https://github.com/bentoml/BentoML>`_ allows you to deploy a large language model (LLM) server with vLLM as the backend, which exposes OpenAI-compatible endpoints. You can serve the model locally or containerize it as an OCI-complicant image and deploy it on Kubernetes.
+
+For details, see the tutorial `vLLM inference in the BentoML documentation <https://docs.bentoml.com/en/latest/use-cases/large-language-models/vllm.html>`_.