trtllm-serve does not support dynamic batching like tritonserver

[Alireza3242]

I want to have a openai compatible api. But i also need the advantages of tritonserver like dynamic batching. I think if you develop a code which uses tritonserver (tensorrt-backend) to deliver openai-api is better.

[njaramish]

Looks like this is supported in Triton, although I have not tried it myself: triton-inference-server/server#7561

There's also an open source project that fills this need (also have not tried it myself): https://github.com/npuichigo/openai_trtllm