You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
To call Triton ensemble models without grpc call, one way is using C API. Is there any Python library so that I can reuse existing ensemble model config files?
I looked at PyTriton and its documentation is focusing on binding python functions into Triton, while my problem is not related to bind python functions into Triton easily. My models are Paddle models and C backend code. That make Triton server a good investment which can be use to serve ensemble models.
Now I'm trying to use Triton combined with RayServe, using python triton GRPC client will incur more grpc call overhead, and not compatible with RayServe auto scaling. So the best way is to use C API in Python ray actor that requires a Python library for calling Triton C API.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
To call Triton ensemble models without grpc call, one way is using C API. Is there any Python library so that I can reuse existing ensemble model config files?
I looked at PyTriton and its documentation is focusing on binding python functions into Triton, while my problem is not related to bind python functions into Triton easily. My models are Paddle models and C backend code. That make Triton server a good investment which can be use to serve ensemble models.
Now I'm trying to use Triton combined with RayServe, using python triton GRPC client will incur more grpc call overhead, and not compatible with RayServe auto scaling. So the best way is to use C API in Python ray actor that requires a Python library for calling Triton C API.
Beta Was this translation helpful? Give feedback.
All reactions