You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
2024-08-30 18:21:25.407568117 [W:onnxruntime:, session_state.cc:1166 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2024-08-30 18:21:25.407597785 [W:onnxruntime:, session_state.cc:1168 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
INFO 2024-08-30 18:21:26,115 infinity_emb INFO: Optimizing model utils_optimum.py:139
WARNING 2024-08-30 18:21:26,118 infinity_emb WARNING: Optimization failed with Tried to use utils_optimum.py:168
ORTOptimizer for the model type , but it is not available yet. Please open an issue or submit a PR at https://github.com/huggingface/optimum.. Going to use the unoptimized model.
It's even slow than use default torch engine of infinity-emb start as
System Info
Who can help?
@JingyaHuang @michaelbenayoun
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction (minimal, reproducible, runnable)
When I run jinaai/jina-reranker-v2-base-multilingual with infinity-emb server
the start-up logs
It's even slow than use default torch engine of infinity-emb start as
Expected behavior
optimum engine is faster than torch engine when model is support, for example "netease-youdao/Rerank"
The text was updated successfully, but these errors were encountered: