[LLM Bench] Defining Framework in Torch Compile Benchmarking (#1354)

It looks like the framework needs to be specified as pytorch for the models to be compile with torch compile, otherwise it takes the OV framework route and never hits the torch compile code. Although the following [line](https://github.com/openvinotoolkit/openvino.genai/blob/b26fc8b7a484e0f66accba89ea9f972c6d23fda7/tools/llm_bench/llm_bench_utils/pt_utils.py#L157) tries to use torch compile on the entire image generation pipeline which causes issues since it is expected to compile the models within the pipeline. --------- Co-authored-by: Ekaterina Aidova <[email protected]> Co-authored-by: Ilya Lavrenov <[email protected]>
openvinotoolkit · Dec 23, 2024 · 5d68567 · 5d68567
1 parent 1179cb6
commit 5d68567
Showing 1 changed file with 3 additions and 0 deletions.
diff --git a/tools/llm_bench/llm_bench_utils/model_utils.py b/tools/llm_bench/llm_bench_utils/model_utils.py
@@ -137,6 +137,9 @@ def analyze_args(args):
 
     model_framework = args.framework
     model_path = Path(args.model)
+    if model_args["torch_compile_backend"]:
+        log.info("Setting Framework to PyTorch Since torch_compile_backend is provided.")
+        model_framework = 'pt'
     if not model_path.exists():
         raise RuntimeError(f'==Failure FOUND==: Incorrect model path:{model_path}')
     if model_framework in ('ov', 'pt'):