Update all-in-one benchmark prompts for continuation task & lookup …

…update for minicpmv (#11827) * Update all-in-one benchmark prompts for continuation task * Small fix * Add pure-text benchmark support for minicpm-v-2_6 * Support lookahead for model.llm generate of minicpmv * Add prompt reference * Small update * Small fix
intel-analytics · Aug 16, 2024 · 96796f9 · 96796f9
1 parent e966e85
commit 96796f9
Show file tree

Hide file tree

Showing 5 changed files with 803 additions and 17 deletions.
diff --git a/python/llm/dev/benchmark/all-in-one/README.md b/python/llm/dev/benchmark/all-in-one/README.md
@@ -4,6 +4,8 @@ All in one benchmark test allows users to test all the benchmarks and record the
 
 Before running, make sure to have [ipex-llm](../../../../../README.md) installed.
 
+> The prompts for benchmarking are from datasets [abisee/cnn_dailymail](https://huggingface.co/datasets/abisee/cnn_dailymail), [Open-Orca/OpenOrca](https://huggingface.co/datasets/Open-Orca/OpenOrca), [THUDM/LongBench](https://huggingface.co/datasets/THUDM/LongBench), etc.
+
 ## Dependencies
 
 ```bash

diff --git a/python/llm/dev/benchmark/all-in-one/prompt/continuation/longbench_2k.txt b/python/llm/dev/benchmark/all-in-one/prompt/continuation/longbench_2k.txt
-Original file line number
+Diff line change
@@ Expand Up @@
     Before running, make sure to have [ipex-llm](../../../../../README.md) installed.
+    > The prompts for benchmarking are from datasets [abisee/cnn_dailymail](https://huggingface.co/datasets/abisee/cnn_dailymail), [Open-Orca/OpenOrca](https://huggingface.co/datasets/Open-Orca/OpenOrca), [THUDM/LongBench](https://huggingface.co/datasets/THUDM/LongBench), etc.
     ## Dependencies
     ```bash
@@ Expand Down @@