Skip to content

Commit

Permalink
Update README.md (#11824)
Browse files Browse the repository at this point in the history
  • Loading branch information
jason-dai authored Aug 16, 2024
1 parent 9e9086c commit 6a8d07d
Showing 1 changed file with 9 additions and 3 deletions.
12 changes: 9 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,9 +158,15 @@ See the demo of running [*Text-Generation-WebUI*](https://ipex-llm.readthedocs.i
Please see the **Perplexity** result below (tested on Wikitext dataset using the script [here](https://github.com/intel-analytics/ipex-llm/tree/main/python/llm/dev/benchmark/perplexity)).
|Perplexity |sym_int4 |q4_k |fp6 |fp8_e5m2 |fp8_e4m3 |fp16 |
|---------------------------|---------|-------|-------|---------|---------|-------|
|Llama-2-7B-chat-hf |6.3638 |6.2179 |6.0924 |6.1796 |6.0980 |6.0963 |
|Mistral-7B-Instruct-v0.1 |6.0025 |5.9581 |5.8930 |5.8884 |5.8820 |5.8734 |
|Qwen1.5-7B-chat |8.8652 |8.8163 |8.5573 |8.8463 |8.5304 |8.6065 |
|Llama-2-7B-chat-hf |6.364 |6.218 |6.092 |6.180 |6.098 |6.096 |
|Mistral-7B-Instruct-v0.2 |5.365 |5.320 |5.270 |5.273 |5.246 |5.244 |
|Baichuan2-7B-chat |6.734 |6.727 |6.527 |6.539 |6.488 |6.508 |
|Qwen1.5-7B-chat |8.865 |8.816 |8.557 |8.846 |8.530 |8.607 |
|Llama-3.1-8B-Instruct |6.705 |6.566 |6.338 |6.383 |6.325 |6.267 |
|gemma-2-9b-it |7.541 |7.412 |7.269 |7.380 |7.268 |7.270 |
|Baichuan2-13B-Chat |6.313 |6.160 |6.070 |6.145 |6.086 |6.031 |
|Llama-2-13b-chat-hf |5.449 |5.422 |5.341 |5.384 |5.332 |5.329 |
|Qwen1.5-14B-Chat |7.529 |7.520 |7.367 |7.504 |7.297 |7.334 |

[^1]: Performance varies by use, configuration and other factors. `ipex-llm` may not optimize to the same degree for non-Intel products. Learn more at www.Intel.com/PerformanceIndex.

Expand Down

0 comments on commit 6a8d07d

Please sign in to comment.