Skip to content

Commit

Permalink
[DOCS] Update model table for master (#23080)
Browse files Browse the repository at this point in the history
Port for #23062
  • Loading branch information
akopytko authored Feb 26, 2024
1 parent c40b675 commit 69736c6
Showing 1 changed file with 26 additions and 24 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,12 @@ Model Accuracy



The following two tables present the absolute accuracy drop calculated as the accuracy difference
between OV-accuracy and the original frame work accuracy for FP32, and the same for INT8, BF16 and
FP16 representations of a model on three platform architectures. Please also refer to notes below
The following two tables present the absolute accuracy drop calculated as the accuracy difference
between OV-accuracy and the original frame work accuracy for FP32, and the same for INT8, BF16 and
FP16 representations of a model on three platform architectures. Please also refer to notes below
The following two tables present the absolute accuracy drop calculated as the accuracy difference
between OV-accuracy and the original frame work accuracy for FP32, and the same for INT8, BF16 and
FP16 representations of a model on three platform architectures. Please also refer to notes below
the table for more information.

* A - Intel® Core™ i9-9000K (AVX2), INT8 and FP32
Expand Down Expand Up @@ -105,28 +108,28 @@ the table for more information.
* - chatGLM2-6b
- lambada openai
- ppl
-
-
- 17.38
- 17.41
- 17.17
* - Llama-2-7b-chat
- Wiki, StackExch, Crawl
- ppl
-
-
- 3.24
- 3.24
- 3.25
- 3.25
* - Stable-Diffusion-V2-1
- LIAON-5B
- CLIP
-
-
-
-
-
-
-
* - Mistral-7b
- proprietary Mistral.ai
- ppl
-
-
- 3.29
- 3.47
- 3.49
Expand Down Expand Up @@ -233,37 +236,36 @@ the table for more information.
* - chatGLM2-6b
- lambada openai
- ppl
-
-
- 17.48
- 17.56
-
-
- 17.49
* - Llama-2-7b-chat
- Wiki, StackExch, Crawl
- ppl
-
-
- 3.26
- 3.26
-
-
-
* - Stable-Diffusion-V2-1
- LIAON-5B
- CLIP
-
-
-
-
-
-
-
-
- 22.48
* - Mistral-7b
- proprietary Mistral.ai
- ppl
-
-
- 3.19
- 3.18
-
-
-
-

Notes: For all accuracy metrics except perplexity a "-", (minus sign), indicates an accuracy drop.
For perplexity (ppl) the values do not indicate a deviation from a reference but are the actual measured
Notes: For all accuracy metrics except perplexity a "-", (minus sign), indicates an accuracy drop.
For perplexity (ppl) the values do not indicate a deviation from a reference but are the actual measured
accuracy for the model.

0 comments on commit 69736c6

Please sign in to comment.