Skip to content

Commit

Permalink
[DOCS] benchmark fix table
Browse files Browse the repository at this point in the history
  • Loading branch information
kblaszczak-intel committed Feb 23, 2024
1 parent 381a550 commit 3091e9c
Showing 1 changed file with 31 additions and 35 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@ Model Accuracy



The following two tables present the absolute accuracy drop calculated as the accuracy difference
between OV-accuracy and the original frame work accuracy for FP32, and the same for INT8, BF16 and
FP16 representations of a model on three platform architectures. Please also refer to notes below
The following two tables present the absolute accuracy drop calculated as the accuracy difference
between OV-accuracy and the original frame work accuracy for FP32, and the same for INT8, BF16 and
FP16 representations of a model on three platform architectures. Please also refer to notes below
The following two tables present the absolute accuracy drop calculated as the accuracy difference
between OV-accuracy and the original frame work accuracy for FP32, and the same for INT8, BF16 and
FP16 representations of a model on three platform architectures. Please also refer to notes below
The following two tables present the absolute accuracy drop calculated as the accuracy difference
between OV-accuracy and the original frame work accuracy for FP32, and the same for INT8, BF16 and
FP16 representations of a model on three platform architectures. Please also refer to notes below
the table for more information.

* A - Intel® Core™ i9-9000K (AVX2), INT8 and FP32
Expand Down Expand Up @@ -108,32 +108,28 @@ the table for more information.
* - chatGLM2-6b
- lambada openai
- ppl
-
-
- 17.38
- 17.41
- 17.17
-
- 17.37 (ref. 17.48)
- 17.41 (ref. 17.48)
- 17.16 (ref. 17.48)
* - Llama-2-7b-chat
- Wiki, StackExch, Crawl
- ppl
-
-
- 3.24
- 3.24
- 3.25
- 3.25
* - Stable-Diffusion-V2-1
- LIAON-5B
- CLIP
-
-
-
-
-
-
-
-
* - Mistral-7b
- proprietary Mistral.ai
- ppl
-
-
- 3.29
- 3.47
- 3.49
Expand Down Expand Up @@ -172,15 +168,15 @@ the table for more information.
- -0.02%
- -0.02%
- -0.02%
- -0.03%
_ -0.03%
* - mask_rcnn_resnet50_atrous_coco
- COCO2017_detection_91cl_bkgr
- coco_orig_precision
- -0.01%
- -0.01%
- %
- -0.18%
- 0.02%
_ 0.02%
* - mobilenet-v2
- ImageNet2012
- accuracy @ top1
Expand Down Expand Up @@ -240,37 +236,37 @@ the table for more information.
* - chatGLM2-6b
- lambada openai
- ppl
-
-
- 17.48
- 17.56
-
-
- 17.49
* - Llama-2-7b-chat
- Wiki, StackExch, Crawl
- ppl
-
-
- 3.26
- 3.26
-
-
-
-
* - Stable-Diffusion-V2-1
- LIAON-5B
- CLIP
-
-
-
-
- 22.48
-
-
-
-
- 22.48
* - Mistral-7b
- proprietary Mistral.ai
- ppl
-
-
- 3.19
- 3.18
-
-
-
-

Notes: For all accuracy metrics except perplexity a "-", (minus sign), indicates an accuracy drop.
For perplexity (ppl) the values do not indicate a deviation from a reference but are the actual measured
Notes: For all accuracy metrics except perplexity a "-", (minus sign), indicates an accuracy drop.
For perplexity (ppl) the values do not indicate a deviation from a reference but are the actual measured
accuracy for the model.

0 comments on commit 3091e9c

Please sign in to comment.