Skip to content

Commit

Permalink
Feature/benchmarks 2021 3 ehl (#5191)
Browse files Browse the repository at this point in the history 
* Added EHL config

* Updated graphs

* improve table formatting

* Wrap <iframe> tag with \htmlonly \endhtmlonly to avoid build errors

* Updated graphs

* Fixed links to TDP and Price for 8380
  • Loading branch information
@andrew-zaytsev
andrew-zaytsev authored Apr 12, 2021
1 parent 1c84064 commit ba35364
Show file tree
Hide file tree
Showing 2 changed files with 57 additions and 57 deletions.
110 changes: 55 additions & 55 deletions docs/benchmarks/performance_benchmarks_openvino.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,73 +29,73 @@ Measuring inference performance involves many variables and is extremely use-cas


\htmlonly
<script src="bert-large-uncased-whole-word-masking-squad-int8-0001-ov-2021-3-338-3.js" id="bert-large-uncased-whole-word-masking-squad-int8-0001-ov-2021-3-338-3"></script>
<script src="bert-large-uncased-whole-word-masking-squad-int8-0001-ov-2021-3-338-5.js" id="bert-large-uncased-whole-word-masking-squad-int8-0001-ov-2021-3-338-5"></script>
\endhtmlonly

\htmlonly
<script src="deeplabv3-tf-ov-2021-3-338-3.js" id="deeplabv3-tf-ov-2021-3-338-3"></script>
<script src="deeplabv3-tf-ov-2021-3-338-5.js" id="deeplabv3-tf-ov-2021-3-338-5"></script>
\endhtmlonly

\htmlonly
<script src="densenet-121-tf-ov-2021-3-338-3.js" id="densenet-121-tf-ov-2021-3-338-3"></script>
<script src="densenet-121-tf-ov-2021-3-338-5.js" id="densenet-121-tf-ov-2021-3-338-5"></script>
\endhtmlonly

\htmlonly
<script src="faster-rcnn-resnet50-coco-tf-ov-2021-3-338-3.js" id="faster-rcnn-resnet50-coco-tf-ov-2021-3-338-3"></script>
<script src="faster-rcnn-resnet50-coco-tf-ov-2021-3-338-5.js" id="faster-rcnn-resnet50-coco-tf-ov-2021-3-338-5"></script>
\endhtmlonly

\htmlonly
<script src="googlenet-v1-tf-ov-2021-3-338-3.js" id="googlenet-v1-tf-ov-2021-3-338-3"></script>
<script src="googlenet-v1-tf-ov-2021-3-338-5.js" id="googlenet-v1-tf-ov-2021-3-338-5"></script>
\endhtmlonly

\htmlonly
<script src="inception-v3-tf-ov-2021-3-338-3.js" id="inception-v3-tf-ov-2021-3-338-3"></script>
<script src="inception-v3-tf-ov-2021-3-338-5.js" id="inception-v3-tf-ov-2021-3-338-5"></script>
\endhtmlonly

\htmlonly
<script src="mobilenet-ssd-cf-ov-2021-3-338-3.js" id="mobilenet-ssd-cf-ov-2021-3-338-3"></script>
<script src="mobilenet-ssd-cf-ov-2021-3-338-5.js" id="mobilenet-ssd-cf-ov-2021-3-338-5"></script>
\endhtmlonly

\htmlonly
<script src="mobilenet-v1-1-0-224-tf-ov-2021-3-338-3.js" id="mobilenet-v1-1-0-224-tf-ov-2021-3-338-3"></script>
<script src="mobilenet-v1-1-0-224-tf-ov-2021-3-338-5.js" id="mobilenet-v1-1-0-224-tf-ov-2021-3-338-5"></script>
\endhtmlonly

\htmlonly
<script src="mobilenet-v2-pytorch-ov-2021-3-338-3.js" id="mobilenet-v2-pytorch-ov-2021-3-338-3"></script>
<script src="mobilenet-v2-pytorch-ov-2021-3-338-5.js" id="mobilenet-v2-pytorch-ov-2021-3-338-5"></script>
\endhtmlonly

\htmlonly
<script src="resnet-18-pytorch-ov-2021-3-338-3.js" id="resnet-18-pytorch-ov-2021-3-338-3"></script>
<script src="resnet-18-pytorch-ov-2021-3-338-5.js" id="resnet-18-pytorch-ov-2021-3-338-5"></script>
\endhtmlonly

\htmlonly
<script src="resnet-50-tf-ov-2021-3-338-3.js" id="resnet-50-tf-ov-2021-3-338-3"></script>
<script src="resnet-50-tf-ov-2021-3-338-5.js" id="resnet-50-tf-ov-2021-3-338-5"></script>
\endhtmlonly


\htmlonly
<script src="se-resnext-50-cf-ov-2021-3-338-3.js" id="se-resnext-50-cf-ov-2021-3-338-3"></script>
<script src="se-resnext-50-cf-ov-2021-3-338-5.js" id="se-resnext-50-cf-ov-2021-3-338-5"></script>
\endhtmlonly

\htmlonly
<script src="squeezenet1-1-cf-ov-2021-3-338-3.js" id="squeezenet1-1-cf-ov-2021-3-338-3"></script>
<script src="squeezenet1-1-cf-ov-2021-3-338-5.js" id="squeezenet1-1-cf-ov-2021-3-338-5"></script>
\endhtmlonly


\htmlonly
<script src="ssd300-cf-ov-2021-3-338-3.js" id="ssd300-cf-ov-2021-3-338-3"></script>
<script src="ssd300-cf-ov-2021-3-338-5.js" id="ssd300-cf-ov-2021-3-338-5"></script>
\endhtmlonly

\htmlonly
<script src="yolo-v3-tf-ov-2021-3-338-3.js" id="yolo-v3-tf-ov-2021-3-338-3"></script>
<script src="yolo-v3-tf-ov-2021-3-338-5.js" id="yolo-v3-tf-ov-2021-3-338-5"></script>
\endhtmlonly

\htmlonly
<script src="yolo-v4-tf-ov-2021-3-338-3.js" id="yolo-v4-tf-ov-2021-3-338-3"></script>
<script src="yolo-v4-tf-ov-2021-3-338-5.js" id="yolo-v4-tf-ov-2021-3-338-5"></script>
\endhtmlonly

\htmlonly
<script src="unet-camvid-onnx-0001-ov-2021-3-338-3.js" id="unet-camvid-onnx-0001-ov-2021-3-338-3"></script>
<script src="unet-camvid-onnx-0001-ov-2021-3-338-5.js" id="unet-camvid-onnx-0001-ov-2021-3-338-5"></script>
\endhtmlonly


Expand Down Expand Up @@ -139,25 +139,25 @@ Testing by Intel done on: see test date for each HW platform below.

**CPU Inference Engines (continue)**

| | Intel® Xeon® Gold 5218T | Intel® Xeon® Platinum 8270 |
| ------------------------------- | ---------------------------- | ---------------------------- |
| Motherboard | Intel® Server Board S2600STB | Intel® Server Board S2600STB |
| CPU | Intel® Xeon® Gold 5218T CPU @ 2.10GHz | Intel® Xeon® Platinum 8270 CPU @ 2.70GHz |
| Hyper Threading | ON | ON |
| Turbo Setting | ON | ON |
| Memory | 12 x 32 GB DDR4 2666MHz | 12 x 32 GB DDR4 2933MHz |
| Operating System | Ubuntu* 18.04 LTS | Ubuntu* 18.04 LTS |
| Kernel Version | 5.3.0-24-generic | 5.3.0-24-generic |
| BIOS Vendor | Intel Corporation | Intel Corporation |
| BIOS Version | SE5C620.86B.02.01.<br>0009.092820190230 | SE5C620.86B.02.01.<br>0009.092820190230 |
| BIOS Release | September 28, 2019 | September 28, 2019 |
| BIOS Settings | Select optimized default settings, <br>change power policy to "performance", <br>save & exit | Select optimized default settings, <br>change power policy to "performance", <br>save & exit |
| Batch size | 1 | 1 |
| Precision | INT8 | INT8 |
| Number of concurrent inference requests |32 | 52 |
| Test Date | March 15, 2021 | March 15, 2021 |
| Power dissipation, TDP in Watt | [105](https://ark.intel.com/content/www/us/en/ark/products/193953/intel-xeon-gold-5218t-processor-22m-cache-2-10-ghz.html#tab-blade-1-0-1) | [205](https://ark.intel.com/content/www/us/en/ark/products/192482/intel-xeon-platinum-8270-processor-35-75m-cache-2-70-ghz.html#tab-blade-1-0-1) |
| CPU Price on Mach 15th, 2021, USD<br>Prices may vary | [1,349](https://ark.intel.com/content/www/us/en/ark/products/193953/intel-xeon-gold-5218t-processor-22m-cache-2-10-ghz.html) | [7,405](https://ark.intel.com/content/www/us/en/ark/products/192482/intel-xeon-platinum-8270-processor-35-75m-cache-2-70-ghz.html) |
| | Intel® Xeon® Gold 5218T | Intel® Xeon® Platinum 8270 | Intel® Xeon® Platinum 8380 |
| ------------------------------- | ---------------------------- | ---------------------------- | -----------------------------------------|
| Motherboard | Intel® Server Board S2600STB | Intel® Server Board S2600STB | Intel Corporation / WilsonCity |
| CPU | Intel® Xeon® Gold 5218T CPU @ 2.10GHz | Intel® Xeon® Platinum 8270 CPU @ 2.70GHz | Intel® Xeon® Platinum 8380 CPU @ 2.30GHz |
| Hyper Threading | ON | ON | ON |
| Turbo Setting | ON | ON | ON |
| Memory | 12 x 32 GB DDR4 2666MHz | 12 x 32 GB DDR4 2933MHz | 16 x 16 GB DDR4 3200MHz |
| Operating System | Ubuntu* 18.04 LTS | Ubuntu* 18.04 LTS | Ubuntu* 18.04 LTS |
| Kernel Version | 5.3.0-24-generic | 5.3.0-24-generic | 5.3.0-24-generic |
| BIOS Vendor | Intel Corporation | Intel Corporation | Intel Corporation |
| BIOS Version | SE5C620.86B.02.01.<br>0009.092820190230 | SE5C620.86B.02.01.<br>0009.092820190230 | WLYDCRB1.SYS.0020.<br>P86.2103050636 |
| BIOS Release | September 28, 2019 | September 28, 2019 | March 5, 2021 |
| BIOS Settings | Select optimized default settings, <br>change power policy to "performance", <br>save & exit | Select optimized default settings, <br>change power policy to "performance", <br>save & exit | Select optimized default settings, <br>change power policy to "performance", <br>save & exit |
| Batch size | 1 | 1 | 1 |
| Precision | INT8 | INT8 | INT8 |
| Number of concurrent inference requests |32 | 52 | 80 |
| Test Date | March 15, 2021 | March 15, 2021 | March 22, 2021 |
| Power dissipation, TDP in Watt | [105](https://ark.intel.com/content/www/us/en/ark/products/193953/intel-xeon-gold-5218t-processor-22m-cache-2-10-ghz.html#tab-blade-1-0-1) | [205](https://ark.intel.com/content/www/us/en/ark/products/192482/intel-xeon-platinum-8270-processor-35-75m-cache-2-70-ghz.html#tab-blade-1-0-1) | [270](https://ark.intel.com/content/www/us/en/ark/products/212287/intel-xeon-platinum-8380-processor-60m-cache-2-30-ghz.html) |
| CPU Price, USD<br>Prices may vary | [1,349](https://ark.intel.com/content/www/us/en/ark/products/193953/intel-xeon-gold-5218t-processor-22m-cache-2-10-ghz.html) (on Mach 15th, 2021) | [7,405](https://ark.intel.com/content/www/us/en/ark/products/192482/intel-xeon-platinum-8270-processor-35-75m-cache-2-70-ghz.html) (on Mach 15th, 2021) | [8,099](https://ark.intel.com/content/www/us/en/ark/products/212287/intel-xeon-platinum-8380-processor-60m-cache-2-30-ghz.html) (on March 26th, 2021) |


**CPU Inference Engines (continue)**
Expand Down Expand Up @@ -208,25 +208,25 @@ Testing by Intel done on: see test date for each HW platform below.

**CPU Inference Engines (continue)**

| | Intel Atom® x5-E3940 | Intel® Core™ i3-8100 |
| -------------------- | ---------------------------------- |----------------------------------- |
| Motherboard | | GIGABYTE* Z390 UD |
| CPU | Intel Atom® Processor E3940 @ 1.60GHz | Intel® Core™ i3-8100 CPU @ 3.60GHz |
| Hyper Threading | OFF | OFF |
| Turbo Setting | ON | OFF |
| Memory | 1 x 8 GB DDR3 1600MHz | 4 x 8 GB DDR4 2400MHz |
| Operating System | Ubuntu* 18.04 LTS | Ubuntu* 18.04 LTS |
| Kernel Version | 5.3.0-24-generic | 5.3.0-24-generic |
| BIOS Vendor | American Megatrends Inc.* | American Megatrends Inc.* |
| BIOS Version | 5.12 | F8 |
| BIOS Release | September 6, 2017 | May 24, 2019 |
| BIOS Settings | Default settings | Select optimized default settings, <br> set OS type to "other", <br>save & exit |
| Batch size | 1 | 1 |
| Precision | INT8 | INT8 |
| Number of concurrent inference requests | 4 | 4 |
| Test Date | March 15, 2021 | March 15, 2021 |
| Power dissipation, TDP in Watt | [9.5](https://ark.intel.com/content/www/us/en/ark/products/96485/intel-atom-x5-e3940-processor-2m-cache-up-to-1-80-ghz.html) | [65](https://ark.intel.com/content/www/us/en/ark/products/126688/intel-core-i3-8100-processor-6m-cache-3-60-ghz.html#tab-blade-1-0-1)|
| CPU Price on Mach 15th, 2021, USD<br>Prices may vary | [34](https://ark.intel.com/content/www/us/en/ark/products/96485/intel-atom-x5-e3940-processor-2m-cache-up-to-1-80-ghz.html) | [117](https://ark.intel.com/content/www/us/en/ark/products/126688/intel-core-i3-8100-processor-6m-cache-3-60-ghz.html) |
| | Intel Atom® x5-E3940 | Intel Atom® x6425RE | Intel® Core™ i3-8100 |
| -------------------- | --------------------------------------|------------------------------- |----------------------------------- |
| Motherboard | | Intel Corporation /<br>ElkhartLake LPDDR4x T3 CRB | GIGABYTE* Z390 UD |
| CPU | Intel Atom® Processor E3940 @ 1.60GHz | Intel Atom® x6425RE<br>Processor @ 1.90GHz | Intel® Core™ i3-8100 CPU @ 3.60GHz |
| Hyper Threading | OFF | OFF | OFF |
| Turbo Setting | ON | ON | OFF |
| Memory | 1 x 8 GB DDR3 1600MHz | 2 x 4GB DDR4 3200 MHz | 4 x 8 GB DDR4 2400MHz |
| Operating System | Ubuntu* 18.04 LTS | Ubuntu* 18.04 LTS | Ubuntu* 18.04 LTS |
| Kernel Version | 5.3.0-24-generic | 5.8.0-050800-generic | 5.3.0-24-generic |
| BIOS Vendor | American Megatrends Inc.* | Intel Corporation | American Megatrends Inc.* |
| BIOS Version | 5.12 | EHLSFWI1.R00.2463.<br>A03.2011200425 | F8 |
| BIOS Release | September 6, 2017 | November 22, 2020 | May 24, 2019 |
| BIOS Settings | Default settings | Default settings | Select optimized default settings, <br> set OS type to "other", <br>save & exit |
| Batch size | 1 | 1 | 1 |
| Precision | INT8 | INT8 | INT8 |
| Number of concurrent inference requests | 4 | 4 | 4 |
| Test Date | March 15, 2021 | March 15, 2021 | March 15, 2021 |
| Power dissipation, TDP in Watt | [9.5](https://ark.intel.com/content/www/us/en/ark/products/96485/intel-atom-x5-e3940-processor-2m-cache-up-to-1-80-ghz.html) | [12](https://ark.intel.com/content/www/us/en/ark/products/207899/intel-atom-x6425re-processor-1-5m-cache-1-90-ghz.html) | [65](https://ark.intel.com/content/www/us/en/ark/products/126688/intel-core-i3-8100-processor-6m-cache-3-60-ghz.html#tab-blade-1-0-1)|
| CPU Price, USD<br>Prices may vary | [34](https://ark.intel.com/content/www/us/en/ark/products/96485/intel-atom-x5-e3940-processor-2m-cache-up-to-1-80-ghz.html) (on March 15th, 2021) | [59](https://ark.intel.com/content/www/us/en/ark/products/207899/intel-atom-x6425re-processor-1-5m-cache-1-90-ghz.html) (on March 26th, 2021) | [117](https://ark.intel.com/content/www/us/en/ark/products/126688/intel-core-i3-8100-processor-6m-cache-3-60-ghz.html) (on March 15th, 2021) |



Expand Down
4 changes: 2 additions & 2 deletions docs/get_started/get_started_dl_workbench.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -71,9 +71,9 @@ The simplified OpenVINO™ DL Workbench workflow is:
## Run Baseline Inference

This section illustrates a sample use case of how to infer a pretrained model from the [Intel® Open Model Zoo](@ref omz_models_group_intel) with an autogenerated noise dataset on a CPU device.

\htmlonly
<iframe width="560" height="315" src="https://www.youtube.com/embed/9TRJwEmY0K4" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

\endhtmlonly

Once you log in to the DL Workbench, create a project, which is a combination of a model, a dataset, and a target device. Follow the steps below:

Expand Down

0 comments on commit ba35364

Please sign in to comment.