Merge remote-tracking branch 'upstream/master' into tj/reference/mult…

…iclass_nms/vector-initialization-bug

.github/workflows/linux_sanitizers.yml

@@ -206,7 +206,7 @@ jobs:
       #
       - name: Upload sccache log
         if: ${{ always() }}
-        uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
+        uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
         with:
           name: sccache_log_${{ matrix.SANITIZER }}
           path: ${{ env.SCCACHE_ERROR_LOG }}

CODE_OF_CONDUCT.md

@@ -0,0 +1,119 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, religion, or sexual identity
+and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the
+  overall community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or
+  advances of any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email
+  address, without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official email address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+openvino_codeofconduct At intel DOT com.
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series
+of actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or
+permanent ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior,  harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within
+the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.0, available at
+[https://www.contributor-covenant.org/version/2/0/code_of_conduct.html][v2.0].

docs/articles_en/about-openvino/performance-benchmarks.rst

@@ -16,14 +16,12 @@ Performance Benchmarks
    Getting Performance Numbers <performance-benchmarks/getting-performance-numbers>
 
 
-This page presents benchmark results for
+This page presents benchmark results for the
 `Intel® Distribution of OpenVINO™ toolkit <https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit.html>`__
 and :doc:`OpenVINO Model Server <../openvino-workflow/model-server/ovms_what_is_openvino_model_server>`, for a representative
 selection of public neural networks and Intel® devices. The results may help you decide which
 hardware to use in your applications or plan AI workload for the hardware you have already
 implemented in your solutions. Click the buttons below to see the chosen benchmark data.
-For a more detailed view of performance numbers for generative AI models, check the
-:doc:`Generative AI Benchmark Results <./performance-benchmarks/generative-ai-performance>`
 
 .. grid:: 1 1 2 2
    :gutter: 4
@@ -36,7 +34,7 @@ For a more detailed view of performance numbers for generative AI models, check
          :outline:
          :expand:
 
-         :material-regular:`bar_chart;1.4em` OpenVINO Benchmark Graphs
+         :material-regular:`bar_chart;1.4em` OpenVINO Benchmark Graphs (general)
 
    .. grid-item::
 
@@ -46,10 +44,35 @@ For a more detailed view of performance numbers for generative AI models, check
          :outline:
          :expand:
 
-         :material-regular:`bar_chart;1.4em` OVMS Benchmark Graphs
+         :material-regular:`bar_chart;1.4em` OVMS Benchmark Graphs (general)
+
+   .. grid-item::
+
+      .. button-link:: ./performance-benchmarks/generative-ai-performance.html
+         :class: ov-toolkit-benchmark-genai
+         :color: primary
+         :outline:
+         :expand:
+
+         :material-regular:`table_view;1.4em` LLM performance for AI PC
+
+   .. grid-item::
+
+      .. button-link:: #
+         :class: ovms-toolkit-benchmark-llm
+         :color: primary
+         :outline:
+         :expand:
+
+         :material-regular:`bar_chart;1.4em` OVMS for GenAI (coming soon)
+
+
+
+
 
 
-Key performance indicators and workload parameters.
+
+**Key performance indicators and workload parameters**
 
 .. tab-set::
 
@@ -65,13 +88,13 @@ Key performance indicators and workload parameters.
    .. tab-item:: Latency
       :sync: latency
 
-      For Vision and NLP models this mhis measures the synchronous execution of inference requests and is reported in
-      milliseconds. Each inference request (for example: preprocess, infer, postprocess) is
-      allowed to complete before the next is started. This performance metric is relevant in
-      usage scenarios where a single image input needs to be acted upon as soon as possible. An
-      example would be the healthcare sector where medical personnel only request analysis of a
-      single ultra sound scanning image or in real-time or near real-time applications for
-      example an industrial robot's response to actions in its environment or obstacle avoidance
+      For Vision and NLP models this measures the synchronous execution of inference requests and
+      is reported in milliseconds. Each inference request (for example: preprocess, infer,
+      postprocess) is allowed to complete before the next one starts. This performance metric is
+      relevant in usage scenarios where a single image input needs to be acted upon as soon as
+      possible. An example would be the healthcare sector where medical personnel only request
+      analysis of a single ultra sound scanning image or in real-time or near real-time applications
+      such as an industrial robot's response to actions in its environment or obstacle avoidance
       for autonomous vehicles.
       For Transformer models like Stable-Diffusion this measures the time it takes to convert the prompt
       or input text into a finished image. It is presented in seconds.
@@ -97,9 +120,10 @@ Key performance indicators and workload parameters.
       * input token length: 1024 (the tokens for GenAI models are in English).
 
 
-.. raw:: html
+**Platforms, Configurations, Methodology**
 
-   <h2>Platforms, Configurations, Methodology</h2>
+To see the methodology used to obtain the numbers and learn how to test performance yourself,
+see the guide on :doc:`getting performance numbers <performance-benchmarks/getting-performance-numbers>`.
 
 For a listing of all platforms and configurations used for testing, refer to the following:
 
@@ -130,59 +154,10 @@ For a listing of all platforms and configurations used for testing, refer to the
          :material-regular:`download;1.5em` Click for Performance Data [XLSX]
 
 
-The OpenVINO benchmark setup includes a single system with OpenVINO™, as well as the benchmark
-application installed. It measures the time spent on actual inference (excluding any pre or post
-processing) and then reports on the inferences per second (or Frames Per Second).
-
-OpenVINO™ Model Server (OVMS) employs the Intel® Distribution of OpenVINO™ toolkit runtime
-libraries and exposes a set of models via a convenient inference API over gRPC or HTTP/REST.
-Its benchmark results are measured with the configuration of multiple-clients-single-server,
-using two hardware platforms connected by ethernet. Network bandwidth depends on both platforms
-and models used. It is set not to be a bottleneck for workload intensity. The connection is
-dedicated only to measuring performance.
-
-.. dropdown:: See more details about OVMS benchmark setup
-
-   The benchmark setup for OVMS consists of four main parts:
 
-   .. image:: ../assets/images/performance_benchmarks_ovms_02.png
-      :alt: OVMS Benchmark Setup Diagram
 
-   * **OpenVINO™ Model Server** is launched as a docker container on the server platform and it
-     listens to (and answers) requests from clients. OpenVINO™ Model Server is run on the same
-     system as the OpenVINO™ toolkit benchmark application in corresponding benchmarking. Models
-     served by OpenVINO™ Model Server are located in a local file system mounted into the docker
-     container. The OpenVINO™ Model Server instance communicates with other components via ports
-     over a dedicated docker network.
 
-   * **Clients** are run in separated physical machine referred to as client platform. Clients
-     are implemented in Python3 programming language based on TensorFlow* API and they work as
-     parallel processes. Each client waits for a response from OpenVINO™ Model Server before it
-     will send a new next request. The role played by the clients is also verification of
-     responses.
-
-   * **Load balancer** works on the client platform in a docker container. HAProxy is used for
-     this purpose. Its main role is counting of requests forwarded from clients to OpenVINO™
-     Model Server, estimating its latency, and sharing this information by Prometheus service.
-     The reason of locating the load balancer on the client site is to simulate real life
-     scenario that includes impact of physical network on reported metrics.
-
-   * **Execution Controller** is launched on the client platform. It is responsible for
-     synchronization of the whole measurement process, downloading metrics from the load
-     balancer, and presenting the final report of the execution.
-
-
-
-.. raw:: html
-
-   <h2>Test performance yourself</h2>
-
-You can also test performance for your system yourself, following the guide on
-:doc:`getting performance numbers <performance-benchmarks/getting-performance-numbers>`.
-
-.. raw:: html
-
-   <h2>Disclaimers</h2>
+**Disclaimers**
 
 * Intel® Distribution of OpenVINO™ toolkit performance results are based on release
   2024.3, as of July 31, 2024.
@@ -192,22 +167,18 @@ You can also test performance for your system yourself, following the guide on
 
 The results may not reflect all publicly available updates. Intel technologies' features and
 benefits depend on system configuration and may require enabled hardware, software, or service
-activation. Learn more at intel.com, or from the OEM or retailer.
+activation. Learn more at intel.com, the OEM, or retailer.
 
 See configuration disclosure for details. No product can be absolutely secure.
 Performance varies by use, configuration and other factors. Learn more at
 `www.intel.com/PerformanceIndex <https://www.intel.com/PerformanceIndex>`__.
-Your costs and results may vary.
 Intel optimizations, for Intel compilers or other products, may not optimize to the same degree
 for non-Intel products.
 
 
 
 
 
-
-
-
 .. raw:: html
 
    <link rel="stylesheet" type="text/css" href="../_static/css/benchmark-banner.css">

docs/articles_en/about-openvino/performance-benchmarks/generative-ai-performance.rst

@@ -4,7 +4,7 @@ Most Efficient Large Language Models for AI PC
 This page is regularly updated to help you identify the best-performing LLMs on the
 Intel® Core™ Ultra processor family and AI PCs.
 
-The tables below list the key performance indicators for a selection of Large Language Models,
+The tables below list key performance indicators for a selection of Large Language Models,
 running on an Intel® Core™ Ultra 7-165H based system, on built-in GPUs.
 
 
@@ -23,24 +23,34 @@ running on an Intel® Core™ Ultra 7-165H based system, on built-in GPUs.
          :class: modeldata stripe
          :name: supportedModelsTableOv
          :header-rows: 1
-         :file:  ../../_static/download/llm_models.csv
+         :file:  ../../_static/benchmarks_files/llm_models.csv
 
 
-For complete information on the system config, see:
-`Hardware Platforms [PDF] <https://docs.openvino.ai/2024/_static/benchmarks_files/OV-2024.4-platform_list.pdf>`__
-
-To view the data in an editable form, you can download the .csv file here:
-
 .. grid:: 1 1 2 2
    :gutter: 4
 
    .. grid-item::
 
-      .. button-link:: ../../_static/download/llm_models.csv
+      All models listed here were tested with the following parameters:
+
+      *  Framework: PyTorch
+      *  Model precision: INT4
+      *  Beam: 1
+      *  Batch size: 1
+
+   .. grid-item::
+
+      .. button-link:: https://docs.openvino.ai/2024/_static/benchmarks_files/OV-2024.4-platform_list.pdf
          :color: primary
          :outline:
          :expand:
 
-         :material-regular:`download;1.5em` Click for OpenVINO LLM results [CSV]
+         :material-regular:`download;1.5em` Get full system info [PDF]
+
+      .. button-link:: ../../_static/benchmarks_files/llm_models.csv
+         :color: primary
+         :outline:
+         :expand:
 
+         :material-regular:`download;1.5em` Get the data in .csv [CSV]