Skip to content

Commit

Permalink
Merge branch 'master' into itikhono/bug_fix/validate_pass
Browse files Browse the repository at this point in the history
  • Loading branch information
itikhono authored Jun 24, 2024
2 parents 9bd5983 + a903fed commit 28c8a2a
Show file tree
Hide file tree
Showing 265 changed files with 11,062 additions and 9,286 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/code_style.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ jobs:
# always provide suggestions even for skipped scripts in ov_shellcheck tagret
- name: ShellCheck action
if: always()
uses: reviewdog/action-shellcheck@3546242c869924d13293e38e6289e00a26468e02 # v1.22.0
uses: reviewdog/action-shellcheck@52f34f737a16c65b8caa8c51ae1b23036afe5685 # v1.23.0
with:
level: style
reporter: github-pr-review
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,15 +61,14 @@ How AUTO Works
##############

To put it simply, when loading the model to the first device on the list fails, AUTO will try to load it to the next device in line, until one of them succeeds.
What is important, **AUTO starts inference with the CPU of the system by default**, as it provides very low latency and can start inference with no additional delays.
What is important, **AUTO starts inference with the CPU of the system by default unless there is model cached for the best suited device**, as it provides very low latency and can start inference with no additional delays.
While the CPU is performing inference, AUTO continues to load the model to the device best suited for the purpose and transfers the task to it when ready.
This way, the devices which are much slower in compiling models, GPU being the best example, do not impact inference at its initial stages.
For example, if you use a CPU and a GPU, the first-inference latency of AUTO will be better than that of using GPU alone.

Note that if you choose to exclude CPU from the priority list or disable the initial
CPU acceleration feature via ``ov::intel_auto::enable_startup_fallback``, it will be
unable to support the initial model compilation stage. The models with dynamic
input/output or :doc:`stateful operations <../stateful-models>`
unable to support the initial model compilation stage. The models with :doc:`stateful operations <../stateful-models>`
will be loaded to the CPU if it is in the candidate list. Otherwise,
these models will follow the normal flow and be loaded to the device based on priority.

Expand Down
2 changes: 1 addition & 1 deletion docs/dev/build_windows.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ Supported configurations:
3. After the build process finishes, export the newly built Python libraries to the user environment variables:
```
set PYTHONPATH=<openvino_repo>/bin/<arch>/Release/python;%PYTHONPATH%
set OPENVINO_LIB_PATHS=<openvino_repo>/bin/<arch>/Release;%OPENVINO_LIB_PATH%
set OPENVINO_LIB_PATHS=<openvino_repo>/bin/<arch>/Release;<openvino_repo>/temp/tbb/bin
```
or install the wheel with pip:
```
Expand Down
Loading

0 comments on commit 28c8a2a

Please sign in to comment.