Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactoring get_cpu_pinning() #22489

Merged
merged 16 commits into from
Feb 23, 2024
4 changes: 2 additions & 2 deletions .github/workflows/windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -913,7 +913,7 @@ jobs:
CPU_Functional_Tests:
name: CPU functional tests
needs: [ Build, Smart_CI ]
timeout-minutes: 70
timeout-minutes: 90
defaults:
run:
shell: pwsh
Expand Down Expand Up @@ -977,7 +977,7 @@ jobs:
run: |
. "${{ env.INSTALL_DIR }}/setupvars.ps1"
python3 ${{ env.PARALLEL_TEST_SCRIPT }} -e ${{ env.INSTALL_TEST_DIR }}/ov_cpu_func_tests.exe -c ${{ env.PARALLEL_TEST_CACHE }} -w ${{ env.INSTALL_TEST_DIR }} -s suite -- --gtest_filter=*smoke*
timeout-minutes: 60
timeout-minutes: 90

- name: Save tests execution time
uses: actions/cache/save@v4
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/windows_conditional_compilation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -362,7 +362,7 @@ jobs:
CPU_Functional_Tests:
name: CPU functional tests
needs: [Build, Smart_CI]
timeout-minutes: 85
timeout-minutes: 90
defaults:
run:
shell: pwsh
Expand Down Expand Up @@ -416,7 +416,7 @@ jobs:
run: |
set path=%path%;${{ env.INSTALL_TEST_DIR }}\tbb\bin;${{ env.INSTALL_TEST_DIR }}\tbb
python3 ${{ env.PARALLEL_TEST_SCRIPT }} -e ${{ env.INSTALL_TEST_DIR }}\ov_cpu_func_tests.exe -w ${{ env.INSTALL_TEST_DIR }} -s suite -rf 0 -- --gtest_print_time=1 --gtest_filter=*smoke*
timeout-minutes: 60
timeout-minutes: 90

- name: Upload Test Results
uses: actions/upload-artifact@v3
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -357,6 +357,9 @@ Read-only properties
- ``ov::device::full_name``
- ``ov::device::capabilities``

.. note::
``ov::affinity`` is replaced by ``ov::hint::enable_cpu_pinning``. As such, it is deprecated in the 2024.0 release and will be removed in the 2025 release.

External Dependencies
###########################################################

Expand Down Expand Up @@ -404,7 +407,16 @@ User can use the following properties to limit available CPU resource for model

``ov::hint::scheduling_core_type`` and ``ov::hint::enable_hyper_threading`` only support Intel® x86-64 CPU on Linux and Windows in current release.

By default, OpenVINO Runtime will enable CPU threads pinning for better performance. User also can use property ``ov::hint::enable_cpu_pinning`` to switch it off. Disable threads pinning might be beneficial in complex applications with several workloads executed in parallel.
In some use cases, OpenVINO Runtime will enable CPU threads pinning by default for better performance. User can also turn it on or off using property ``ov::hint::enable_cpu_pinning``. Disable threads pinning might be beneficial in complex applications with several workloads executed in parallel. The following table describes the default setting for ``ov::hint::enable_cpu_pinning`` in different use cases.

==================================================== ================================
Use Case Default Setting of CPU Pinning
==================================================== ================================
All use cases with Windows OS False
Stream contains both Pcore and Ecore with Linux OS False
Stream only contains Pcore or Ecore with Linux OS True
All use cases with Mac OS False
==================================================== ================================

.. tab-set::

Expand All @@ -427,7 +439,7 @@ user can check the :doc:`optimization guide <openvino_docs_deployment_optimizati

.. note::

``ov::hint::enable_cpu_pinning`` only support Linux in current release.
``ov::hint::enable_cpu_pinning`` is not supported on multi-socket platforms with Windows OS.

Denormals Optimization
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Expand Down
12 changes: 7 additions & 5 deletions src/plugins/intel_cpu/src/config.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -93,20 +93,21 @@ void Config::readProperties(const ov::AnyMap& prop, const ModelType modelType) {
OPENVINO_SUPPRESS_DEPRECATED_START
} else if (key == ov::affinity.name()) {
try {
ov::Affinity affinity = val.as<ov::Affinity>();
changedCpuPinning = true;
ov::Affinity affinity = val.as<ov::Affinity>();
#if defined(__APPLE__)
enableCpuPinning = false;
threadBindingType = affinity == ov::Affinity::NONE ? IStreamsExecutor::ThreadBindingType::NONE
: IStreamsExecutor::ThreadBindingType::NUMA;
#else
enableCpuPinning =
(affinity == ov::Affinity::CORE || affinity == ov::Affinity::HYBRID_AWARE) ? true : false;
switch (affinity) {
case ov::Affinity::NONE:
threadBindingType = IStreamsExecutor::ThreadBindingType::NONE;
break;
case ov::Affinity::CORE: {
#if (defined(__APPLE__) || defined(_WIN32))
threadBindingType = IStreamsExecutor::ThreadBindingType::NUMA;
#else
threadBindingType = IStreamsExecutor::ThreadBindingType::CORES;
#endif
} break;
case ov::Affinity::NUMA:
threadBindingType = IStreamsExecutor::ThreadBindingType::NUMA;
Expand All @@ -121,6 +122,7 @@ void Config::readProperties(const ov::AnyMap& prop, const ModelType modelType) {
key,
". Expected only ov::Affinity::CORE/NUMA/HYBRID_AWARE.");
}
#endif
} catch (const ov::Exception&) {
OPENVINO_THROW("Wrong value ",
val.as<std::string>(),
Expand Down
35 changes: 17 additions & 18 deletions src/plugins/intel_cpu/src/cpu_map_scheduling.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
#include "cpu_streams_calculation.hpp"
#include "openvino/core/parallel.hpp"
#include "openvino/runtime/system_conf.hpp"
#include "openvino/runtime/threading/cpu_streams_info.hpp"

namespace ov {
namespace intel_cpu {
Expand Down Expand Up @@ -71,32 +72,30 @@ std::vector<std::vector<int>> apply_hyper_threading(bool& input_ht_hint,

bool get_cpu_pinning(bool& input_value,
const bool input_changed,
const int num_streams,
const Config::LatencyThreadingMode latency_threading_mode,
const std::vector<std::vector<int>>& proc_type_table) {
int result_value;
int num_sockets = get_default_latency_streams(latency_threading_mode);
bool latency = num_streams <= num_sockets && num_streams > 0;
const std::vector<std::vector<int>>& proc_type_table,
const std::vector<std::vector<int>>& streams_info_table) {
bool result_value;

#if defined(__APPLE__)
result_value = false;
#elif defined(_WIN32)
result_value = ((input_changed) && (proc_type_table.size() == 1)) ? input_value : false;
#else
if (input_changed) {
result_value = input_value;
sunxiaoxia2022 marked this conversation as resolved.
Show resolved Hide resolved
} else {
result_value = true;
if (proc_type_table[0][EFFICIENT_CORE_PROC] > 0 &&
proc_type_table[0][EFFICIENT_CORE_PROC] < proc_type_table[0][ALL_PROC]) {
result_value = latency ? false : true;
// The following code disables pinning in case stream contains both Pcore and Ecore
if (streams_info_table.size() >= 3) {
if ((streams_info_table[0][PROC_TYPE] == ALL_PROC) &&
(streams_info_table[1][PROC_TYPE] != EFFICIENT_CORE_PROC) &&
(streams_info_table[2][PROC_TYPE] == EFFICIENT_CORE_PROC)) {
result_value = false;
}
}
}
#if (OV_THREAD == OV_THREAD_TBB || OV_THREAD == OV_THREAD_TBB_AUTO)
# if defined(_WIN32)
if (proc_type_table.size() > 1) {
result_value = false;
}
# endif
# if defined(__APPLE__)
result_value = false;
# endif
#endif

input_value = result_value;

return result_value;
Expand Down
10 changes: 4 additions & 6 deletions src/plugins/intel_cpu/src/cpu_map_scheduling.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -44,16 +44,14 @@ std::vector<std::vector<int>> apply_hyper_threading(bool& input_ht_hint,
* @brief whether pinning cpu cores according to enableCpuPinning property
* @param[in] input_type indicate value of property enableCpuPinning.
* @param[in] input_changed indicate if value is set by user.
* @param[in] num_streams number of streams
* @param[in] latency_threading_mode is the scope of candidate processors per stream for latency hint
* @param[in] proc_type_table candidate processors available at this time
* @param[in] proc_type_table indicate processors information of this platform
* @param[in] streams_info_table indicate streams detail of this model
* @return whether pinning threads to cpu cores
*/
bool get_cpu_pinning(bool& input_value,
const bool input_changed,
const int num_streams,
const Config::LatencyThreadingMode latency_threading_mode,
const std::vector<std::vector<int>>& proc_type_table);
const std::vector<std::vector<int>>& proc_type_table,
const std::vector<std::vector<int>>& streams_info_table);

} // namespace intel_cpu
} // namespace ov
9 changes: 4 additions & 5 deletions src/plugins/intel_cpu/src/cpu_streams_calculation.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -534,11 +534,7 @@ std::vector<std::vector<int>> generate_stream_info(const int streams,
config.changedHyperThreading,
ov::util::to_string(config.hintPerfMode),
proc_type_table);
auto cpu_reservation = get_cpu_pinning(config.enableCpuPinning,
config.changedCpuPinning,
streams,
config.latencyThreadingMode,
proc_type_table);

if (-1 == preferred_nthreads_per_stream) {
model_prefer_threads = get_model_prefer_threads(streams, proc_type_table, model, config);
}
Expand All @@ -553,6 +549,9 @@ std::vector<std::vector<int>> generate_stream_info(const int streams,
config.latencyThreadingMode,
proc_type_table);

auto cpu_reservation =
get_cpu_pinning(config.enableCpuPinning, config.changedCpuPinning, proc_type_table, streams_info_table);

config.streamExecutorConfig = IStreamsExecutor::Config{"CPUStreamsExecutor",
config.streams,
config.threadsPerStream,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -133,19 +133,17 @@ TEST_F(OVClassConfigTestCPU, smoke_PluginSetConfigStreamsNum) {

TEST_F(OVClassConfigTestCPU, smoke_PluginSetConfigAffinity) {
ov::Core ie;
ov::Affinity value = ov::Affinity::NONE;

#if (defined(__APPLE__) || defined(_WIN32))
auto numaNodes = ov::get_available_numa_nodes();
auto coreTypes = ov::get_available_cores_types();
#if defined(__APPLE__)
ov::Affinity value = ov::Affinity::CORE;
auto defaultBindThreadParameter = ov::Affinity::NONE;
if (coreTypes.size() > 1) {
defaultBindThreadParameter = ov::Affinity::HYBRID_AWARE;
} else if (numaNodes.size() > 1) {
defaultBindThreadParameter = ov::Affinity::NUMA;
}
#else
ov::Affinity value = ov::Affinity::NUMA;
# if defined(_WIN32)
auto defaultBindThreadParameter = ov::Affinity::NONE;
# else
auto defaultBindThreadParameter = ov::Affinity::CORE;
# endif
auto coreTypes = ov::get_available_cores_types();
if (coreTypes.size() > 1) {
defaultBindThreadParameter = ov::Affinity::HYBRID_AWARE;
Expand All @@ -154,10 +152,15 @@ TEST_F(OVClassConfigTestCPU, smoke_PluginSetConfigAffinity) {
ASSERT_NO_THROW(value = ie.get_property("CPU", ov::affinity));
ASSERT_EQ(defaultBindThreadParameter, value);

const ov::Affinity affinity = defaultBindThreadParameter == ov::Affinity::HYBRID_AWARE ? ov::Affinity::NUMA : ov::Affinity::HYBRID_AWARE;
const ov::Affinity affinity =
defaultBindThreadParameter == ov::Affinity::HYBRID_AWARE ? ov::Affinity::NUMA : ov::Affinity::HYBRID_AWARE;
ASSERT_NO_THROW(ie.set_property("CPU", ov::affinity(affinity)));
ASSERT_NO_THROW(value = ie.get_property("CPU", ov::affinity));
#if defined(__APPLE__)
ASSERT_EQ(ov::Affinity::NUMA, value);
#else
ASSERT_EQ(affinity, value);
#endif
}

TEST_F(OVClassConfigTestCPU, smoke_PluginSetConfigAffinityCore) {
Expand All @@ -167,12 +170,20 @@ TEST_F(OVClassConfigTestCPU, smoke_PluginSetConfigAffinityCore) {

ASSERT_NO_THROW(ie.set_property("CPU", ov::affinity(affinity)));
ASSERT_NO_THROW(value = ie.get_property("CPU", ov::hint::enable_cpu_pinning));
#if defined(__APPLE__)
ASSERT_EQ(false, value);
#else
ASSERT_EQ(true, value);
#endif

affinity = ov::Affinity::HYBRID_AWARE;
ASSERT_NO_THROW(ie.set_property("CPU", ov::affinity(affinity)));
ASSERT_NO_THROW(value = ie.get_property("CPU", ov::hint::enable_cpu_pinning));
#if defined(__APPLE__)
ASSERT_EQ(false, value);
#else
ASSERT_EQ(true, value);
#endif

affinity = ov::Affinity::NUMA;
ASSERT_NO_THROW(ie.set_property("CPU", ov::affinity(affinity)));
Expand Down
Loading
Loading