Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GHA][MO][openvino-dev] Remove tests for legacy openvino-dev from GHA #27495

Merged
merged 32 commits into from
Nov 19, 2024
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
5c87a54
[GHA][MO][openvino-dev] Remove tests for legacy openvino-dev from GHA
rkazants Nov 10, 2024
60c1720
Remove rest of legacy artifacts
rkazants Nov 11, 2024
44c3c88
Update .github/workflows/job_python_unit_tests.yml
rkazants Nov 11, 2024
083868d
Update .github/workflows/windows_vs2019_release.yml
rkazants Nov 11, 2024
17e5278
Update .github/workflows/windows_vs2019_release.yml
rkazants Nov 11, 2024
ca167f2
Update .github/workflows/job_python_unit_tests.yml
rkazants Nov 11, 2024
c8c80e1
Install additional deps for ovc unit tests
rkazants Nov 11, 2024
46f76d4
Update job_python_unit_tests.yml
rkazants Nov 11, 2024
d20bedc
Update windows_vs2019_release.yml
rkazants Nov 11, 2024
3f8edb0
Update .github/workflows/job_onnx_models_tests.yml
rkazants Nov 11, 2024
2364357
Fix issues with ONNX models tests
rkazants Nov 12, 2024
084e87e
Merge remote-tracking branch 'origin/remove_legacy_tests' into remove…
rkazants Nov 12, 2024
3cf7585
Merge remote-tracking branch 'upstream/master' into remove_legacy_tests
rkazants Nov 12, 2024
74a6fbe
Fix ovc param
rkazants Nov 12, 2024
7183348
Remove invalid test cases using type specification in ovc tool
rkazants Nov 14, 2024
35af2a8
Install onnx req file
rkazants Nov 14, 2024
89ba377
Update tests/layer_tests/ovc_python_api_tests/test_pytorch.py
rkazants Nov 14, 2024
8afbc10
Apply suggestions from code review
rkazants Nov 14, 2024
8eebeb1
Apply suggestions from code review
rkazants Nov 14, 2024
bf6aae6
Install ONNX tests reqs from dedicated file
rkazants Nov 14, 2024
7ccf4e2
Merge remote-tracking branch 'origin/remove_legacy_tests' into remove…
rkazants Nov 14, 2024
6f2d9eb
Merge remote-tracking branch 'upstream/master' into remove_legacy_tests
rkazants Nov 14, 2024
71569db
Update .github/workflows/windows_vs2019_release.yml
rkazants Nov 14, 2024
61eb712
Update windows_vs2019_release.yml
rkazants Nov 14, 2024
aa1aa5c
Update .github/workflows/job_python_unit_tests.yml
rkazants Nov 14, 2024
41f9c8f
Update .github/workflows/windows_vs2019_release.yml
rkazants Nov 14, 2024
1f2e554
Merge remote-tracking branch 'upstream/master' into remove_legacy_tests
rkazants Nov 18, 2024
70007c2
Merge remote-tracking branch 'origin/remove_legacy_tests' into remove…
rkazants Nov 18, 2024
1695f3d
Remove legacy cases
rkazants Nov 18, 2024
1196026
Install ONNX deps for tests
rkazants Nov 18, 2024
586c7e2
Update CMakeLists.txt
rkazants Nov 19, 2024
c580876
Update .github/dependabot.yml
rkazants Nov 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -108,8 +108,6 @@
/tools/ @openvinotoolkit/openvino-tools-maintainers
/tools/benchmark_tool/ @openvinotoolkit/openvino-ie-python-api-maintainers
/tools/legacy/ @openvinotoolkit/openvino-samples-maintainers
/tools/openvino_dev/ @openvinotoolkit/openvino-tools-maintainers @openvinotoolkit/openvino-ie-python-api-maintainers
/tools/mo/ @openvinotoolkit/openvino-mo-maintainers
/tools/ovc/ @openvinotoolkit/openvino-ovc-maintainers
/thirdparty/open_model_zoo/ @openvinotoolkit/omz-maintainers

Expand Down
2 changes: 1 addition & 1 deletion .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ updates:
- "rkazants"
versioning-strategy: increase-if-necessary

# Model Optimizer, openvino_dev and Benchmark tool
# ovc tool
rkazants marked this conversation as resolved.
Show resolved Hide resolved
- package-ecosystem: pip
directory: "/tools"
schedule:
Expand Down
6 changes: 0 additions & 6 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -100,10 +100,6 @@
'category: LP transformations':
- 'src/common/low_precision_transformations/**/*'

'category: MO':
- 'tools/mo/**/*'
- 'tests/layer_tests/mo_python_api_tests/**/*'

'category: OVC':
- 'tools/ovc/**/*'
- 'tests/layer_tests/ovc_python_api_tests/**/*'
Expand All @@ -119,7 +115,6 @@
- any: ['src/bindings/js/node/CMakeLists.txt',
'src/bindings/js/node/package.json',
'src/bindings/js/node/package-lock.json']
- 'tools/openvino_dev/**/*'

'category: PDPD FE':
- 'src/frontends/paddle/**/*'
Expand Down Expand Up @@ -183,7 +178,6 @@

'category: tools':
- any: ['tools/**',
'!tools/mo/**/*',
'!tools/ovc/**/*']

'category: transformations':
Expand Down
9 changes: 0 additions & 9 deletions .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,12 +50,6 @@ jobs:
python3 -m pip install -r ${{ github.workspace }}/src/frontends/onnx/tests/requirements.txt
# For running TensorFlow frontend unit tests
python3 -m pip install -r ${{ github.workspace }}/src/frontends/tensorflow/tests/requirements.txt
# For MO unit tests
python3 -m pip install -r ${{ github.workspace }}/tools/mo/requirements_caffe.txt
python3 -m pip install -r ${{ github.workspace }}/tools/mo/requirements_kaldi.txt
python3 -m pip install -r ${{ github.workspace }}/tools/mo/requirements_onnx.txt
python3 -m pip install -r ${{ github.workspace }}/tools/mo/requirements_tf2.txt
python3 -m pip install -r ${{ github.workspace }}/tools/mo/requirements_dev.txt

- name: Build OpenVINO with CMake
uses: ashutoshvarma/action-cmake-build@ade188313bc7eaa6f14349569a64d8bc716342ff # master
Expand Down Expand Up @@ -84,9 +78,6 @@ jobs:
- name: Install wheel packages
run: cmake -DCOMPONENT=python_wheels -DCMAKE_INSTALL_PREFIX=${{ github.workspace }}/install_pkg -P '${{ github.workspace }}/build/cmake_install.cmake'

- name: Install python wheels
run: python3 -m pip install openvino-dev --find-links=${{ github.workspace }}/install_pkg/tools

- name: List binaries
run: ls -la ${{ github.workspace }}/bin/intel64/${{ env.CMAKE_BUILD_TYPE }}

Expand Down
9 changes: 3 additions & 6 deletions .github/workflows/job_onnx_models_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,14 +86,11 @@ jobs:
run: |
# Install the core OV wheel
python3 -m pip install ./openvino-*.whl

extras_to_install="onnx"

# Find and install OV dev wheel
ov_dev_wheel_name=$(find . -name 'openvino_dev*.whl')
python3 -m pip install $ov_dev_wheel_name[$extras_to_install]
working-directory: ${{ env.INSTALL_WHEELS_DIR }}

- name: Install ONNX Models tests requirements
run: python3 -m pip install -r ${INSTALL_TEST_DIR}/requirements_onnx

rkazants marked this conversation as resolved.
Show resolved Hide resolved
- name: Install Python tests dependencies
run: |
# To enable pytest parallel features
Expand Down
87 changes: 13 additions & 74 deletions .github/workflows/job_python_unit_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,28 +91,31 @@ jobs:
should-setup-pip-paths: ${{ runner.os == 'Linux' }}
self-hosted-runner: ${{ runner.os == 'Linux' }}

#
# Tests
#

- name: Install OpenVINO Python wheels
run: |
# Install the core OV wheel
python3 -m pip install ./openvino-*.whl

rkazants marked this conversation as resolved.
Show resolved Hide resolved
extras_to_install="caffe,kaldi,onnx,tensorflow2,pytorch"

# Find and install OV dev wheel
ov_dev_wheel_name=$(find . -name 'openvino_dev*.whl')
python3 -m pip install $ov_dev_wheel_name[$extras_to_install]
working-directory: ${{ env.INSTALL_WHEELS_DIR }}
rkazants marked this conversation as resolved.
Show resolved Hide resolved

- name: Install Python API tests dependencies
run: |
# To enable pytest parallel features
python3 -m pip install pytest-xdist[psutil]
python3 -m pip install -r ${INSTALL_TEST_DIR}/bindings/python/requirements_test.txt
python3 -m pip install -r ${INSTALL_TEST_DIR}/mo/requirements_dev.txt

- name: Install Python Layer tests dependencies and for OVC unit tests
run: |
# For torchvision to OpenVINO preprocessing converter
python3 -m pip install -r ${INSTALL_TEST_DIR}/python/preprocess/torchvision/requirements.txt

# layer test requirements
python3 -m pip install -r ${LAYER_TESTS_INSTALL_DIR}/requirements.txt

- name: Install ONNX tests dependencies
run: |
# ONNX tests requirements
python3 -m pip install -r ${LAYER_TESTS_INSTALL_DIR}/requirements_onnx
rkazants marked this conversation as resolved.
Show resolved Hide resolved

#
# Tests
Expand All @@ -127,18 +130,6 @@ jobs:
--junitxml=${INSTALL_TEST_DIR}/TEST-Pyngraph.xml \
--ignore=${INSTALL_TEST_DIR}/pyopenvino/tests/test_utils/test_utils.py

- name: Model Optimizer unit tests
if: fromJSON(inputs.affected-components).MO.test
run: |
if [[ "${{ runner.os }}" == "Linux" ]] && [[ "${{ runner.arch }}" != "ARM64" ]]; then
# required for MxNet
apt-get install -y libgomp1 libquadmath0
fi

# Skips under tickets: 133405, 122666
python3 -m pytest -s ${INSTALL_TEST_DIR}/mo/unit_tests \
--junitxml=${INSTALL_TEST_DIR}/TEST-ModelOptimizer.xml

- name: Python ONNX operators tests
if: (fromJSON(inputs.affected-components).Python_API.test ||
fromJSON(inputs.affected-components).ONNX_FE.test) &&
Expand All @@ -153,32 +144,6 @@ jobs:
if: fromJSON(inputs.affected-components).MO.test
run: python3 -m pytest -s ${INSTALL_TEST_DIR}/ovc/unit_tests --junitxml=${INSTALL_TEST_DIR}/TEST-OpenVinoConversion.xml

- name: Install Python Layer tests dependencies
run: |
# For torchvision to OpenVINO preprocessing converter
python3 -m pip install -r ${INSTALL_TEST_DIR}/python/preprocess/torchvision/requirements.txt

# layer test requirements
python3 -m pip install -r ${LAYER_TESTS_INSTALL_DIR}/requirements.txt

- name: MO Python API Tests
if: fromJSON(inputs.affected-components).MO.test
run: |
# Import 'test_utils' installed in '<package_test>/tests/python/openvino'
export LD_LIBRARY_PATH=${PIP_INSTALL_PATH}/openvino/libs:$LD_LIBRARY_PATH
export PYTHONPATH=${INSTALL_TEST_DIR}/python

if [[ "${{ runner.os }}" == "Linux" ]] && [[ "${{ runner.arch }}" == "ARM64" ]]; then
# Find gomp lib
GOMP_LIB=$(find "${PIP_INSTALL_PATH}/torch/lib/../../torch.libs/" -name '*libgomp-*so*')
export LD_PRELOAD=${GOMP_LIB}
fi

python3 -m pytest ${LAYER_TESTS_INSTALL_DIR}/mo_python_api_tests -n logical --junitxml=${INSTALL_TEST_DIR}/TEST-test_mo_convert.xml
env:
TEST_DEVICE: CPU
TEST_PRECISION: FP16

- name: OVC Python API Tests
if: fromJSON(inputs.affected-components).MO.test
run: |
Expand All @@ -205,16 +170,6 @@ jobs:
export LD_LIBRARY_PATH=${PIP_INSTALL_PATH}/openvino/libs:$LD_LIBRARY_PATH
python3 -m pytest ${LAYER_TESTS_INSTALL_DIR}/py_frontend_tests --junitxml=${INSTALL_TEST_DIR}/TEST-test_py_fontend.xml

- name: ONNX Layer Tests
if: ${{ fromJSON(inputs.affected-components).ONNX_FE.test }}
run: |
# requires 'unit_tests' from 'tools/mo'
export PYTHONPATH=${INSTALL_TEST_DIR}/mo:$PYTHONPATH
python3 -m pytest ${LAYER_TESTS_INSTALL_DIR}/onnx_tests -m "not launch_only_if_manually_specified and precommit" --junitxml=${INSTALL_TEST_DIR}/TEST-onnx.xml
env:
TEST_DEVICE: CPU
TEST_PRECISION: FP16

- name: JAX Layer Tests - JAX FE
if: ${{ fromJSON(inputs.affected-components).JAX_FE.test && runner.arch != 'ARM64' && runner.os != 'macOS' }}
run: python3 -m pytest ${LAYER_TESTS_INSTALL_DIR}/jax_tests/ -m precommit_jax_fe --junitxml=${INSTALL_TEST_DIR}/TEST-jax_fe.xml
Expand All @@ -230,22 +185,6 @@ jobs:
TEST_DEVICE: CPU
TEST_PRECISION: FP16

- name: TensorFlow 1 Layer Tests - Legacy FE
if: fromJSON(inputs.affected-components).TF_FE.test
run: python3 -m pytest ${{ env.LAYER_TESTS_INSTALL_DIR }}/tensorflow_tests/test_tf_Roll.py --use_legacy_frontend --ir_version=10 --junitxml=${{ env.INSTALL_TEST_DIR }}/TEST-tf_Roll.xml
env:
TEST_DEVICE: CPU
TEST_PRECISION: FP16

- name: TensorFlow 2 Layer Tests - Legacy FE
# no longer workable since TF 2.17
# will be removed in 2024.5
if: ${{ 'false' }}
run: python3 -m pytest ${{ env.LAYER_TESTS_INSTALL_DIR }}/tensorflow2_keras_tests/test_tf2_keras_activation.py --use_legacy_frontend --ir_version=11 -k "sigmoid" --junitxml=${{ env.INSTALL_TEST_DIR }}/TEST-tf2_Activation.xml
env:
TEST_DEVICE: CPU
TEST_PRECISION: FP16

- name: Clone API snippets
if: runner.os != 'macOS'
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
Expand Down
58 changes: 0 additions & 58 deletions .github/workflows/mo.yml

This file was deleted.

44 changes: 6 additions & 38 deletions .github/workflows/windows_vs2019_release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -296,9 +296,6 @@ jobs:
$ovCoreWheelPath=Get-ChildItem -Path . -Filter openvino-*.whl | % { $_.FullName }
python3 -m pip install "$ovCoreWheelPath"

rkazants marked this conversation as resolved.
Show resolved Hide resolved
# Find and install the dev OV wheel
$ovDevWheelPath=Get-ChildItem -Path . -Filter openvino_dev*.whl | % { $_.FullName }
python3 -m pip install "$ovDevWheelPath[caffe,kaldi,onnx,tensorflow2,pytorch]"
working-directory: ${{ env.INSTALL_WHEELS_DIR }}
rkazants marked this conversation as resolved.
Show resolved Hide resolved

- name: Install Python API tests dependencies
Expand All @@ -309,43 +306,26 @@ jobs:
# For torchvision to OpenVINO preprocessing converter
python3 -m pip install -r ${{ env.INSTALL_TEST_DIR }}/python/preprocess/torchvision/requirements.txt

# TODO: replace with Python API tests requirements
python3 -m pip install -r ${{ env.INSTALL_TEST_DIR }}/mo/requirements_dev.txt
rkazants marked this conversation as resolved.
Show resolved Hide resolved
# For validation of Python API
python3 -m pip install -r ${{ env.INSTALL_TEST_DIR }}/bindings/python/requirements_test.txt
rkazants marked this conversation as resolved.
Show resolved Hide resolved

# For getting rid of SSL issues during model downloading for unit tests
python3 -m pip install certifi

- name: Set SSL_CERT_FILE for model downloading for unit tests
run: echo SSL_CERT_FILE=$(python3 -m certifi) >> $env:GITHUB_ENV

rkazants marked this conversation as resolved.
Show resolved Hide resolved
- name: Python API Tests
#if: fromJSON(needs.smart_ci.outputs.affected_components).Python_API.test # Ticket: 127101
shell: cmd
run: |
set PYTHONPATH=${{ env.LAYER_TESTS_INSTALL_DIR }};%PYTHONPATH%
python3 -m pytest -sv ${{ env.INSTALL_TEST_DIR }}/pyopenvino ${{ env.PYTHON_STATIC_ARGS }} --junitxml=${{ env.INSTALL_TEST_DIR }}/TEST-Pyngraph.xml --ignore=${{ env.INSTALL_TEST_DIR }}/pyopenvino/tests/test_utils/test_utils.py

- name: Model Optimizer UT
if: fromJSON(needs.smart_ci.outputs.affected_components).MO.test
shell: cmd
run: |
python3 -m pytest -s ${{ env.INSTALL_TEST_DIR }}/mo/unit_tests --junitxml=${{ env.INSTALL_TEST_DIR }}/TEST-ModelOptimizer.xml

- name: Install Python Layer tests dependencies
run: |
# layer test requirements
python3 -m pip install -r ${{ env.LAYER_TESTS_INSTALL_DIR }}/requirements.txt

- name: ONNX Layer Tests
if: fromJSON(needs.smart_ci.outputs.affected_components).ONNX_FE.test
- name: Python API Tests
#if: fromJSON(needs.smart_ci.outputs.affected_components).Python_API.test # Ticket: 127101
shell: cmd
run: |
:: requires 'unit_tests' from 'tools/mo'
set PYTHONPATH=${{ env.INSTALL_TEST_DIR }}\mo;%PYTHONPATH%
python3 -m pytest ${{ env.LAYER_TESTS_INSTALL_DIR }}/onnx_tests -n logical -m "not launch_only_if_manually_specified and precommit" --junitxml=${INSTALL_TEST_DIR}/TEST-onnx.xml
env:
TEST_DEVICE: CPU
TEST_PRECISION: FP16
set PYTHONPATH=${{ env.LAYER_TESTS_INSTALL_DIR }};%PYTHONPATH%
python3 -m pytest -sv ${{ env.INSTALL_TEST_DIR }}/pyopenvino ${{ env.PYTHON_STATIC_ARGS }} --junitxml=${{ env.INSTALL_TEST_DIR }}/TEST-Pyngraph.xml --ignore=${{ env.INSTALL_TEST_DIR }}/pyopenvino/tests/test_utils/test_utils.py

- name: TensorFlow Lite Layer Tests - TFL FE
if: fromJSON(needs.smart_ci.outputs.affected_components).TFL_FE.test
Expand All @@ -366,18 +346,6 @@ jobs:
--junitxml=${{ env.INSTALL_TEST_DIR }}/TEST-onnx_frontend.xml ^
--ignore=${{ env.INSTALL_TEST_DIR }}/onnx/test_python/test_zoo_models.py

- name: MO Python API Tests
if: fromJSON(needs.smart_ci.outputs.affected_components).MO.test
shell: cmd
run: |
:: Used for 'test_utils' installed in '<test_package>\python\openvino\test_utils'
set PYTHONPATH=${{ env.INSTALL_TEST_DIR }}\python\openvino\test_utils;${{ env.INSTALL_TEST_DIR }}\python;%PYTHONPATH%

python3 -m pytest ${{ env.LAYER_TESTS_INSTALL_DIR }}/mo_python_api_tests --junitxml=${{ env.INSTALL_TEST_DIR }}/TEST-test_mo_convert.xml
env:
TEST_DEVICE: CPU
TEST_PRECISION: FP16

- name: OVC Python API Tests
if: fromJSON(needs.smart_ci.outputs.affected_components).MO.test
shell: cmd
Expand Down
1 change: 1 addition & 0 deletions tests/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ add_subdirectory(e2e_tests)

install(FILES requirements_pytorch DESTINATION tests COMPONENT tests EXCLUDE_FROM_ALL)
install(FILES requirements_tensorflow DESTINATION tests COMPONENT tests EXCLUDE_FROM_ALL)
install(FILES requirements_onnx DESTINATION tests COMPONENT tests EXCLUDE_FROM_ALL)
rkazants marked this conversation as resolved.
Show resolved Hide resolved
3 changes: 2 additions & 1 deletion tests/layer_tests/common/mo_convert_test_class.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,8 @@ def _test(self, temp_dir, test_params, ref_params):
core = Core()

test_params.update({"model_name": 'model_test', "output_dir": temp_dir})
ref_params.update({"model_name": 'model_ref', "output_dir": temp_dir})
ref_output_path = Path(temp_dir, 'model_ref.xml').absolute().as_posix()
ref_params.update({"output_model": ref_output_path})

self.generate_ir_python_api(**test_params)

Expand Down
Loading
Loading