Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOCS][Layer Tests] Update layer test documentation #23485

Merged
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 29 additions & 27 deletions tests/layer_tests/README.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,36 @@
# Layer tests
# Layer Tests

This folder layer tests framework code and test files.
The layer tests primarily aim to validate support for PyTorch, TensorFlow, TensorFlow Lite, and ONNX frameworks' operations by OpenVINO.
The test pipeline includes the following steps:
1. Creation of a model with the tested operation using framework API
2. Conversion of the created model using OpenVINO's `convert_model` method
rkazants marked this conversation as resolved.
Show resolved Hide resolved
3. Inference of both the original and converted models using the framework and OpenVINO on random input data
4. Checking whether the inference results from OpenVINO and the framework are the same or different within a tolerance threshold

## Getting Started
## Setup Environment

#### Pre-requisites

* OpenVINO should be configured as usual.

#### Setup

* Install requirements:
```bash
pip3 install -r requirements.txt
To set up the environment for launching layer tests, perform the following steps:
1. Install the OpenVINO wheel package. If you're testing changes in OpenVINO, build your local wheel package for installation.
Find instructions on how to build on [wiki page](https://github.com/openvinotoolkit/openvino/blob/master/docs/dev/build.md).
```sh
pip install openvino.whl
```
* Set up environment variables for layer tests (if you use wheel package path to python api could be removed):
```bash
export PYTHONPATH="path_to_openvino"/tests/layer_tests/:"path_to_openvino"/tools/mo:"path to python api"
2. (Optional) Install the OpenVINO Tokenizers wheel package if you're testing the support of operations using conversion and operation extensions from OpenVINO Tokenizers.
If you're testing changes in OpenVINO Tokenizers, build your local wheel package for installation.
Find instructions on how to build on [GitHub page](https://github.com/openvinotoolkit/openvino_tokenizers)
```sh
pip install openvino_tokenizers.whl
```
* If there is need to use specific libs it is possible to specify path to them using OV_LIBRARY_PATH env variable
```bash
export OV_LIBRARY_PATH="path_to_libs"
3. Install requirements for running layer tests.
```sh
cd tests/layer_tests
pip install -r requirements.txt
```
* To parametrize tests by device and precision (optional)
```bash
export TEST_DEVICE="CPU;GPU"
export TEST_PRECISION="FP32;FP16"
```

mvafin marked this conversation as resolved.
Show resolved Hide resolved
## Run tests
```bash
py.test
```
## Run Tests

Example to run the TensorFlow layer test for the `tf.raw_ops.Unique` operation on CPU:
```sh
rkazants marked this conversation as resolved.
Show resolved Hide resolved
export TEST_DEVICE="CPU"
pytest tensorflow_tests/test_tf_Unique.py
```
Loading