Skip to content

Commit

Permalink
[OV2.0] Preprocessing documentation (openvinotoolkit#10451)
Browse files Browse the repository at this point in the history
* [OV2.0] Preprocessing documentation - first draft

* Small update

* Added ov::Layout overview

* Fix code style

* Preprocessing details - ~50% done

* Corrected links

* Fixed comments, added more docs

* Minor updates

* Couple more links

* Fixed comments

* Remove 'future' link
  • Loading branch information
nosovmik authored Feb 21, 2022
1 parent 65d1575 commit f825330
Show file tree
Hide file tree
Showing 14 changed files with 1,176 additions and 52 deletions.
1 change: 1 addition & 0 deletions docs/OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

openvino_docs_IE_DG_Integrate_with_customer_application_new_API
openvino_docs_OV_Runtime_UG_Model_Representation
openvino_docs_OV_Runtime_UG_Preprocessing_Overview
<!-- rename to "Changing input shapes" -->
openvino_docs_IE_DG_ShapeInference
openvino_docs_IE_DG_Device_Plugins
Expand Down
3 changes: 3 additions & 0 deletions docs/OV_Runtime_UG/img/preprocess_not_fit.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
154 changes: 154 additions & 0 deletions docs/OV_Runtime_UG/layout_overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
# Layout API overview {#openvino_docs_OV_Runtime_UG_Layout_Overview}

## Introduction

In few words, with layout `NCHW` it is easier to understand what model's shape `{8, 3, 224, 224}` means. Without layout it is just a 4-dimensional tensor.


Concept of layout helps you (and your application) to understand what does each particular dimension of input/output tensor mean. For example, if your input has shape `{1, 3, 720, 1280}` and layout "NCHW" - it is clear that `N(batch) = 1`, `C(channels) = 3`, `H(height) = 720` and `W(width) = 1280`. Without layout information `{1, 3, 720, 1280}` doesn't give any idea to your application what these number mean and how to resize input image to fit model's expectations.


Reasons when you may want to care about input/output layout:
- Perform model modification:
- Apply [preprocessing](./preprocessing_overview.md) steps, like subtract means, divide by scales, resize image, convert RGB<->BGR
- Set/get batch for a model
- Same operations, used during model conversion phase, see [Model Optimizer model conversion](../MO_DG/prepare_model/convert_model/Converting_Model.md)
- Improve readability of a model's input and output

## Layout syntax

### Short
The easiest way is to fully specify each dimension with one alphabetical letter

@sphinxdirective

.. tab:: C++

.. doxygensnippet:: docs/snippets/ov_layout.cpp
:language: cpp
:fragment: [ov:layout:simple]

.. tab:: Python

.. doxygensnippet:: docs/snippets/ov_layout.py
:language: python
:fragment: [ov:layout:simple]

@endsphinxdirective

This assigns 'N' to first dimension, 'C' to second, 'H' to 3rd and 'W' to 4th

### Advanced
Advanced syntax allows assigning a word to a dimension. To do this, wrap layout with square brackets `[]` and specify each name separated by comma `,`

@sphinxdirective

.. tab:: C++

.. doxygensnippet:: docs/snippets/ov_layout.cpp
:language: cpp
:fragment: [ov:layout:complex]

.. tab:: Python

.. doxygensnippet:: docs/snippets/ov_layout.py
:language: python
:fragment: [ov:layout:complex]

@endsphinxdirective


### Partially defined layout
If some dimension is not important, it's name can be set to `?`

@sphinxdirective

.. tab:: C++

.. doxygensnippet:: docs/snippets/ov_layout.cpp
:language: cpp
:fragment: [ov:layout:partially_defined]

.. tab:: Python

.. doxygensnippet:: docs/snippets/ov_layout.py
:language: python
:fragment: [ov:layout:partially_defined]

@endsphinxdirective


### Dynamic layout
If number of dimensions is not important, ellipsis `...` can be used to specify variadic number of dimensions.

@sphinxdirective

.. tab:: C++

.. doxygensnippet:: docs/snippets/ov_layout.cpp
:language: cpp
:fragment: [ov:layout:dynamic]

.. tab:: Python

.. doxygensnippet:: docs/snippets/ov_layout.py
:language: python
:fragment: [ov:layout:dynamic]

@endsphinxdirective

### Predefined names

Layout has pre-defined some widely used in computer vision dimension names:
- N/Batch - batch size
- C/Channels - channels dimension
- D/Depth - depth
- H/Height - height
- W/Width - width

These names are used in [PreProcessing API](./preprocessing_overview.md) and there is a set of helper functions to get appropriate dimension index from layout

@sphinxdirective

.. tab:: C++

.. doxygensnippet:: docs/snippets/ov_layout.cpp
:language: cpp
:fragment: [ov:layout:predefined]

.. tab:: Python

.. doxygensnippet:: docs/snippets/ov_layout.py
:language: python
:fragment: [ov:layout:predefined]

@endsphinxdirective


### Equality

Layout names are case-insensitive, which means that ```Layout("NCHW") == Layout("nChW") == Layout("[N,c,H,w]")```

### Dump layout

Layout can be converted to string in advanced syntax format. Can be useful for debugging and serialization purposes

@sphinxdirective

.. tab:: C++

.. doxygensnippet:: docs/snippets/ov_layout.cpp
:language: cpp
:fragment: [ov:layout:dump]

.. tab:: Python

.. doxygensnippet:: docs/snippets/ov_layout.py
:language: python
:fragment: [ov:layout:dump]

@endsphinxdirective

## See also

* <code>ov::Layout</code> C++ class documentation
Loading

0 comments on commit f825330

Please sign in to comment.