Skip to content

Commit

Permalink
Doc Migration (master) (openvinotoolkit#1377)
Browse files Browse the repository at this point in the history
* Doc Migration from Gitlab (openvinotoolkit#1289)

* doc migration

* fix

* Update FakeQuantize_1.md

* Update performance_benchmarks.md

* Updates graphs for FPGA

* Update performance_benchmarks.md

* Change DL Workbench structure (#1)

* Changed DL Workbench structure

* Fixed tags

* fixes

* Update ie_docs.xml

* Update performance_benchmarks_faq.md

* Fixes in DL Workbench layout

* Fixes for CVS-31290

* [DL Workbench] Minor correction

* Fix for CVS-30955

* Added nGraph deprecation notice as requested by Zoe

* fix broken links in api doxy layouts

* CVS-31131 fixes

* Additional fixes

* Fixed POT TOC

* Update PAC_Configure.md

PAC DCP 1.2.1 install guide.

* Update inference_engine_intro.md

* fix broken link

* Update opset.md

* fix

* added opset4 to layout

* added new opsets to layout, set labels for them

* Update VisionAcceleratorFPGA_Configure.md

Updated from 2020.3 to 2020.4

Co-authored-by: domi2000 <[email protected]>
  • Loading branch information
ntyukaev and Domi2000 authored Jul 20, 2020
1 parent 4037613 commit 6572d24
Show file tree
Hide file tree
Showing 532 changed files with 30,454 additions and 313 deletions.
6 changes: 6 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -63,3 +63,9 @@
#*.PDF diff=astextplain
#*.rtf diff=astextplain
#*.RTF diff=astextplain

*.PNG filter=lfs diff=lfs merge=lfs -text
*.png filter=lfs diff=lfs merge=lfs -text
*.jpg filter=lfs diff=lfs merge=lfs -text
*.gif filter=lfs diff=lfs merge=lfs -text
*.vsdx filter=lfs diff=lfs merge=lfs -text
212 changes: 212 additions & 0 deletions docs/HOWTO/Custom_Layers_Guide.md

Large diffs are not rendered by default.

83 changes: 83 additions & 0 deletions docs/HOWTO/add_regression_test_vpu.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Regression tests howto {#openvino_docs_HOWTO_add_regression_test_vpu}

## Purpose

This document contains instructions for correctly modifying a set of regression tests.

## Common

Regression tests for Myriad and HDDL plugins are on the path:
`inference-engine/tests/functional/vpu/regression_tests/`

The tests are divided into 4 groups:
* Classification
* Detection
* Raw-results
* Compilation
* VPU hetero

Testing framework – [Google Test](https://github.com/google/googletest/).
Each group contains [parameterized](https://github.com/google/googletest/blob/master/googletest/docs/advanced.md) tests. The main idea is that to add a new test, you only need to add a new parameter. Except for scenarios different from the generalized case.

## Classsification and Detection tests

These groups contains two cases:

* For generalized scenario (` VpuNoClassificationRegression, VpuNoDetectionRegression`)
* For specific scenario (` VpuNoClassificationRegressionSpecific, VpuNoDetectionRegressionSpecific`)

### Generalized scenario

If You want test new parameter(batch, precision, model and etc.) then You need to edit the existing initialization of parameterized tests or create a new one.
Example of initialization of parameterized tests:

``` c++
INSTANTIATE_TEST_CASE_P(
VPURegTestWithResources_nightly,
VpuNoClassificationRegression,
Combine(ValuesIn(VpuTestParamsContainer::testingPlugin()),
Values(Precision::FP16),
Values(1), // batches
Values(true), //IsHwAdaptiveMode
Values(false), //DoReshape
Values(3, 5, 7), //Resources
Values(false), //IsIgnoreStatistic
Values(ClassificationSrcParam{ModelName::GoogleNetV1, SourceImages::kCat3, 0.01, Regression::EMean::eValues})),
VpuNoClassificationRegression::getTestCaseName);
```
### Specific scenario
If You need a test to perform some actions that are not provided in the generalized scenario, then add a specific test case. As with the generalized scenario You can change parameters for these tests.
Example of specific test case:
``` c++
TEST_P(VpuNoClassificationRegressionSpecific, onAlexNetWithNetworkConfig) {
DISABLE_ON_WINDOWS_IF(HDDL_PLUGIN);
DISABLE_IF(do_reshape_);
if (!hw_adaptive_mode_) {
config_[VPU_CONFIG_KEY(NETWORK_CONFIG)] = "data=data,scale=1";
}
assertThat().classificationResultsForInferRequestAPI()
.on(SourceImages::kDog2)
.withInputPrecision(in_precision_)
.times(batch_)
.withBatch(batch_)
.onModel(ModelName::AlexNet)
.setMean(Regression::EMean::eImage)
.onFP16()
.withTopK(1)
.withPluginConfig(config_)
.equalToReferenceWithDelta(0.04);
}
```

## Raw-results tests

There is no generalized scenario and recommendations are the same as for specific test cases for Classification/Detection groups.

## Compilation tests

The tests are in the `vpu_classification_regression.cpp` file and contains only one scenario ` VpuNoRegressionWithCompilation `. To add a new test just update parameters just as in generalized scenarion of Classification/Detection test groups.
94 changes: 94 additions & 0 deletions docs/HOWTO/fuzzing-HOWTO.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# Fuzzing howto {#openvino_docs_HOWTO_fuzzing_HOWTO}

## Intended Audience

This document is for a developer who wants to contribute fuzz tests.

## Purpose

This document walks you through creating your first fuzzer, running it and evaluating its quality.

## Prerequisites

- Linux OS or Mac OS.

- [American Fuzzy Loop](http://lcamtuf.coredump.cx/afl/) if building with GCC.

## Steps

1. Create a fuzz test in the existing project at `./tests/fuzz`. Fuzz test must
follow `<test name>-fuzzer.cc` naming scheme and implement a
`LLVMFuzzerTestOneInput` entry point.

``` bash
cat << EOF > ./tests/fuzz/test_name-fuzzer.cc
#include <stdint.h>
#include <cstdlib>
extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) {
// put your fuzzing code here and use data+size as input.
return 0; // always return 0
}
EOF
```

2. Implement test logic under `LLVMFuzzerTestOneInput`.

See example fuzz test at `tests/fuzz/read_network-fuzzer.cc`.

3. Build fuzz tests with `-DENABLE_FUZZING=ON` flag for cmake.

``` bash
mkdir -p build && \
(cd build && \
CXX=afl-g++ CC=afl-gcc cmake -DCMAKE_BUILD_TYPE=Debug -DENABLE_FUZZING=ON -DENABLE_TESTS=ON .. && \
make fuzz --jobs=$(getconf _NPROCESSORS_ONLN))
```

4. Prepare sample inputs for your fuzz test to teach fuzzer engine on input
structure

``` bash
(cd bin/intel64/Debug && \
mkdir test_name-corpus && \
echo sample input > test_name-corpus/in1.txt)
```

5. Evaluate fuzz test with `afl-fuzz` fuzzing engine

Run fuzz test:

``` bash
(cd bin/intel64/Debug && \
afl-fuzz -i test_name-corpus -o test_name-out -- ./test_name-fuzzer @@
```
While fuzz test is running it prints out statistics. Besides just crashes `uniq
crashes` and hangs `uniq hangs` you should care about fuzz test quality:
- Fuzz test should be fast - speed of execution `exec speed` should be at least
100 exec/s. Speed less than 20 exec/s is not acceptable.
- Fuzz test should be able to explore new code paths `map coverage` and
`findings in depth`. Confirm it is increasing while fuzz test is running.
6. Reproduce fuzz test findings
All issues found by fuzz test are stored as a file in output folder specified
earlier via `-o` afl-fuzz option. To reproduce an issue run fuzz test executable
with an issue file as an argument.
## Summary
We have created a simple fuzz test, run it and asses its results.
## Extension
Try run parallel fuzzing with the help of
[afl-utils](https://gitlab.com/rc0r/afl-utils).
## Tips or FAQs
GCC 7 in Ubuntu 18.04 LTS has a
[defect](https://bugs.launchpad.net/ubuntu/+source/afl/+bug/1774816). Upgrade
GCC 7 for AFL to work. GCC version `Ubuntu 7.3.0-27ubuntu1~18.04` works OK.
3 changes: 3 additions & 0 deletions docs/HOWTO/img/IE_extensions_flow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions docs/HOWTO/img/MEG_generic_flow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions docs/HOWTO/img/MO_extensions_flow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions docs/HOWTO/img/mo_caffe_priorities.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 6572d24

Please sign in to comment.