This module contains a PyTorch*-based framework and samples for neural networks compression. The framework is organized as a Python* module that can be built and used in a standalone mode. The framework architecture is unified to make it easy to add different compression methods. The samples demonstrate the usage of compression algorithms for three different use cases on public models and datasets: Image Classification, Object Detection and Semantic Segmentation.
- Support of quantization, binarization, and sparsity algorithms with fine-tuning
- Automatic model graph transformation. The model is wrapped by the custom class and additional layers are inserted in the graph. The transformations are configurable.
- Common interface for compression methods
- GPU-accelerated layers for fast model fine-tuning
- Distributed training support
- Configuration file examples for sparsity, quantization, and sparsity with quantization. Each type of compression requires only one additional fine-tuning stage.
- Export models to ONNX* format that is supported by the OpenVINO™ toolkit.
For more details about framework architecture, refer to the NNCF description.
The NNCF can be used in two different ways:
- A standalone package that can be installed and integrated into a custom training pipeline. For example, the NNCF can be used with the mmdetection pip package into the Python environment. For more information about NNCF standalone usage, refer to this manual.
- Training sample that demonstrates model compression capabilities
Any of these two options implies providing a configuration file which contains hyperparameters of training and compression algorithms. Each of compression samples contains examples of configuration files which implements compression of standard DL models. For more information about configuration files please refer to the corresponding manual file.
To run the samples,refer to the corresponding tutorials:
- Ubuntu* 16.04 or later (64-bit)
- Python* 3.5 or later
- NVidia CUDA* Toolkit 9.0 or later
- PyTorch* 1.0 or higher.
We suggest to install or use the package in the Python virtual environment.
- Install the following dependencies:
sudo apt-get install python3-dev - Activate environment and install the project dependencies running
pip install -r requirements.txt - Use as a standalone package:
- Clone repository
- In the project folder, run
python setup.py bdist_wheelto build package - Install package running
pip install dist/nncf-<vesion>.whl
- Use project samples
- Clone repository
- In the project folder run
python setup.py developto install NNCF in your environment
| Model | Dataset | PyTorch FP32 baseline | PyTorch compressed accuracy |
|---|---|---|---|
| ResNet-50 INT8 quantized | ImageNet | 76.13 | 76.49 |
| ResNet-50 INT8 w/ 60% of sparsity (RB) | ImageNet | 76.13 | 75.2 |
| Inception v3 INT8 quantized | ImageNet | 77.32 | 78.06 |
| Inception v3 INT8 w/ 60% of sparsity (RB) | ImageNet | 77.32 | 76.8 |
| MobileNet v2 INT8 quantized | ImageNet | 71.8 | 71.3 |
| MobileNet v2 INT8 w/ 51% of sparsity (RB) | ImageNet | 71.8 | 70.9 |
| SqueezeNet v1.1 INT8 quantized | ImageNet | 58.19 | 57.9 |
| ResNet-18 XNOR binarization | ImageNet | 69.75 | 61.71 |
| SSD300-BN INT8 quantized | VOC12+07 | 78.28 | 78.22 |
| SSD300-BN INT8 w/ 70% of sparsity (Magnitude) | VOC12+07 | 78.28 | 77.94 |
| SSD512-BN INT8 quantized | VOC12+07 | 80.26 | 80.48 |
| SSD512-BN INT8 w/ 70% of sparsity (Magnitude) | VOC12+07 | 80.26 | 80.11 |
| UNet INT8 quantized | CamVid | 71.95 | 71.82 |
| UNet INT8 w/ 60% of sparsity (Magnitude) | CamVid | 71.95 | 71.90 |
| UNet INT8 quantized | Mapillary | 56.23 | 56.16 |
| UNet INT8 w/ 60% of sparsity (Magnitude) | Mapillary | 56.23 | 54.30 |
| ICNet INT8 quantized | CamVid | 67.89 | 67.69 |
| ICNet INT8 w/ 60% of sparsity (Magnitude) | CamVid | 67.89 | 67.53 |