Add OpenVINO NNCF Support (Using --int8 flag)

[kisaragychihaya]

🤖 Generated by Copilot at 2925311

Summary

🚀🔧📝

Added INT8 quantization option for OpenVINO models in export.py. This allows users to reduce the model size and inference latency by quantizing the weights and activations to 8-bit integers.

Sing, O Muse, of the skillful coder who devised
a way to make the models of OpenVINO more wise,
by adding int8 quantization, a noble feat,
that shrinks the size and boosts the speed of export.py.

Walkthrough

• Add support for INT8 quantization for OpenVINO models (link, link, link, link)

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Added INT8 quantization support for OpenVINO exports in the YOLOv5 repository.

📊 Key Changes

• export_openvino function now takes additional parameters: int8, data.
• New quantization process for INT8, including a new dataset generator gen_dataloader and a transformer function transform_fn for preparing input tensors.
• Conditional logic to perform INT8 quantization if the int8 flag is set; otherwise, proceed with the existing model conversion.
• New requirements checked for INT8 quantization: nncf library is now required for this process.

🎯 Purpose & Impact

• Purpose: To improve the performance of YOLOv5 models on devices compatible with OpenVINO by enabling INT8 quantization, which reduces the model size and inference time while maintaining accuracy.
• Impact: Users who rely on OpenVINO for deploying their models can now benefit from the reduced computational requirements of INT8 models, possibly allowing deployment on more resource-constrained devices. Enhanced flexibility and potentially better performance for AI models in production environments. 🚀🧠

[glenn-jocher]

@kisaragychihaya this looks much better!! Thanks for the changes.

[glenn-jocher]

@kisaragychihaya I tested this and it works!! Merging now.

Can you do us a huge favor and also submit a similar PR for YOLOv8 at https://github.com/ultralytics/ultralytics?