Skip to content

Commit

Permalink
Add export command for compressed VLMs (openvinotoolkit#1218)
Browse files Browse the repository at this point in the history
Added a command for exporting compressed VLMs similar to how it is done
for LMs.
  • Loading branch information
nikita-savelyevv authored and ilya-lavrenov committed Nov 20, 2024
1 parent 1b17dcb commit f9eca94
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,11 @@ For more examples check out our [LLM Inference Guide](https://docs.openvino.ai/2
### Converting and compressing the model from Hugging Face library
```sh
optimum-cli export openvino --model openbmb/MiniCPM-V-2_6 --trust-remote-code MiniCPM-V-2_6
#(Basic) download and convert to OpenVINO MiniCPM-V-2_6 model
optimum-cli export openvino --model openbmb/MiniCPM-V-2_6 --trust-remote-code --weight-format fp16 MiniCPM-V-2_6
#(Recommended) Same as above but with compression: language model is compressed to int4, other model components are compressed to int8
optimum-cli export openvino --model openbmb/MiniCPM-V-2_6 --trust-remote-code --weight-format int4 MiniCPM-V-2_6
```

### Run generation using VLMPipeline API in Python
Expand Down

0 comments on commit f9eca94

Please sign in to comment.