Skip to content

Files

Latest commit

 

History

History
20 lines (17 loc) · 2.51 KB

INSTALLATION.md

File metadata and controls

20 lines (17 loc) · 2.51 KB

Installation

On Linux and Windows, AutoGPTQ can be installed through pre-built wheels for specific PyTorch versions:

AutoGPTQ version CUDA/ROCm version Installation Built against PyTorch
latest (0.7.1) CUDA 11.8 pip install auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/cu118/ 2.2.1+cu118
latest (0.7.1) CUDA 12.1 pip install auto-gptq 2.2.1+cu121
latest (0.7.1) ROCm 5.7 pip install auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/rocm571/ 2.2.1+rocm5.7
0.7.0 CUDA 11.8 pip install auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/cu118/ 2.2.0+cu118
0.7.0 CUDA 12.1 pip install auto-gptq 2.2.0+cu121
0.7.0 ROCm 5.7 pip install auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/rocm571/ 2.2.0+rocm5.7
0.6.0 CUDA 11.8 pip install auto-gptq==0.6.0 --extra-index-url https://huggingface.github.io/autogptq-index/whl/cu118/ 2.1.1+cu118
0.6.0 CUDA 12.1 pip install auto-gptq==0.6.0 2.1.1+cu121
0.6.0 ROCm 5.6 pip install auto-gptq==0.6.0 --extra-index-url https://huggingface.github.io/autogptq-index/whl/rocm561/ 2.1.1+rocm5.6
0.5.1 CUDA 11.8 pip install auto-gptq==0.5.1 --extra-index-url https://huggingface.github.io/autogptq-index/whl/cu118/ 2.1.0+cu118
0.5.1 CUDA 12.1 pip install auto-gptq==0.5.1 2.1.0+cu121
0.5.1 ROCm 5.6 pip install auto-gptq==0.5.1 --extra-index-url https://huggingface.github.io/autogptq-index/whl/rocm561/ 2.1.0+rocm5.6

AutoGPTQ is not available on macOS.