diff --git a/docs/source/algorithms.mdx b/docs/source/algorithms.mdx index d9db5cb04..65e5567a4 100644 --- a/docs/source/algorithms.mdx +++ b/docs/source/algorithms.mdx @@ -5,7 +5,7 @@ This is an overview of the `bnb.functional` API in `bitsandbytes` that we think ## Using Int8 Matrix Multiplication -For straight Int8 matrix multiplication with mixed precision decomposition you can use ``bnb.matmul(...)``. To enable mixed precision decomposition, use the threshold parameter: +For straight Int8 matrix multiplication without mixed precision decomposition you can use ``bnb.matmul(...)``. To enable mixed precision decomposition, use the threshold parameter: ```py bnb.matmul(..., threshold=6.0) diff --git a/docs/source/installation.mdx b/docs/source/installation.mdx index d1acb2cd6..d846c35a5 100644 --- a/docs/source/installation.mdx +++ b/docs/source/installation.mdx @@ -19,29 +19,30 @@ Welcome to the installation guide for the `bitsandbytes` library! This document ## CUDA[[cuda]] -`bitsandbytes` is currently only supported on CUDA GPUs for CUDA versions **11.0 - 12.5**. However, there's an ongoing multi-backend effort under development, which is currently in alpha. If you're interested in providing feedback or testing, check out [the multi-backend section below](#multi-backend). +`bitsandbytes` is currently only supported on CUDA GPUs for CUDA versions **11.0 - 12.6**. However, there's an ongoing multi-backend effort under development, which is currently in alpha. If you're interested in providing feedback or testing, check out [the multi-backend section below](#multi-backend). ### Supported CUDA Configurations[[cuda-pip]] -The latest version of `bitsandbytes` builds on the following configurations: +The latest version of the distributed `bitsandbytes` package is built with the following configurations: -| **OS** | **CUDA Version** | **Compiler** | +| **OS** | **CUDA Toolkit** | **Host Compiler** | |-------------|------------------|----------------------| | **Linux** | 11.7 - 12.3 | GCC 11.4 | -| | 12.4+ | GCC 13.2 | -| **Windows** | 11.7 - 12.4 | MSVC 19.38+ (VS2022) | +| | 12.4 - 12.6 | GCC 13.2 | +| **Windows** | 11.7 - 12.6 | MSVC 19.42+ (VS2022) | -For Linux systems, ensure your hardware meets the following requirements: +For CUDA systems, ensure your hardware meets the following requirements: -| **Feature** | **Hardware Requirement** | -|---------------------------------|--------------------------------------------------------------------| -| LLM.int8() | NVIDIA Turing (RTX 20 series, T4) or Ampere (RTX 30 series, A4-A100) GPUs | -| 8-bit optimizers/quantization | NVIDIA Kepler (GTX 780 or newer) | +| **Feature** | **Minimum Hardware Requirement** | +|---------------------------------|---------------------------------------------------------------| +| LLM.int8() | NVIDIA Turing (RTX 20 series, T4) or newer GPUs | +| 8-bit optimizers/quantization | NVIDIA Maxwell (GTX 900 series, TITAN X, M40) or newer GPUs * | +| NF4/FP4 quantization | NVIDIA Maxwell (GTX 900 series, TITAN X, M40) or newer GPUs * | > [!WARNING] -> `bitsandbytes >= 0.39.1` no longer includes Kepler binaries in pip installations. This requires [manual compilation using](#cuda-compile) the `cuda11x_nomatmul_kepler` configuration. - -To install from PyPI. +> `bitsandbytes >= 0.45.0` no longer supports Kepler GPUs. +> +> Support for Maxwell GPUs is deprecated and will be removed in a future release. For the best results, a Turing generation device or newer is recommended. ```bash pip install bitsandbytes @@ -79,7 +80,7 @@ For Linux and Windows systems, compiling from source allows you to customize the -To compile from source, you need CMake >= **3.22.1** and Python >= **3.8** installed. Make sure you have a compiler installed to compile C++ (`gcc`, `make`, headers, etc.). +To compile from source, you need CMake >= **3.22.1** and Python >= **3.9** installed. Make sure you have a compiler installed to compile C++ (`gcc`, `make`, headers, etc.). For example, to install a compiler and CMake on Ubuntu: @@ -115,7 +116,7 @@ pip install -e . # `-e` for "editable" install, when developing BNB (otherwise Windows systems require Visual Studio with C++ support as well as an installation of the CUDA SDK. -To compile from source, you need CMake >= **3.22.1** and Python >= **3.8** installed. You should also install CUDA Toolkit by following the [CUDA Installation Guide for Windows](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html) guide from NVIDIA. +To compile from source, you need CMake >= **3.22.1** and Python >= **3.9** installed. You should also install CUDA Toolkit by following the [CUDA Installation Guide for Windows](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html) guide from NVIDIA. Refer to the following table if you're using another CUDA Toolkit version.