Code

FP6 quantization end-to-end.#5234

Code

Merged

loadams merged 33 commits intomastermicrosoft/DeepSpeed:masterfrom features/rebase-quant-fp6microsoft/DeepSpeed:features/rebase-quant-fp6Copy head branch name to clipboardMar 8, 2024

+2,497-6

Commits

Commits on Mar 5, 2024

• FP6 quantization end-to-end.

  Show description for a4562ab
  

  3 peoplecommitted
  a4562ab

• Update CUDA kernels and clean codes.

  JamesTheZ

  committed
  91bb4d7

• Make the quantizer on GPU.

  JamesTheZ

  committed
  1c2131d

• [WIP] Fix the bug of FP16-to-FP6 data packing.

  JamesTheZ

  committed
  1ba45fd

• Add FP6 end-to-end unit tests

  

  arashb

  authored and

  JamesTheZ

  committed
  ff6c3c3

• Refine the FP16-to-FP6 cast logic.

  JamesTheZ

  committed
  368a763

• Add unit tests for FP6 quantizer

  

  arashb

  authored and

  JamesTheZ

  committed
  6c45a84

• Fix FP16-FP6 cast problems.

  JamesTheZ

  committed
  90b710d

• Update FP6 kernels.

  JamesTheZ

  committed
  f8e3acf

• Fix the bug of subnormal FP6 casting and the 2bit/4bit tensor allocation.

  JamesTheZ

  committed
  b025c5a

• Clean code.

  JamesTheZ

  committed
  6ed67f7

• pre-commit

  JamesTheZ

  committed
  20b543c

• Deal with the subnormal FP6 and FP16 values and refine the UT.

  JamesTheZ

  committed
  c43947a

• Update according to review comments.

  JamesTheZ

  committed
  a6d2f2f

• Fix the CI workflow problem for FP6 end-to-end.

  JamesTheZ

  committed
  62a2d49

• Fix at::nullopt and at::optional conflicts.

  JamesTheZ

  committed
  118af37

• Refine split-k setting.

  JamesTheZ

  committed
  56eb8b9

• Remove debug files.

  JamesTheZ

  committed
  0ddbfd1

• Only compiler the kernel body for SM >= 8.0.

  JamesTheZ

  committed
  35c82f2

• Fix the GPU architecture requirement of FP6 kernel.

  JamesTheZ

  committed
  63489d1

• Update deepspeed/inference/v2/config_v2.py

  mrwyattii

  authored
  ed00ac9

• Update deepspeed/inference/v2/config_v2.py

  mrwyattii

  authored
  b15a1a1

• refactor fp6 tests, fix import error

  mrwyattii

  committed
  c2e6ebb

• Update deepspeed/inference/v2/modules/implementations/linear/quantized_linear.py

  mrwyattii

  authored
  fb8887c

• Update requirements.txt

  mrwyattii

  authored
  77f3883

• revert testing to fix A6000 test

  mrwyattii

  authored
  f6bcdee

Commits on Mar 6, 2024

• Update pydantic version

  loadams

  committed
  e1a4ce0

• fix pydantic import

  mrwyattii

  committed
  e86611f

• Fix some review comments.

  JamesTheZ

  committed
  7e28144

• Pin pydantic to latest version

  loadams

  committed
  f8454a0

• Add the missed torch import.

  JamesTheZ

  committed
  bed775e

Commits on Mar 7, 2024

• Merge branch 'master' into features/rebase-quant-fp6

  loadams

  authored
  f34312a

• Merge branch 'master' into features/rebase-quant-fp6

  loadams

  authored
  4a91788