Code

Peft deepspeed resume#1227

Code

Merged

winglian merged 12 commits intomainaxolotl-ai-cloud/axolotl:mainfrom peft-deepspeed-resumeaxolotl-ai-cloud/axolotl:peft-deepspeed-resumeCopy head branch name to clipboardJan 31, 2024

+35-26

Commits

Commits on Jan 31, 2024

• import deepspeed integration

  winglian

  committed
  327136c

  Copy full SHA for 327136c

• monkeypatch peft adapater with deepspeed for resume from checkpoint

  winglian

  committed
  7257274

  Copy full SHA for 7257274

• fix patch

  winglian

  committed
  cb1ac04

  Copy full SHA for cb1ac04

• fix patches attempt 2

  winglian

  committed
  ab67fd6

  Copy full SHA for ab67fd6

• make sure to set lora_model_dir

  winglian

  committed
  0a123b5

  Copy full SHA for 0a123b5

• skip pylint for deepspeed.utils

  winglian

  committed
  8d5d5dd

  Copy full SHA for 8d5d5dd

• pick up upstream fix in transformers

  winglian

  committed
  839637c

  Copy full SHA for 839637c

• remove monkeypatch for deepspeed/peft fix

  winglian

  committed
  e58f030

  Copy full SHA for e58f030

• no need to set the lora_model_dir on resume

  winglian

  committed
  8882d32

  Copy full SHA for 8882d32

• unset load_in_*bit when using quant config

  winglian

  committed
  5b866a2

  Copy full SHA for 5b866a2

• guard before del

  winglian

  committed
  3fa8746

  Copy full SHA for 3fa8746

• better handling of load_in* kwargs

  winglian

  committed
  1cf6245

  Copy full SHA for 1cf6245