Code

DPO cleanup#1126

Code

Merged

winglian merged 23 commits intomainaxolotl-ai-cloud/axolotl:mainfrom dpo-cleanupaxolotl-ai-cloud/axolotl:dpo-cleanupCopy head branch name to clipboardJan 23, 2024

+440-106

Commits

Commits on Jan 23, 2024

• cleanup dpo to be a little more extensible, add zephyr/nectar strategy

  winglian

  committed
  6a23720

  Copy full SHA for 6a23720

• fix eos slash

  winglian

  committed
  2f7242d

  Copy full SHA for 2f7242d

• support for eval split

  winglian

  committed
  74cbe2a

  Copy full SHA for 74cbe2a

• fix kwargs

  winglian

  committed
  1f28327

  Copy full SHA for 1f28327

• handle empty evals

  winglian

  committed
  cb2f774

  Copy full SHA for cb2f774

• don't load peft model for dpo

  winglian

  committed
  a91e0cb

  Copy full SHA for a91e0cb

• ensure dpo traning args gets bf16 for peft if applicable

  winglian

  committed
  bcae290

  Copy full SHA for bcae290

• fix duplicate kwargs for bf16

  winglian

  committed
  a24c756

  Copy full SHA for a24c756

• make sure to respect the configured lr scheduler

  winglian

  committed
  d5e12dd

  Copy full SHA for d5e12dd

• supprt trainer callback to push config to wandb

  winglian

  committed
  9a746fa

  Copy full SHA for 9a746fa

• set dataloader preload args

  winglian

  committed
  60f566c

  Copy full SHA for 60f566c

• ensure that we are loading the lora when merging

  winglian

  committed
  c41391d

  Copy full SHA for c41391d

• Update src/axolotl/utils/data.py

  Show description for 1cfd179
  

  winglian

  and

  plaguss

  committed
  1cfd179

  Copy full SHA for 1cfd179

• support local datasets for dpo

  Show description for 02fc8f9
  

  winglian

  and

  plaguss

  committed
  02fc8f9

  Copy full SHA for 02fc8f9

• chore: lint

  winglian

  committed
  7141fd1

  Copy full SHA for 7141fd1

• dpo/kto/ipo smoke tests w lora, simplify dpo dataset type names

  winglian

  committed
  44a6f2d

  Copy full SHA for 44a6f2d

• add split to dpo tests

  winglian

  committed
  7f3b7ce

  Copy full SHA for 7f3b7ce

• fix rebase/merging error

  winglian

  committed
  52a227d

  Copy full SHA for 52a227d

• handle edge case w logging

  winglian

  committed
  064b20e

  Copy full SHA for 064b20e

• use accelerator for dpo datasets so it doesn't break the logger

  winglian

  committed
  c49315f

  Copy full SHA for c49315f

• missing args

  winglian

  committed
  72fb877

  Copy full SHA for 72fb877

• validate checkpoint is an adapter for now

  winglian

  committed
  29663d8

  Copy full SHA for 29663d8

• log warning when dataset strategy is not loadable

  winglian

  committed
  76b5c2d

  Copy full SHA for 76b5c2d